Missing dict.txt?

Created by: hunterlang

Edit by Admin: If you see issues with the gpt2 tokenizer, please refer to #132

---

Hi, I've gotten to the point of running metaseq-api-local but I'm having issues figuring out where to find the missing stuff in constants.py.

I gathered I need gpt2-merges.txt and gpt2-vocab.json, so I downloaded those from here. But based on the example ls output in constants.py, I also need a dict.txt, and when I run metaseq-api-local I get

File "/home/hlang/metaseq/metaseq/data/dictionary.py", line 235, in add_from_file
    with open(PathManager.get_local_path(f), "r", encoding="utf-8") as fd:
FileNotFoundError: [Errno 2] No such file or directory: '/home/hlang/opt_models/dict.txt'

Here I have MODEL_SHARED_FOLDER='/home/hlang/opt_models/'.

Is there a place to download the dict.txt for each model?