How to load pretrained weight in finetuning?

Created by: Dod-o

❓ Questions and Help

Hi team, I want to fine-tune OPT on my own dataset using code in metaseq. I want to fine-tune the 30B opt model so I guess hf is not a good choice.

Is there any way to continue training on the pre-trained model easily?

Thanks