DGX 2 with 16 V100

Created by: Skyy93

What is your question?

We have a DGX 2 with 16 V100 32GB and would like to run the 175B model from OPT. How can is reshard a consolidated .pt to 16 shard to overcome the memory limit?

What have you tried?

Loading the model on 8 graphic cards but this results in OOM errors