Hello, I have a question while running the Biomedical QA example.
I downloaded the trained model, finished the setup, and tried to run it on the server, but I ran into an out-of-memory error.
In the paper, it says that they used 4 A100 GPUs for the experiments, so I also tried to increase the number of GPUs. However, the current model does not seem to support GPU parallelization, so even if I put 2 GPUs in CUDA_VISIBLE_DEVICES, it only uses 1 GPU. Since that is insufficient, I keep getting out-of-memory errors.
How am I supposed to use this properly?
Right now, I’m following examples/biomedgpt_r1.ipynb and running the code below.
from open_biomed.models.foundation_models.biomedgpt import BioMedGPTR14Chat
from open_biomed.data import Protein, Text
base_dir = os.path.dirname(os.path.abspath(file))
ckpt_dir = os.path.join(base_dir, "checkpoints")
for sub in ["protein"]:
p = os.path.join(ckpt_dir, sub)
assert os.path.isdir(p), f"Missing folder: {p}"
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
agent = BioMedGPTR14Chat.from_pretrained(ckpt_dir, device)
agent.reset()
protein_seq = "MDCVLRSYLLLAFGFLICLFLFCLVVFIWFVYKQILFRTTAQSNEARHNHSTVV"
agent.append_protein(Protein.from_fasta(protein_seq))
q = "There is a mutation from G to A at position 14. What is the mutational effects?"
ans = agent.chat(Text.from_str(q))
print("Human:", q)
print("Assistant:", ans)
Hello, I have a question while running the Biomedical QA example.
I downloaded the trained model, finished the setup, and tried to run it on the server, but I ran into an out-of-memory error.
In the paper, it says that they used 4 A100 GPUs for the experiments, so I also tried to increase the number of GPUs. However, the current model does not seem to support GPU parallelization, so even if I put 2 GPUs in CUDA_VISIBLE_DEVICES, it only uses 1 GPU. Since that is insufficient, I keep getting out-of-memory errors.
How am I supposed to use this properly?
Right now, I’m following examples/biomedgpt_r1.ipynb and running the code below.
from open_biomed.models.foundation_models.biomedgpt import BioMedGPTR14Chat
from open_biomed.data import Protein, Text
base_dir = os.path.dirname(os.path.abspath(file))
ckpt_dir = os.path.join(base_dir, "checkpoints")
for sub in ["protein"]:
p = os.path.join(ckpt_dir, sub)
assert os.path.isdir(p), f"Missing folder: {p}"
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
agent = BioMedGPTR14Chat.from_pretrained(ckpt_dir, device)
agent.reset()
protein_seq = "MDCVLRSYLLLAFGFLICLFLFCLVVFIWFVYKQILFRTTAQSNEARHNHSTVV"
agent.append_protein(Protein.from_fasta(protein_seq))
q = "There is a mutation from G to A at position 14. What is the mutational effects?"
ans = agent.chat(Text.from_str(q))
print("Human:", q)
print("Assistant:", ans)