Repository navigation
Fine-tuning - Problem #10
Description
Activity
Oh, I didn’t freeze the encoder. I’ll try again.
One more question: for example, if I’m performing bile duct segmentation—a task that this model doesn’t currently support—can I keep the encoder frozen?I need to segment: the liver, gallbladder, ribs, and liver lesions,
as well as the hepatic vessels and bile ducts.
So, I should first freeze the encoder and then fine-tune each organ one by one, minimizing the number of fine-tuning iterations as much as possible, right?You should not finetune one by one. There are two ways: 1) Combine them into one single dataset then use the script provided in thie repo.
You just need to combine all dataset into one and create a label mapping.
https://github.com/NVIDIA-Medtech/NV-Segment-CTMR/blob/main/NV-Segment-CTMR/docs/finetune.md.
So by converting your dataset, you map liver to 1, gallbladder to 2, ribs to 3 e.t.c. The reason for this is to differentiate those classes when combining into a single dataset. then you create a global mapping list using the existing definition https://github.com/NVIDIA-Medtech/NV-Segment-CTMR/blob/main/NV-Segment-CTMR/configs/metadata.json. The reason for this is to reuse the class definition from pretrained checkpoint.
[1,1 (this 1 is the liver in NV-Segment-CTMR) ], [2, 10 (10 is the gallblader)], [3, 63 (left rib1, you can use any number because there is no single rib label, we can just overide it)].2). This model is trained with a huge collection of individual datasets, the training script is here(https://github.com/Project-MONAI/VISTA/blob/main/vista3d/scripts/train.py). You can try to simplify it or just use the vibe coding to write a script to finetune. Just load the model and then finetune using the ai generated finetuning codes.
from monai.networks.nets.vista3d import vista3d132 vista3d132.load_state_dict(pretrained_ckpt, strict=True)In other words:
Method 1, I must have a multi label dataset
Method 2, I can use multiple different binary classification datasets
Right?Method 2, based on your suggestions, Claude Code has indeed generated multiple fine-tuning methods for binary datasets for me. Thank you very much
@YY2968218397 we just put up a much better CTMR joint model yesterday. The same link. You can try that one.
hf download nvidia/NV-Segment-CTMR --local-dir models/ && \ mv models/vista3d_pretrained_model/model.pt models/model.pt
I have two dataset，hepaticTumor and hepaticVessel,both are binary label。
First,I finetuning hepaticTumor, starting with model.pt, the performance in hepaticTumor is good,save the hepaticTumor.pt;
Second, I finetuning the hepaticVessel,starting with the hepaticTumor.pt, the performance in hepaticVessel is good,save the hepaticVessel.pt;
Last, I use hepaticVessel.pt, evaluate the performance in hepaticTumor,the dice change,become low.
Do you have any suggestion?How you finetuning all the labels?One by one?