This project generates segmentation prompts for radiology tasks based on clinical notes. Given a clinical note, the system produces structured prompts to assist segmentation in medical imaging.
- Synthetic clinical notes with paired segmentation prompts generated via ChatGPT.
- Columns:
id,note,organ,diagnosis,stage,prompt. - For demo purposes only; not for clinical use.
- Fine-tuned a text-to-text model to generate prompts directly from notes.
- Limited success due to small dataset; outputs often generic or incomplete.
- Encoded notes using
all-MiniLM-L6-v2sentence embeddings. - Used cosine similarity to retrieve closest prompts from dataset.
- Delivered reliable results without extensive training.
| Technique | Dataset Size Requirement | Training Time | Performance on Small Data | Complexity | Suitability for This Project |
|---|---|---|---|---|---|
| Fine-tuning FLAN-T5 | Large | High | Poor | High | Not ideal due to limited data |
| Retrieval-Augmented Gen. | Small to Medium | Low | Good | Medium | Practical, effective for this use |
- Fine-tuning large language models demands substantial, diverse datasets to achieve quality output.
- Retrieval-based methods using embeddings and similarity search provide a strong baseline for small datasets without complex training.
- Working with biomedical text requires careful dataset preparation and clear prompt design to guide models effectively.
- Practical implementation challenges include managing tensor vs numpy data types and Hugging Face’s tokenization intricacies.
- Experimentation and flexibility are crucial—switching methods when one doesn’t fit is part of research and engineering.
- Python 3.8+
- transformers
- sentence-transformers
- datasets
- scikit-learn
- huggingface_hub
Install dependencies with:
pip install -r requirements.txt