gpt-oss-20b-FineTuning

Fine-tuning workflow for an open-source LLM (20B-class) using LoRA with Hugging Face Transformers + TRL. Includes a ready-to-run notebook (FineTunning.ipynb) and a sample training set (train_final.json).

---

✨ What’s inside

• LoRA fine-tuning (parameter-efficient) via TRL SFTTrainer
• Notebook workflow for step-by-step runs
• JSON/JSONL dataset format for chat/instruction data
• Packed sequences for efficient long-context training
• Exportable adapters (merge or serve as LoRA)
• Quick inference snippets (Transformers + vLLM)

---

🧰 Requirements

• Python 3.10+

• Linux or WSL recommended

• GPU: 24GB+ VRAM (LoRA works on consumer GPUs; more VRAM = larger batch)

• Packages:

  pip install "transformers==4.42.0" "trl==0.9.6" "peft==0.11.1" \
              "bitsandbytes==0.43.1" "accelerate==0.31.0" "datasets==2.19.0" \
              "evaluate" "scikit-learn"

---

📁 Repository structure

.
├── FineTunning.ipynb      # Step-by-step LoRA fine-tuning notebook
├── train_final.json       # Sample training data (instruction/chat style)
└── README.md

The notebook and dataset are already in the repo. ([GitHub][1])

---

📦 Data format

Use JSON or JSONL with chat messages (recommended). Minimal example (one training example per line if JSONL):

{
  "system": "You are a helpful assistant.",
  "messages": [
    {"role": "user", "content": "Explain batch normalization simply."},
    {"role": "assistant", "content": "Batch norm normalizes activations..."}
  ]
}

If your file is a plain JSON array (train_final.json), it should contain objects like the example above. You can add any number of items.

Tips

• Remove duplicates and near-duplicates.
• Keep responses clean (no PII, consistent tone).
• 5k–50k high-quality pairs usually fine for SFT.

---

🧩 Inference

Transformers (with LoRA adapters)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Dhruvil03/gpt-oss-20b-Dhruvil"      # merged weights

tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.padding_side = "left"
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype="auto", device_map="auto"
)

messages = [
    {"role": "user", "content": "Is naturopathy as effective as conventional therapy for treatment of menopausal symptoms?"},
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
                                      return_tensors="pt").to(model.device)
attn = (inputs != tokenizer.pad_token_id).long()

out = model.generate(inputs, attention_mask=attn,
                     max_new_tokens=512, do_sample=True, temperature=0.6)
print(tokenizer.decode(out[0], skip_special_tokens=True))

---

🔧 Tuning knobs (good defaults)

• LR: 1e-4 (LoRA adapters)
• Epochs: 2–3 with early stop if val loss plateaus
• LoRA: r=16–32, alpha≈2×r, dropout=0.05–0.1
• Grad steps: increase gradient_accumulation_steps if VRAM is tight
• Max length: match model context (4k/8k). Enable packing=True

---

❓ FAQ

Q: Can I fine-tune on a single GPU?
Yes—LoRA is designed for this. Reduce per-device batch size and increase gradient accumulation if you OOM.

Q: Do I need to merge adapters?
No. You can serve with adapters or merge for single-weight export (easier with vLLM/AWQ/GPTQ).

Q: Does train_final.json have to be JSONL?
No. JSON arrays work with the loader above; JSONL is better for large datasets.

---

🙏 Acknowledgements

• Hugging Face Transformers
• TRL
• PEFT
• bitsandbytes