GSM8K-Teacher-CoT-120B (2025 Release)

High-Quality Short Chain-of-Thought Distillation Dataset Plain-Text • No LaTeX • No ChatML • Deterministic Answers

This repository contains a clean, plain-text Chain-of-Thought (CoT) dataset distilled from OpenAI: gpt-oss-120b using a strict short-CoT prompt. The dataset targets small reasoning models (7B–14B) such as Granite 4.0H, and follows the 2025 CoT distillation standard:

• no LaTeX
• no ChatML or markup
• no XML / markdown tokens
• short, deterministic reasoning
• final answers explicitly normalized

---

Dataset Overview

• File: gsm8k_teacher_cot.jsonl
• Samples: ~7.2k (full GSM8K train)
• Format: JSONL
• Teacher model: OpenAI — gpt-oss-120b
• Style: Short CoT, strict formatting
• Domain: Grade-school math problems
• Intended use: Training & distillation for small reasoning models

---

Dataset Format

Each line is a JSON object with three fields:

{
  "question": "...",
  "cot": "...",
  "final_answer": "72"
}

Field descriptions

• question Original GSM8K problem, plain text.

• cot Short, concise reasoning generated by the 120B teacher. No LaTeX, no special formatting.

• final_answer A normalized string containing the final numeric answer.

---

Quality & Style Guarantees

✔ Short structured CoT

Always 1–6 sentences, deterministic.

✔ No formatting artifacts

No LaTeX, no $...$, no markdown, no ChatML tags, no XML.

✔ Deterministic output

final_answer always present, always a number-as-string.

✔ Teacher reproducibility (OpenAI: gpt-oss-120b)

• temperature: 0.2
• top-p: 1.0
• max_tokens: 512
• prompting: strict short-CoT instructions
• output constraints: no LaTeX, no special tokens, plain text only

---

Intended Use

This dataset is optimized for:

• Reasoning specialization for small models (7B–14B)
• Supervised fine-tuning (SFT)
• Distillation from a 120B teacher
• Math reasoning tasks
• Research on compact CoT formats

Compatible with:

• Hugging Face datasets
• Axolotl
• OpenCompass
• LigerTune / ReFT pipelines
• Any SFT trainer using JSONL input

---

Loading Example

from datasets import load_dataset

ds = load_dataset("your-username/gsm8k-teacher-cot-120b")
print(ds["train"][0])

---

Disclaimer

• The chain-of-thought (cot) is LLM-generated, not human-authored.
• GSM8K problems are © their original authors; only model-generated reasoning is included.
• Do not treat the CoT as verified mathematics—this dataset is for training purposes only.

---

Motivation

This dataset was created as part of a 2025 project to train Granite 4.0H (~1B) into a competitive reasoning model using:

• high-quality distillation from a 120B teacher
• strict short-CoT supervision
• clean plain-text formatting
• deterministic final answers

The aim is to push the limits of small-model reasoning with minimal, high-signal CoT.

---

📫 Contact / Contributions

If you'd like to:

• contribute additional CoT variants
• add a long-CoT or step-by-step version
• benchmark your own small model
• propose improvements

Feel free to open an issue or discussion.