Skip to content

Latest commit

 

History

13 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SmolGPT-Fables v1

SmolGPT-Fables watercolor banner with butterflies

Hugging Face model Live ZeroGPU demo SmolGPT-Fables iOS app Training stories dataset CI status Apache 2.0 license

A compact 1.71B-parameter model for writing structured one-to-six-scene fables.

1. Choose how to run it

Experience Best for Start here
Hosted Studio Trying the model without installing anything Open SmolGPT-Fable-Studio
Local Transformers Running the full model from this repository python generate.py --prompt "..."
GGUF llama.cpp-compatible desktop runtimes Download Q4_K_M
iPhone / Apple Private Core ML generation Open SmolGPT-Fables-iOS

2. Quick start

git clone https://github.com/neonforestmist/SmolGPT-Fables.git
cd SmolGPT-Fables
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python generate.py --prompt "Write a three-scene folklore story about a ferryperson and a fox carrying a village's missing shadow."

The first local run downloads the pinned full-precision model from Hugging Face. Use python generate.py --help to select CPU, CUDA, or Apple MPS and adjust the generation length or sampling settings.

3. Model variants

Variant Size Format Best for
SmolGPT-Fables v1 3.42 GB BF16 safetensors Transformers and full model precision
SmolGPT-Fables v1 Compact 1.06 GB Q4_K_M GGUF llama.cpp-compatible runtimes
SmolGPT-Fables v1 Apple 0.96 GB Core ML INT4 Native Apple apps

The standard and GGUF files are published in the main model repository. The Apple package and tokenizer assets are published separately in the Core ML repository. All three variants represent the same 1.71B-parameter SmolGPT-Fables v1 model.

4. Evaluation

4.1 Studio/product-pipeline evaluation

This 60-prompt development suite exercises the Studio-shaped workflow: its structured story briefs, production prompt contract, generation settings, and behavior gates. It measures the configured product pipeline rather than the raw model in isolation, and it is not a broad language-model benchmark.

Check Result
Stories containing every requested full name 60 / 60 (100%)
First name combined with the wrong surname 0 / 60
Studio behavior gates All passed
Strict writing suite 32 / 60 (53.3%)
Minimum distinct-trigram rate 0.8394 (target 0.8500)

The strict suite most often found incomplete causal resolution and scene-length misses. These results describe this specific Studio workflow, not guarantees for every prompt or every inference setup.

4.2 Raw-model comparative evaluation

This separate comparison sends the same ten minimal story briefs directly to SmolGPT-Fables, SmolLM2, and Qwen through one harness. It scores the raw decoded text without the Studio's product-specific behavior gates. Decoding was deterministic. Bold marks the best direction in each row.

Story check (10 shared briefs) SmolGPT-Fables v1 SmolLM2 1.7B Instruct Qwen2.5 1.5B Instruct
Requested names retained 6 / 10 (60%) 8 / 10 (80%) 9 / 10 (90%)
Story detail anchors retained 12 / 20 (60%) 14 / 20 (70%) 14 / 20 (70%)
Exact requested scene count 9 / 10 (90%) 5 / 10 (50%) 0 / 10 (0%)
Story contract pass (names + scene count) 5 / 10 (50%) 4 / 10 (40%) 0 / 10 (0%)

Why the name-retention totals differ

The 60 / 60 result above and the 6 / 10 result here come from different prompt suites, harnesses, and scoring conditions. The first evaluates the configured Studio/product pipeline; the second isolates raw model generations under a shared cross-model contract. The values answer different questions and should not be compared as if they came from one test run.

The complete prompt suite, pinned revisions, raw SmolGPT-Fables example captures, and per-prompt results are in the benchmarks/smolgpt_fables_story_comparison_v1.json report. This is a focused product measurement, not a broad language-model leaderboard.

4.3 General-capability sanity check

SmolGPT-Fables is specialized for structured creative-writing behavior; it is not intended to make SmolLM2 universally stronger at unrelated language-model tasks. This compact zero-shot check documents that tradeoff directly. Bold marks the highest measured score in each row.

Task SmolGPT-Fables v1 SmolLM2 1.7B Instruct Qwen2.5 1.5B Instruct
IFEval (instruction-level strict accuracy) 54.55% 59.60% 53.54%
HellaSwag (normalized accuracy) 62.50% 67.19% 54.69%
ARC Easy (normalized accuracy) 67.19% 67.19% 54.69%
PIQA (normalized accuracy) 68.75% 71.88% 73.44%
WinoGrande (accuracy) 62.50% 67.19% 65.62%
MMLU (accuracy) 48.77% 50.44% 60.12%

All three pinned revisions were evaluated together with lm-evaluation-harness 0.4.9, zero-shot chat templates, and a fixed 64-example cap per task; MMLU applies that cap across its subject tasks. These are directional sanity checks rather than full-test-set leaderboard results. The exact revisions, raw scores, hardware, and metric selection are recorded in benchmarks/smolgpt_fables_lm_eval_v1.json, and the reproducible runner is in tools/benchmark_smolgpt_fables.py.

5. Core ML validation

The separate Apple package passed the published conversion and numerical smoke checks in its coreml_manifest.json:

Check Result
Core ML conversion Passed
INT4 compression Passed
Stateful prediction smoke test Passed
FP16 / INT4 cosine similarity 0.9529
Matching top token Yes
Top-5 token overlap 4 / 5

The reference iOS implementation downloads the Core ML package once, verifies checksums, compiles it on-device, and generates locally.

6. Examples from the hosted Studio

These three representative cards show complete, captured outputs from SmolGPT-Fables v1 generations in the Studio's Examples tab. They are stacked for easy reading: two three-scene stories and one six-scene story. The hosted Studio contains six complete examples in total.

The Names of Stars · Romance · 3 scenes · Mara Vale, Ilyan Rook
### Scene 01:
The cliffside observatory above the winter port held a brass compass, its needle trembling in the cold air. Mara Vale, a rival mapmaker, stood beside the window, her eyes fixed on the stars streaming across the sky. Ilyan Rook, her counterpart, stood at the edge of the room, his gaze fixed on the map spread out before him. The air was thick with tension, as the two mapmakers knew that the season when stars changed their names would soon arrive, and with it, the chance to claim the greatest prize in the world of cartography.

### Scene 02:
As the night wore on, Mara and Ilyan found themselves drawn to the same corner of the room, where a single, unmarked map lay spread out before them. The map was old, its ink faded to a deep brown, but it was clear that it had been drawn by a master hand. Mara and Ilyan both knew that the map was the key to unlocking the greatest prize in the world of cartography, and they both knew that they had to have it.

### Scene 03:
As the night wore on, Mara and Ilyan found themselves engaged in a heated debate over the map. Mara argued that the map was too old to be of any use, while Ilyan argued that the map was too valuable to be shared. The debate raged on, with neither side willing to back down. But just as it seemed that the debate would never end, a sudden movement caught everyone's eye.
The Last Crossing · Folklore · 3 scenes · Orra and the fox
# The Last Crossing

### Scene 01:
The braided river where crossings are paid for with useful stories swirled in mist as Orra stepped aboard her ferry. Frost creaked beneath the lantern, and she counted the last passenger before river freeze - a fox carrying a village's missing shadow. Its fur shimmered like moonlight, and its eyes gleamed with ancient wisdom.

### Scene 02:
Orra learned the shadow belonged to the village. It had been lost in the chaos of harvest, when laughter and tears mingled in the wind. The fox sat beside her, its paws brushing the edge of the bell that hummed softly, echoing the village's heartbeat. She realized the crossing was not just a journey but a gift.

### Scene 03:
As the river froze, Orra set the fox ashore, the shadow unfolding from its mouth like a wing. The fox vanished into the ice, leaving behind the bell and the memory of the village's joy. Orra watched as the river reshaped itself around them, and the story returned home.
The Teapot at Number Nine · Cozy fantasy · 6 scenes · Mina Bell, Pip
### Scene 01:
The tiny row house where the teapot remembered every neighbor's favorite song held its breath as Mina Bell entered. She had returned the borrowed teapot, but the house's annual porch supper was already in full swing. The teapot, its china glinting in the soft light, seemed to be watching Mina with a quiet curiosity.

### Scene 02:
As Mina approached the porch, the teapot began to sing. Its voice was low and melodious, filling the air with the scent of old china and fresh tea. The neighbors, who had been chatting and laughing, paused in their conversation, their faces filled with wonder.

### Scene 03:
Mina stood frozen, her eyes fixed on the teapot as it sang. She had never seen anything like it before. The teapot, sensing her wonder, continued to sing, its voice growing louder and more vibrant with each passing moment.

### Scene 04:
As the teapot sang, the porch supper began to take shape. The neighbors, who had been watching Mina with curiosity, began to gather around the teapot, their faces filled with wonder. The teapot, sensing their excitement, continued to sing, its voice growing louder and more vibrant with each passing moment.

### Scene 05:
As the teapot sang, the neighbors began to share stories and laughter. The teapot, sensing their joy, continued to sing, its voice growing louder and more vibrant with each passing moment. Mina, who had been watching the scene unfold, felt a sense of wonder and connection that she had never felt before.

### Scene 06:
As the teapot sang, the neighbors began to disperse, each carrying a small piece of the teapot's magic. Mina, who had been watching the scene unfold, felt a sense of wonder and connection that she had never felt before. She knew that she would carry the teapot's magic with her always, and that she would never forget the sense of wonder and connection that it had brought into her life.

These are observed generations from the published v1 revision, not promises that every prompt will use the same words.

7. Training and provenance

SmolGPT-Fables v1 was adapted from HuggingFaceTB/SmolLM2-1.7B-Instruct using the smolgpt-markdown-stories dataset.

8. Limitations

  • Names and requested scene counts are learned behavior, not hard guarantees.
  • Longer stories can lose continuity or end weakly.
  • The model may repeat phrases or flatten cultural nuance.
  • It is English-focused and intended for creative writing, not factual advice.
  • Review and edit generated writing before sharing it.

9. Citation

If SmolGPT-Fables v1 is useful in your work, please cite the model release:

@misc{lozada2026smolgptfables,
  author = {Lukas Lozada Perez},
  title = {SmolGPT-Fables v1},
  year = {2026},
  howpublished = {Hugging Face model card},
  url = {https://huggingface.co/neonforestmist/smolgpt-fables}
}

For work that uses the underlying model or training data, also cite SmolLM2 and the smolgpt-markdown-stories dataset.

10. License

Apache-2.0. Review the SmolLM2 model card and the dataset terms alongside this card when assessing use and redistribution.

About

Compact 1.71B story-writing model for readable one-to-six-scene fables, with Hugging Face, Studio, and Core ML companions.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages