Reproducibility package for the published article:
Context Sufficiency in Transformers: Information-Theoretic Limits, Native Functional Dependence, Representational Accessibility, and Predictive Readout
Dimitris Karydas, Dimosthenis Margaritis and Helen C. Leligou. Computers 2026, 15(11), 769.
This repository provides two complementary levels of reproducibility:
- Analysis reproduction from archived outputs — reproduces the reported tables, figures, and summary statistics without retraining models.
- Full experimental provenance — retains the original experiment notebooks used to generate the archived outputs.
For the fastest verification, open and run:
reproduce_paper.ipynb
The notebook automatically searches the repository root and results/ for the archived experiment outputs.
A successful run ends with:
PAPER REPRODUCTION FROM EXISTING RESULTS: PASS
For immediate visual inspection without rerunning, see:
reproduce_paper_executed.ipynb
The executed notebook contains the reproduced tables, figures, audit outputs, and final PASS result.
.
├── README.md
├── requirements.txt
├── requirements-experiments.txt
├── reproduce_paper.ipynb
├── reproduce_paper_executed.ipynb
├── notebooks/ # original experiment notebooks
├── results/ # archived completed experiment outputs
└── figures/ # reproduced publication and supplementary figures
For analysis-only reproduction:
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
jupyter notebook reproduce_paper.ipynbFor full experimental reruns, additionally install:
pip install -r requirements-experiments.txtGPU-enabled PyTorch installation may depend on the local CUDA environment; use the appropriate PyTorch build for the target system.
For the latent-state HMM extension, the final information-gap calculation uses matched/same-history quantities. Older negative-gap intermediate outputs are obsolete and are not part of the final analysis.
The small final HMM CSV/JSON outputs needed by the unified reproduction notebook are mirrored in results/. The complete HMM archive is maintained separately from the lightweight GitHub inputs. The archival deposit designated in the article is Zenodo 10.5281/zenodo.23528707; the deposit must be published before that reserved DOI provides public access. The full HMM archive is not required for analysis-only reproduction.
For CPU-only verification from the frozen archives:
python -m pip install -r requirements-analysis.txt
python scripts/verify_inputs.py
python scripts/run_reproduction.pyThe runner uses a fresh temporary directory and writes to a new
validation-run/ directory. It verifies input checksums, runs every analysis
cell, requires all 15 figures, and fails on numerical-audit errors. See
REPRODUCIBILITY.md for tolerances and reproducibility/EXPERIMENTS.md for
GPU rerun prerequisites and diagnostics. This verifies analysis from archived
outputs; it does not establish a new full-training replication.
REPRODUCIBILITY.mdmaps each analysis component to its experiment notebook, archived outputs, and reproduced figures.CITATION.cffprovides repository citation metadata.
The figures/ directory contains the 15 reproduced figures generated from the archived outputs, including the corrected HMM gap-decomposition figure.
The unified notebook reproduces the reported analyses from completed outputs. It is intentionally not described as end-to-end retraining reproduction.
The experiment notebooks under notebooks/ are provided for provenance and full reruns, including the scratch Markov experiments, Qwen representation studies, causal intervention, robustness and coverage analyses, Pythia cross-family validation, true-state predictive-readout control, and the latent-state HMM extension.
The article is published in Computers; the publication-facing metadata is maintained on the default branch.
The exact reproducibility snapshot cited in the article is commit ad6cdf33e91a767634b5cbfd06e85b8cbe0fd14d. Later documentation updates do not replace this frozen snapshot or change its experiment data. The separately prepared snapshot ZIP and full HMM ZIP retain their original checksums for the archival deposit.
Analysis reproduction uses the archived outputs and requires no GPU training rerun. Complete historical GPU environment details were not recorded for every experiment; see reproducibility/EXPERIMENTS.md for the documented provenance and limitations.
The project-authored code and documentation are distributed under the MIT license; see LICENSE. Third-party model weights and dependencies retain their own licenses.