Mechanistic analysis of a GPT-2–like model exploring the compositionality gap in transformers. Using Logit Lens and Causal Tracing, the study identifies and overcomes a deep-layer bottleneck via dataset enhancement addressing the stated Compositionality Gap (NeurIPS24).
tokenizer hpc-clusters explainable-ai ai-research explainability compositionality neurips llm logitlens mechanistic-interpretability llm-training causal-tracing gpt-2-model ai-deployment neurips-paper
-
Updated
Aug 9, 2026 - Python