You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[](https://huggingface.co/spaces/djilyn/Vector-Translator)
6
+
[](https://colab.research.google.com/github/Djilyan-auguste/vector-translator/blob/main/experiments.ipynb)
A mechanistic interpretability research project that extracts, decodes, and tests causal control of semantic concepts from the internal activations of GPT-2 small (124M parameters).
9
+
> **[🚀 Live Demo](https://huggingface.co/spaces/djilyn/Vector-Translator)** — Type a sentence and watch GPT-2 small “think” layer by layer.
6
10
7
-
The project proceeds through five phases: (P0) identifying the decision layer via logit lens analysis; (P1–P2) building a labeled dataset and training linear probes to decode concepts from residual stream activations; (P3) training a non-linear MLP translator and comparing its performance to linear baselines; and (P4) testing whether extracted concept directions are causally steerable via activation steering.
11
+
**Vector Translator** is an end-to-end mechanistic interpretability project that extracts, decodes, and tests *causal control* of semantic concepts from the internal activations of GPT-2 small (124M parameters).
12
+
13
+
```
14
+
Input text → residual stream → decoded concepts → causal steering test
15
+
P0 P1–P3 P4
16
+
```
17
+
18
+
---
19
+
20
+
## ✨ Why this matters
21
+
22
+
Large language models are black boxes: they produce fluent text, but we rarely know *how* they represent meaning. This project reverse-engineers GPT-2 by treating its hidden states as an object of study in themselves.
23
+
24
+
### Key takeaway
25
+
26
+
> **Decoding is not control.**
27
+
>
28
+
> We can linearly decode concepts like DATE, NUMBER, or NOUN from GPT-2’s residual stream with high accuracy. But the directions that best *describe* those concepts do not *cause* them when injected back into the model. This is a real, falsifiable negative result — and it tells us exactly what to build next.
29
+
30
+
This matters because:
31
+
-**Safety:** before we can align or steer large models, we must know which internal directions are actually causal.
32
+
-**Transparency:** if we can map activations to human concepts, we can audit model behavior instead of trusting it blindly.
33
+
-**Efficiency:** small, interpretable concept probes can replace expensive black-box probing in some debugging workflows.
**[🤗 Vector Translator — Live Demo](https://huggingface.co/spaces/djilyn/Vector-Translator)**
14
42
15
-
We project the residual stream at each transformer layer onto the vocabulary using the unembedding matrix \(W_U\)[nostalgebraist, 2020]. The key finding is that prediction quality crystallizes at layer 6, not at the final layer 12. Rank percentile (the fraction of vocabulary ranking above the true token) improves from 12.5% at the input to 2.1% at layer 6, a 6× improvement. This pattern is robust across text types:
43
+
Enter any text, pick a layer, and see the top-5 predictions, rank percentile, and probability curves evolve through GPT-2’s layers.
*The demo shows the Logit Lens (P0): layer-by-layer prediction crystallization for the prompt "The cat sat on the mat and looked".*
47
+
48
+
> 💡 **Note:** The current demo focuses on P0 (Logit Lens). A P2/P3 tab showing concept probes and MLP decoding is on the roadmap.
49
+
50
+
---
51
+
52
+
## 📊 Results Summary
53
+
54
+
### P0 — Logit Lens: The “Decision Layer”
55
+
56
+
We project the residual stream at each layer onto the vocabulary using the unembedding matrix \(W_U\)[nostalgebraist, 2020]. The key finding is that prediction quality **crystallizes at layer 6**, not at the final layer 12. Rank percentile (the fraction of vocabulary ranking above the true token) improves from 12.5% at the input to **2.1% at layer 6**, a 6× improvement. This pattern is robust across text types:
16
57
17
58
| Text Type | Input Rank | Layer 6 Rank | Improvement Factor |
@@ -22,7 +63,7 @@ We project the residual stream at each transformer layer onto the vocabulary usi
22
63
| Factual | 81.5% | 2.8% | 29× |
23
64
| Ambiguous | 83.0% | 2.0% | 42× |
24
65
25
-
Rank percentile is used in preference to raw probability because GPT-2 small rarely assigns high absolute probabilities to correct tokens; a "good" prediction typically has probability below 0.01%. Rank percentile remains interpretable across model scales.
66
+
Rank percentile is used instead of raw probability because GPT-2 small rarely assigns high absolute probabilities to correct tokens; a "good" prediction typically has probability below 0.01%. Rank percentile remains interpretable across model scales.
### P4 — Activation Steering: A Falsifiable Negative Result
74
115
75
116
We test whether mean-difference directions extracted from P1 activations are causally steerable. For each concept, we compute `mean(positive) - mean(negative)`, normalize, and inject at layer 6 with scaling factors `alpha ∈ [-20, 20]` via TransformerLens hooks [Nanda, 2022].
76
117
@@ -83,56 +124,92 @@ We test whether mean-difference directions extracted from P1 activations are cau
**Result:** Mean-difference directions are correlational, not causal. They capture where concept tokens tend to cluster in activation space, but this cluster is not a steerable direction. This is a real negative result: it confirms that decoding (P2/P3) and control (P4) are distinct problems, and points to future work using adversarial contrast pairs, PCA, and orthogonalization as in refusal direction research [Arditi et al., 2024].
127
+
#### What this negative result means
128
+
129
+
**Mean-difference directions are correlational, not causal.** They capture where concept tokens tend to cluster in activation space, but this cluster is not a steerable direction.
130
+
131
+
This is not a failure — it is a **scientific result**:
132
+
1. It confirms that **decoding (P2/P3) and control (P4) are distinct problems**.
133
+
2. It validates the methodology: we proposed a hypothesis, tested it, and falsified it.
134
+
3. It points directly to future work: **adversarial contrast pairs + PCA + orthogonalization**, as used in refusal direction research [Arditi et al., 2024].
135
+
136
+
In other words: we now know *why* the simple approach fails and *exactly* what to try next.
0 commit comments