Skip to content

Commit a8bfb5a

Browse files
committed
fix: rename semantic drift to lexical drift, update default model names
Jaccard distance measures vocabulary replacement rate, not semantic similarity. Renamed all labels and comments accordingly. Updated hardcoded default models to current versions (gpt-5.4-mini, claude-haiku-4-5-20251001, gemini-2.5-flash).
1 parent f02823f commit a8bfb5a

2 files changed

Lines changed: 17 additions & 20 deletions

File tree

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,11 @@
1-
# Demo 3: Provider Sensitivity - Multi-provider configuration
2-
# Fill in keys for the providers you want to compare.
1+
# Provider Sensitivity - Multi-provider config
2+
# Fill in your API keys for each provider you want to compare.
33
# The demo will automatically skip providers that are not ready.
44

5-
# Default provider/model on startup
6-
provider=ollama
7-
model=llama3.2:3b
5+
provider=openai
6+
model=gpt-5.4-mini
87
temperature=0.4
9-
max_tokens=250
8+
max_tokens=1000
109
timeout_seconds=45
1110

1211
# Cloud API keys (leave empty to skip provider, fill in to enable)
@@ -15,12 +14,10 @@ anthropic_api_key=
1514
gemini_api_key=
1615

1716
# Ollama local endpoint (no key required)
18-
# Start with: ollama serve
19-
# Ensure your selected model is pulled, for example: ollama pull llama3.2
2017
ollama_base_url=http://localhost:11434
2118

22-
# Optional per-provider overrides
23-
openai_model=gpt-4o-mini
24-
anthropic_model=claude-3-5-haiku-latest
25-
gemini_model=gemini-2.0-flash
19+
# Per-provider model overrides
20+
openai_model=gpt-5.4-mini
21+
anthropic_model=claude-haiku-4-5-20251001
22+
gemini_model=gemini-2.5-flash
2623
ollama_model=llama3.2:3b

demos/provider-sensitivity/provider-sensitivity.nlogox

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<?xml version="1.0" encoding="utf-8"?>
22
<model version="NetLogo 7.0.3" snapToGrid="true">
3-
<code><![CDATA[; ABOUTME: Telephone Game demo — parallel provider chains paraphrase a seed message, revealing semantic drift.
3+
<code><![CDATA[; ABOUTME: Telephone Game demo — parallel provider chains paraphrase a seed message, revealing lexical drift.
44
; ABOUTME: Uses turtles, directed links, Jaccard-based drift scoring, optional thinking mode.
55
66
extensions [llm]
@@ -12,7 +12,7 @@ turtles-own [
1212
chain-position ;; 0 = seed, 1..N-1 = paraphrase positions
1313
current-message ;; message this turtle holds
1414
original-message ;; seed message (for drift comparison)
15-
drift-score ;; [0,1] semantic drift from original
15+
drift-score ;; [0,1] lexical drift from original (Jaccard distance on word sets)
1616
length-ratio ;; response length / original length
1717
processed? ;; has this turtle completed its paraphrase?
1818
error? ;; did this turtle's LLM call fail?
@@ -478,9 +478,9 @@ end
478478
;; ===== DEFAULT MODELS =====
479479
480480
to-report get-default-model [prov]
481-
if prov = "openai" [ report "gpt-4o-mini" ]
482-
if prov = "anthropic" [ report "claude-3-5-haiku-latest" ]
483-
if prov = "gemini" [ report "gemini-2.0-flash" ]
481+
if prov = "openai" [ report "gpt-5.4-mini" ]
482+
if prov = "anthropic" [ report "claude-haiku-4-5-20251001" ]
483+
if prov = "gemini" [ report "gemini-2.5-flash" ]
484484
if prov = "ollama" [ report "llama3.2:3b" ]
485485
report "unknown"
486486
end
@@ -644,8 +644,8 @@ end
644644
<slider x="15" step="1" y="170" max="10" width="230" display="chain-length" height="50" min="3" direction="Horizontal" default="6.0" variable="chain-length"></slider>
645645
<switch x="15" y="225" height="40" on="true" variable="show-labels?" width="230" display="show-labels?"></switch>
646646
<switch x="15" y="265" height="40" on="false" variable="thinking-mode?" width="230" display="thinking-mode?"></switch>
647-
<plot x="45" autoPlotX="true" yMax="1.0" autoPlotY="true" yAxis="Drift" y="505" xMin="0.0" height="165" legend="true" xMax="10.0" yMin="0.0" width="550" xAxis="Chain Position" display="Semantic Drift by Provider">
648-
<setup>set-current-plot "Semantic Drift by Provider"</setup>
647+
<plot x="45" autoPlotX="true" yMax="1.0" autoPlotY="true" yAxis="Drift" y="505" xMin="0.0" height="165" legend="true" xMax="10.0" yMin="0.0" width="550" xAxis="Chain Position" display="Lexical Drift by Provider">
648+
<setup>set-current-plot "Lexical Drift by Provider"</setup>
649649
<update><![CDATA[let idx 0
650650
foreach ready-providers-list [ prov ->
651651
create-temporary-plot-pen prov
@@ -680,7 +680,7 @@ foreach ready-providers-list [ prov ->
680680
</widgets>
681681
<info>## WHAT IS IT?
682682

683-
A Telephone Game for LLMs. Parallel chains of turtles pass a seed message through repeated paraphrasing by different LLM providers. Each position in the chain asks its provider to rephrase what it received. As the message travels, semantic drift accumulates — and different providers drift at different rates and in different ways.
683+
A Telephone Game for LLMs. Parallel chains of turtles pass a seed message through repeated paraphrasing by different LLM providers. Each position in the chain asks its provider to rephrase what it received. As the message travels, lexical drift accumulates — vocabulary shifts as each provider substitutes its own word choices, and different providers drift at different rates.
684684

685685
## HOW IT WORKS
686686

0 commit comments

Comments
 (0)