Summary
GeminiClient leaves thinking_config at None, which means "model default". On the gemini-2.5-* family the default is thinking ON, and thinking tokens are drawn from the same output budget as the answer. Graphiti's structured-extraction prompts return long JSON, so the JSON is cut mid-object and the run dies parsing it.
Version: graphiti-core 0.30.2 (repo at eaa4128, "Bump graphiti-core to 0.30.2 (#1858)").
Where
graphiti_core/llm_client/gemini_client.py:100 — thinking_config: types.ThinkingConfig | None = Nonegraphiti_core/llm_client/gemini_client.py:126 — self.thinking_config = thinking_configgraphiti_core/llm_client/gemini_client.py:299 — passed straight through to GenerateContentConfig(..., thinking_config=self.thinking_config)
Nothing between those lines distinguishes "caller did not ask for thinking" from "thinking off". None reaches the API, the API applies the model default, and on gemini-2.5-flash that is thinking enabled.
Symptom
Any sufficiently long extraction (a multi-KB document) fails with a JSON parse error whose column is wherever the budget ran out:
Expecting value: line 1 column 2071 (char 2070)
Retrying does not help: the truncation point moves but the failure is the same. It looks like a malformed-response bug in the LLM client, which is why it is easy to misattribute.
Reproduction
Quickstart with GeminiClient(config=LLMConfig(api_key=..., model='gemini-2.5-flash', temperature=0)) and add_episode on a document of roughly 10 KB or more. Ingesting an 81-document corpus, this was ~100% fatal until fixed.
Fix that works
from google.genai import types as genai_types
GeminiClient(
config=LLMConfig(api_key=key, model='gemini-2.5-flash', temperature=0),
thinking_config=genai_types.ThinkingConfig(thinking_budget=0),
)With thinking_budget=0 the same 81-document corpus ingests cleanly.
Suggested change
Default thinking_budget=0 for gemini-2.5-* when the caller passes no thinking_config, rather than deferring to the model default — structured extraction wants the whole budget spent on the schema-shaped answer. A caller who wants thinking can still pass a ThinkingConfig explicitly. Failing that, _resolve_max_tokens (gemini_client.py:175) could account for the thinking allocation, and the docstring at line 110 could warn that None means thinking on for 2.5 models, not off.
Summary
GeminiClientleavesthinking_configatNone, which means "model default". On thegemini-2.5-*family the default is thinking ON, and thinking tokens are drawn from the same output budget as the answer. Graphiti's structured-extraction prompts return long JSON, so the JSON is cut mid-object and the run dies parsing it.Version: graphiti-core 0.30.2 (repo at
eaa4128, "Bump graphiti-core to 0.30.2 (#1858)").Where
graphiti_core/llm_client/gemini_client.py:100—thinking_config: types.ThinkingConfig | None = Nonegraphiti_core/llm_client/gemini_client.py:126—self.thinking_config = thinking_configgraphiti_core/llm_client/gemini_client.py:299— passed straight through toGenerateContentConfig(..., thinking_config=self.thinking_config)Nothing between those lines distinguishes "caller did not ask for thinking" from "thinking off".
Nonereaches the API, the API applies the model default, and ongemini-2.5-flashthat is thinking enabled.Symptom
Any sufficiently long extraction (a multi-KB document) fails with a JSON parse error whose column is wherever the budget ran out:
Retrying does not help: the truncation point moves but the failure is the same. It looks like a malformed-response bug in the LLM client, which is why it is easy to misattribute.
Reproduction
Quickstart with
GeminiClient(config=LLMConfig(api_key=..., model='gemini-2.5-flash', temperature=0))andadd_episodeon a document of roughly 10 KB or more. Ingesting an 81-document corpus, this was ~100% fatal until fixed.Fix that works
With
thinking_budget=0the same 81-document corpus ingests cleanly.Suggested change
Default
thinking_budget=0forgemini-2.5-*when the caller passes nothinking_config, rather than deferring to the model default — structured extraction wants the whole budget spent on the schema-shaped answer. A caller who wants thinking can still pass aThinkingConfigexplicitly. Failing that,_resolve_max_tokens(gemini_client.py:175) could account for the thinking allocation, and the docstring at line 110 could warn thatNonemeans thinking on for 2.5 models, not off.