feat(chatflow): stream LLM reasoning to a live thinking panel by lin-snow · Pull Request #37460 · langgenius/dify

lin-snow · 2026-06-15T08:37:25Z

Summary

Streams LLM chain-of-thought (separated mode) to a live Chatflow "thinking" panel over a dedicated out-of-band channel, keeping answer clean, and persists it so it survives refresh / shows in history.

Backend: route graphon NodeRunReasoningChunkEvent → QueueReasoningChunkEvent → reasoning_chunk SSE via a pure-emit handler (never writes answer); persist terminal reasoning_content per LLM node into message_metadata (zero migration).
Frontend: subscribe to reasoning_chunk, accumulate by node, render ReasoningPanel (shares a ThinkingDetails shell with ThinkBlock); rehydrate from metadata on history reload.

Depends on graphon #180. Draft until the graphon release + pin bump lands — backend CI is blocked until then. Context: #37184.

Screenshots

Before	After
panel goes dark while streaming	live "Thinking… (Xs)" → "Thought (Xs)"

Checklist

This change requires a documentation update, included: Dify Document
I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.
I ran make lint && make type-check (backend) and cd web && pnpm exec vp staged (frontend) to appease the lint gods

github-actions · 2026-06-15T08:38:52Z

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-06-18 03:13:58.441012494 +0000
+++ /tmp/pyrefly_pr.txt	2026-06-18 03:13:47.829001211 +0000
@@ -1,3 +1,5 @@
+ERROR Could not import `NodeRunReasoningChunkEvent` from `graphon.graph_events` [missing-module-attribute]
+  --> core/app/apps/workflow_app_runner.py:78:5
 ERROR Argument `SimpleNamespace` is not assignable to parameter `dataset` with type `Dataset` in function `dify_vdb_alibabacloud_mysql.alibabacloud_mysql_vector.AlibabaCloudMySQLVectorFactory.init_vector` [bad-argument-type]
   --> providers/vdb/vdb-alibabacloud-mysql/tests/unit_tests/test_alibabacloud_mysql_factory.py:42:38
 ERROR Argument `SimpleNamespace` is not assignable to parameter `dataset` with type `Dataset` in function `dify_vdb_alibabacloud_mysql.alibabacloud_mysql_vector.AlibabaCloudMySQLVectorFactory.init_vector` [bad-argument-type]
@@ -3766,65 +3768,65 @@
 ERROR `Literal['unknown']` is not assignable to attribute `stopped_by` with type `QueueStopEvent.StopBy` [bad-assignment]
   --> tests/unit_tests/core/app/entities/test_queue_entities.py:11:28
 ERROR Cannot index into `bool` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR Cannot index into `float` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR Cannot index into `int` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR Cannot index into `list[JsonValue]` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR Cannot index into `str` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR `None` is not subscriptable [unsupported-operation]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:27:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:32:16
 ERROR Cannot index into `bool` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR Cannot index into `float` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR Cannot index into `int` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR Cannot index into `list[JsonValue]` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR Cannot index into `str` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR `None` is not subscriptable [unsupported-operation]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:28:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:33:16
 ERROR Cannot index into `bool` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR Cannot index into `float` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR Cannot index into `int` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR Cannot index into `list[JsonValue]` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR Cannot index into `str` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR `None` is not subscriptable [unsupported-operation]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:51:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:56:16
 ERROR Cannot index into `bool` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR Cannot index into `float` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR Cannot index into `int` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR Cannot index into `list[JsonValue]` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR Cannot index into `str` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR `None` is not subscriptable [unsupported-operation]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:52:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:57:16
 ERROR Cannot index into `bool` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR Cannot index into `float` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR Cannot index into `int` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR Cannot index into `list[JsonValue]` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR Cannot index into `str` [bad-index]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR `None` is not subscriptable [unsupported-operation]
-  --> tests/unit_tests/core/app/entities/test_task_entities.py:53:16
+  --> tests/unit_tests/core/app/entities/test_task_entities.py:58:16
 ERROR Object of class `Mapping` has no attribute `close` [missing-attribute]
    --> tests/unit_tests/core/app/features/rate_limiting/test_rate_limit.py:396:9
 ERROR Argument `TestRateLimitGenerator.test_should_handle_generator_without_close_method.SimpleGenerator` is not assignable to parameter `generator` with type `Generator[str] | Mapping[str, Any]` in function `core.app.features.rate_limiting.rate_limit.RateLimit.generate` [bad-argument-type]
@@ -6039,6 +6041,12 @@
    --> tests/unit_tests/factories/test_file_factory.py:286:16
 ERROR `in` is not supported between `Literal['.txt']` and `None` [not-iterable]
    --> tests/unit_tests/factories/test_file_factory.py:287:16
+ERROR Missing argument `re_sign_file_url_answer` in function `fields.message_fields.ExploreMessageListItem.__init__` [missing-argument]
+  --> tests/unit_tests/fields/test_message_fields.py:23:38
+ERROR Missing argument `re_sign_file_url_answer` in function `fields.message_fields.ExploreMessageListItem.__init__` [missing-argument]
+  --> tests/unit_tests/fields/test_message_fields.py:30:38
+ERROR Missing argument `re_sign_file_url_answer` in function `fields.message_fields.MessageListItem.__init__` [missing-argument]
+  --> tests/unit_tests/fields/test_message_fields.py:35:34
 ERROR `dict[str, str | None]` is not assignable to TypedDict key `site` with type `list[dict[str, Any]] | list[dict[str, str]] | str` [bad-typed-dict-key]
    --> tests/unit_tests/libs/_human_input/support.py:102:32
 ERROR `dict[str, str]` is not assignable to TypedDict key `site` with type `list[FormInputConfig] | list[dict[str, Any]] | str` [bad-typed-dict-key]

github-actions · 2026-06-15T08:39:34Z

Pyrefly Type Coverage

Metric	Base	PR	Delta
Type coverage	48.79%	48.79%	-0.01%
Strict coverage	48.30%	48.29%	-0.01%
Typed symbols	28,339	28,339	0
Untyped symbols	30,047	30,055	+8
Modules	2907	2908	+1

codecov · 2026-06-15T08:40:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.18%. Comparing base (4304044) to head (feb778b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #37460   +/-   ##
=======================================
  Coverage   86.17%   86.18%           
=======================================
  Files        4824     4827    +3     
  Lines      243843   243895   +52     
  Branches    45096    45127   +31     
=======================================
+ Hits       210142   210211   +69     
+ Misses      29599    29582   -17     
  Partials     4102     4102

Flag	Coverage Δ
dify-ui	`94.94% <ø> (ø)`
web	`86.43% <100.00%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Consume graphon's NodeRunReasoningChunkEvent: route it to a new QueueReasoningChunkEvent and a reasoning_chunk SSE event via a pure-emit handler that never touches the answer. Persist terminal reasoning_content per LLM node into message_metadata (zero migration).

Subscribe to the reasoning_chunk SSE event, accumulate reasoning by node, and render a live ReasoningPanel (reusing the ThinkBlock visuals via a shared ThinkingDetails shell). Rehydrate reasoning from message metadata on history reload.

…ation persistence - ReasoningPanel: derive text inline instead of useMemo([content]); the live stream mutates the reasoningContent object in place under a stable reference, so a content-keyed memo froze the panel after the first delta. Add a test that re-renders with the same mutated object to lock in streaming. - Explore/installed-app message API: add ExploreMessageListItem + ExploreMessageInfiniteScrollPagination so message_metadata (incl. reasoning) is surfaced, letting chat-with-history rehydrate the thinking panel on reload. Base MessageListItem / service_api public contract left unchanged. - Persist reasoning per LLM node by accumulating across iteration/loop passes (append, not overwrite) to match the live stream; guard on isinstance(str).

… map Add frontend tests for the previously-uncovered reasoning wiring flagged by Codecov: - service/base.ts: handleStream dispatch of reasoning_chunk -> onReasoning - chat/hooks.ts: useChat onReasoning handlers (handleSend + handleResume paths), per-node accumulation, node_id fallback, is_final - answer/index.tsx: ReasoningPanel slot in both the normal and human-input layouts Also treat an empty reasoningContent map as no reasoning. The terminal reasoning dict is persisted for every chatflow message, so rehydrated answers carry {}; !!{} is truthy, which would mount a null panel and suppress the loading animation. Guard the slots with a hasReasoning check instead.

Add onReasoning handlers to both debug-and-preview useChat paths so streamed LLM reasoning deltas accumulate per node into reasoningContent and latch reasoningFinished on the terminal marker. Lift the reasoning-done decision out of ReasoningPanel into the Answer caller: the panel now takes a single derived `done` prop. Done latches on the first answer delta (the only mid-node signal), the reasoning terminal marker, or the response ending — fixing the think->answer handoff and freezing the elapsed timer. Update tests for the new prop.

Codecov flagged the two onReasoning callbacks added in 790cc73 (debug-and-preview hooks.ts) as uncovered. Add tests mirroring the base chat hooks coverage: - handleSend path: per-node accumulation, node_id fallback to '_', is_final, reasoning never leaking into content - handleResume path: accumulation onto the resumed answer node, empty-reasoning guard, node_id fallback

lin-snow self-assigned this Jun 15, 2026

lin-snow added 🐞 bug Something isn't working 💪 enhancement New feature or request labels Jun 15, 2026

github-actions Bot added the web This relates to changes on the web. label Jun 15, 2026

lin-snow force-pushed the feat/chatflow-reasoning-stream branch 4 times, most recently from 5f3b14a to 790cc73 Compare June 18, 2026 02:52

lin-snow added 6 commits June 18, 2026 11:09

lin-snow force-pushed the feat/chatflow-reasoning-stream branch from 790cc73 to feb778b Compare June 18, 2026 03:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(chatflow): stream LLM reasoning to a live thinking panel#37460

feat(chatflow): stream LLM reasoning to a live thinking panel#37460
lin-snow wants to merge 6 commits into
mainfrom
feat/chatflow-reasoning-stream

lin-snow commented Jun 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 15, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

lin-snow commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Screenshots

Checklist

Uh oh!

github-actions Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pyrefly Diff

Uh oh!

github-actions Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pyrefly Type Coverage

Uh oh!

codecov Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lin-snow commented Jun 15, 2026 •

edited

Loading

github-actions Bot commented Jun 15, 2026 •

edited

Loading

github-actions Bot commented Jun 15, 2026 •

edited

Loading

codecov Bot commented Jun 15, 2026 •

edited

Loading