Feat/traces logs enhance

README.md

@@ -531,6 +531,19 @@ Once a channel is connected, you can interact with DeerFlow directly from the ch
 
 > Messages without a command prefix are treated as regular chat — DeerFlow creates a thread and responds conversationally.
 
+#### Request Trace Correlation
+
+Gateway request trace correlation is disabled by default so existing HTTP responses and log formats stay unchanged. To enable it, set:
+
+```yaml
+logging:
+  enhance:
+    enabled: true
+    format: text
+```
+
+When enabled, every Gateway HTTP response includes `X-Trace-Id`, logs include `trace_id`, and Langfuse traces created by that request include `metadata.deerflow_trace_id` with the same value.
+
 #### LangSmith Tracing
 
 DeerFlow has built-in [LangSmith](https://smith.langchain.com) integration for observability. When enabled, all LLM calls, agent runs, and tool executions are traced and visible in the LangSmith dashboard.
@@ -565,6 +578,7 @@ If you are using a self-hosted Langfuse instance, set `LANGFUSE_BASE_URL` to you
 - `user_id` = effective user from `get_effective_user_id()` (falls back to `default` in no-auth mode)
 - `trace_name` = assistant id (defaults to `lead-agent`)
 - `tags` = `[env:<DEER_FLOW_ENV>, model:<model_name>]` (omitted when not set)
+- `metadata.deerflow_trace_id` = DeerFlow request correlation id, matching `X-Trace-Id` when request trace correlation is enabled
 
 These are injected into `RunnableConfig.metadata` at the graph invocation root for both the gateway path (`runtime/runs/worker.py::run_agent`) and the embedded path (`client.py::DeerFlowClient.stream`), so any LangChain-compatible callback can read them. Set `DEER_FLOW_ENV` (or `ENVIRONMENT`) to tag traces by deployment environment.
 

backend/AGENTS.md

@@ -248,7 +248,7 @@ Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** direc
 
 **Config Hot-Reload Boundary**: Gateway dependencies route through `get_app_config()` on every request, so per-run fields like `models[*].max_tokens`, `summarization.*`, `title.*`, `memory.*`, `subagents.*`, `tools[*]`, and the agent system prompt pick up `config.yaml` edits on the next message. `AppConfig` is intentionally **not** cached on `app.state` — `lifespan()` keeps a local `startup_config` variable for one-shot bootstrap work and passes it to `langgraph_runtime(app, startup_config)`.
 
-Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig`, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, `log_level`, `channels`, `channel_connections`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`.
+Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig`, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `database`, `checkpointer`, `run_events`, `stream_bridge`, `sandbox`, `log_level`, `logging`, `channels`, `channel_connections`. Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`.
 
 Configuration priority:
 1. Explicit `config_path` argument
@@ -555,6 +555,33 @@ A terminal-native UI over the embedded harness, exposed as the `deerflow` consol
 
 **Tests**: `tests/test_tui_*.py` — pure layers via plain pytest, the app/palette/overlays via Textual's pilot harness with a fake in-process session, and `test_tui_persistence.py` for the `threads_meta` round-trip.
 
+### Request Trace Context (`packages/harness/deerflow/trace_context.py`)
+
+Request trace correlation is controlled by `logging.enhance.enabled` at **both** entry points, gated through the shared helper `deerflow.config.app_config.is_trace_correlation_enabled` so the Gateway and embedded paths cannot drift:
+
+- **Gateway HTTP**: `app.gateway.trace_middleware.TraceMiddleware` binds one request-level trace id per HTTP request, inheriting inbound `X-Trace-Id` when present or generating a new id otherwise. The middleware writes the final value to every HTTP response at `http.response.start`, which covers SSE / streaming responses without consuming the body.
+- **Embedded / TUI / CLI**: `DeerFlowClient.stream()` mints (or inherits) a request-level trace id per turn only when the flag is on. When it is off, no fresh id is minted — a caller that explicitly wraps `stream()` in `request_trace_context(...)` still opts in, because the downstream `get_current_trace_id()` read propagates that value into Langfuse metadata regardless of the flag. Because `stream()` is a sync generator (which shares the caller's context), the id binding is set/reset around each `next()` step rather than around `yield from`: this keeps LangGraph node execution and its log records inside the binding, while returning control to the caller with the ContextVar restored — avoids cross-request leak between yields and `ValueError: <Token> was created in a different Context` on GC-driven close of an abandoned generator (regression pinned by `tests/test_client_langfuse_metadata.py::test_stream_does_not_leak_trace_id_to_caller_context_between_yields` and `::test_stream_abandoned_generator_close_does_not_raise_cross_context`).
+
+The same ContextVar value is injected into enhanced log records as `trace_id` and into Langfuse metadata as `deerflow_trace_id`.
+
+`logging` is registered as a **restart-required** field
+(`STARTUP_ONLY_FIELDS["logging"]`): `configure_logging()` installs the trace-context
+filter and enhanced formatter on root handlers only during app.py lifespan startup,
+and `TraceMiddleware` captures `logging.enhance.enabled` once when the FastAPI app
+is constructed (via `resolve_trace_enabled(get_app_config())` in `create_app()`,
+itself a thin alias for `is_trace_correlation_enabled`). This keeps the response
+`X-Trace-Id` header, log `trace_id` fields, and Langfuse `deerflow_trace_id`
+coherent — a runtime `config.yaml` edit to `logging.enhance.*` needs a Gateway
+restart to take effect. The `deerflow_trace_id` chain inherits this guarantee
+transitively because every injection point ultimately reads the same
+`trace_context` ContextVar that the middleware alone populates. `DeerFlowClient`
+reads its own `self._app_config` snapshot (captured at `__init__`) through the
+same helper for the embedded gate.
+
+`deerflow_trace_id` is a DeerFlow correlation metadata key, not Langfuse's native
+trace id and not a DeerFlow `run_id`. Keep the existing subagent `trace_id` field
+separate: that short id is still only for subagent execution logs/status.
+
 ### Tracing System (`packages/harness/deerflow/tracing/`)
 
 LangSmith and Langfuse are both supported. The wiring lives in two layers:
@@ -570,13 +597,15 @@ LangSmith and Langfuse are both supported. The wiring lives in two layers:
 | `langfuse_user_id`    | `get_effective_user_id()` (`default` in no-auth); for subagents, captured from `runtime.context` at `task_tool` time via `resolve_runtime_user_id()` |
 | `langfuse_trace_name` | `RunRecord.assistant_id` / client `agent_name` (defaults to `lead-agent`); for subagents, `subagent:<name>` (lowercased, `_` → `-`) |
 | `langfuse_tags`       | `env:<DEER_FLOW_ENV>` + `model:<model_name>`  |
+| `deerflow_trace_id`   | Current request/entry trace id from `deerflow.trace_context`; matches `X-Trace-Id` for enhanced Gateway HTTP requests. Gated by `logging.enhance.enabled` in both gateway and embedded paths via `is_trace_correlation_enabled` — off by default; embedded callers can still opt in per-turn by wrapping `stream()` in `request_trace_context(...)` |
 
 Returns `{}` when Langfuse is not in the enabled providers — LangSmith-only deployments are unaffected. Set `DEER_FLOW_ENV` (or `ENVIRONMENT`) to tag traces by deployment environment. Tests live in `tests/test_tracing_factory.py`, `tests/test_tracing_metadata.py`, `tests/test_worker_langfuse_metadata.py`, `tests/test_client_langfuse_metadata.py`, and `tests/test_subagent_executor.py::TestSubagentTracingWiring`.
 
 ### Config Schema
 
 **`config.yaml`** key sections:
 - `models[]` - LLM configs with `use` class path, `supports_thinking`, `supports_vision`, provider-specific fields
+- `logging.enhance` - Optional request trace correlation (`enabled`, `format`) for Gateway `X-Trace-Id`, log `trace_id`, and Langfuse `deerflow_trace_id`
 - vLLM reasoning models should use `deerflow.models.vllm_provider:VllmChatModel`; for Qwen-style parsers prefer `when_thinking_enabled.extra_body.chat_template_kwargs.enable_thinking`, and DeerFlow will also normalize the older `thinking` alias
 - `tools[]` - Tool configs with `use` variable path and `group`
 - `tool_groups[]` - Logical groupings for tools

backend/app/gateway/__init__.py

@@ -1,4 +1,12 @@
-from .app import app, create_app
 from .config import GatewayConfig, get_gateway_config
 
 __all__ = ["app", "create_app", "GatewayConfig", "get_gateway_config"]
+
+
+def __getattr__(name: str):
+    """Lazily expose the FastAPI app without initializing it on package import."""
+    if name in {"app", "create_app"}:
+        from .app import app, create_app
+
+        return app if name == "app" else create_app
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

backend/app/gateway/app.py

@@ -29,8 +29,9 @@
     threads,
     uploads,
 )
+from app.gateway.trace_middleware import TraceMiddleware, resolve_trace_enabled
 from deerflow.config import app_config as deerflow_app_config
-from deerflow.config.app_config import apply_logging_level
+from deerflow.logging_config import DEFAULT_LOG_DATE_FORMAT, DEFAULT_LOG_FORMAT, configure_logging
 from deerflow.uploads.manager import cleanup_stale_upload_staging_files
 
 AppConfig = deerflow_app_config.AppConfig
@@ -39,8 +40,8 @@
 # Default logging; lifespan overrides from config.yaml log_level.
 logging.basicConfig(
     level=logging.INFO,
-    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
-    datefmt="%Y-%m-%d %H:%M:%S",
+    format=DEFAULT_LOG_FORMAT,
+    datefmt=DEFAULT_LOG_DATE_FORMAT,
 )
 
 logger = logging.getLogger(__name__)
@@ -173,7 +174,7 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
     # snapshot on `app.state` to keep that contract enforceable.
     try:
         startup_config = get_app_config()
-        apply_logging_level(startup_config.log_level)
+        configure_logging(startup_config)
         logger.info("Configuration loaded successfully")
         warn_if_auth_disabled_enabled()
     except Exception as e:
@@ -370,6 +371,14 @@ def create_app() -> FastAPI:
             allow_headers=["*"],
         )
 
+    # Request trace correlation: when logging.enhance.enabled=true, bind one
+    # trace id per Gateway HTTP request and write it to response start headers.
+    # `logging` is registered as restart-required (see reload_boundary.py) so we
+    # snapshot the flag from the startup AppConfig instead of reading live; a
+    # runtime toggle would otherwise leave the log formatter (installed once by
+    # configure_logging() at lifespan startup) out of sync with the middleware.
+    app.add_middleware(TraceMiddleware, enabled=_resolve_trace_enabled_for_app_construction())
+
     # Include routers
     # Models API is mounted at /api/models
     app.include_router(models.router)
@@ -431,5 +440,15 @@ async def health_check() -> dict[str, str]:
     return app
 
 
+def _resolve_trace_enabled_for_app_construction() -> bool:
+    """Resolve the trace middleware flag without making imports require config.yaml."""
+    try:
+        return resolve_trace_enabled(get_app_config())
+    except FileNotFoundError:
+        # Startup lifespan still performs strict config loading before serving.
+        logger.debug("config.yaml not found while constructing Gateway app; TraceMiddleware disabled for this app instance")
+        return False
+
+
 # Create app instance for uvicorn
 app = create_app()

backend/app/gateway/routers/suggestions.py

@@ -1,5 +1,6 @@
 import json
 import logging
+import os
 import re
 
 from fastapi import APIRouter, Depends, Request
@@ -10,6 +11,8 @@
 from app.gateway.deps import get_config
 from deerflow.config.app_config import AppConfig
 from deerflow.models import create_chat_model
+from deerflow.runtime.user_context import get_effective_user_id
+from deerflow.tracing import inject_langfuse_metadata
 
 logger = logging.getLogger(__name__)
 
@@ -175,7 +178,16 @@ async def generate_suggestions(
 
     try:
         model = create_chat_model(name=body.model_name, thinking_enabled=False, app_config=config)
-        response = await model.ainvoke([SystemMessage(content=system_instruction), HumanMessage(content=user_content)], config={"run_name": "suggest_agent"})
+        invoke_config: dict = {"run_name": "suggest_agent"}
+        inject_langfuse_metadata(
+            invoke_config,
+            thread_id=thread_id,
+            user_id=get_effective_user_id(),
+            assistant_id="suggest_agent",
+            model_name=body.model_name,
+            environment=os.environ.get("DEER_FLOW_ENV") or os.environ.get("ENVIRONMENT"),
+        )
+        response = await model.ainvoke([SystemMessage(content=system_instruction), HumanMessage(content=user_content)], config=invoke_config)
         raw = _extract_response_text(response.content)
         suggestions = _parse_json_string_list(raw) or []
         cleaned = [s.replace("\n", " ").strip() for s in suggestions if s.strip()]

backend/app/gateway/trace_middleware.py

@@ -0,0 +1,63 @@
+"""Gateway request trace middleware."""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+from starlette.datastructures import Headers, MutableHeaders
+from starlette.types import ASGIApp, Message, Receive, Scope, Send
+
+from deerflow.config.app_config import is_trace_correlation_enabled
+from deerflow.trace_context import TRACE_ID_HEADER, request_trace_context
+
+logger = logging.getLogger(__name__)
+
+
+class TraceMiddleware:
+    """Bind a request-level trace id and write it to HTTP response headers.
+
+    The ``enabled`` flag is a **startup snapshot** rather than a per-request
+    live read: ``logging`` is registered as restart-required in
+    ``deerflow.config.reload_boundary.STARTUP_ONLY_FIELDS`` because
+    ``configure_logging()`` only installs the trace-context filter and
+    formatter during app.py lifespan startup. Reading ``logging.enhance.enabled``
+    live here would let a runtime config edit surface the response
+    ``X-Trace-Id`` header and Langfuse ``deerflow_trace_id`` immediately while
+    the log formatter stays on its startup value, contradicting the
+    restart-required contract IDE hover surfaces on ``AppConfig.logging``.
+    """
+
+    def __init__(self, app: ASGIApp, *, enabled: bool):
+        self.app = app
+        self.enabled = bool(enabled)
+
+    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
+        if scope["type"] != "http" or not self.enabled:
+            await self.app(scope, receive, send)
+            return
+
+        headers = Headers(scope=scope)
+        incoming_trace_id = headers.get(TRACE_ID_HEADER)
+
+        with request_trace_context(incoming_trace_id) as trace_id:
+
+            async def send_with_trace(message: Message) -> None:
+                if message["type"] == "http.response.start":
+                    response_headers = MutableHeaders(scope=message)
+                    response_headers[TRACE_ID_HEADER] = trace_id
+                await send(message)
+
+            await self.app(scope, receive, send_with_trace)
+
+
+def resolve_trace_enabled(config: Any) -> bool:
+    """Read ``logging.enhance.enabled`` from an ``AppConfig``-like object.
+
+    Thin backwards-compatible alias around
+    :func:`deerflow.config.app_config.is_trace_correlation_enabled`, kept so
+    existing gateway callers and tests do not have to switch imports. Both
+    the Gateway middleware and the embedded ``DeerFlowClient`` resolve the
+    gate through the same harness helper so their behaviour cannot drift.
+    """
+    return is_trace_correlation_enabled(config)