Skip to content

task-export.md

Latest commit

 

History

History
501 lines (387 loc) · 28.6 KB

task-export.md

File metadata and controls

501 lines (387 loc) · 28.6 KB
Raw

Task Export

Shofer provides two export formats for task history: Markdown (conversation transcript) and JSON (structured trace). Both export the full raw message exchange between the user and the LLM, including tool calls, tool results, and reasoning blocks.

Quick Reference

Markdown JSON
Button Export (DownloadIcon) Export JSON (FileJsonIcon)
File extension .md .json
Human-readable ✅ Any text editor ❌ Needs JSON viewer
Machine-readable ❌ Free-form text ✅ Schema'd, scriptable
Token counts ❌ ✅ Per-call + aggregate
Cost (USD) ❌ ✅ Per-call + aggregate
API protocol / model ❌ ✅ Per call
Task metadata ❌ ✅ taskId, mode, timestamps
Reasoning Inline [Reasoning] block Dedicated reasoning field
Tool calls Text [Tool Use: name] + key:value Structured JSON with input/result
Use case Reading, sharing, auditing Analytics, cost tracking, evals, replay

How to Export

  1. Start a task in Shofer
  2. Click the task title bar at the top of the chat panel to expand it
  3. Use the buttons in the action row:
    • Export (download icon) — exports as Markdown (.md)
    • Export JSON (file icon) — exports as JSON (.json)
  4. Choose a save location in the file dialog

Both buttons are also available from the History panel (clock icon in the VS Code title bar) for completed tasks.

Markdown Export

The Markdown export produces a plain-text transcript of the conversation. Each turn is separated by --- and labeled with the role.

Format

**User:**

<user_message>
1+1
</user_message>
...

---

**Assistant:**

[Reasoning]
The user asked "1+1" which equals 2...
2

---

**User:**

[ERROR] You did not use a tool...
...

---

**Assistant:**

[Reasoning]
...
[Tool Use: attempt_completion]
Result: 1 + 1 = 2

Content Blocks

Block Markdown Representation
Text Inline text
Reasoning [Reasoning]\n{content}
Tool use [Tool Use: {name}]\n{key: value, ...}
Tool result [Tool{ (Error)}]\n{content}
Images [Image]

Source

Generated by export-markdown.ts from the task's api_conversation_history.jsonl.

JSON Export

The JSON export produces a structured trace designed for programmatic consumption. Each API call is a separate entry with metadata, messages, tool calls, and reasoning.

Schema

interface JsonExportTrace {
	version: 1 // Schema version
	taskId: string // UUID of the task
	task: string // Human-readable task description
	mode?: string // Mode slug (e.g., "code", "architect")
	createdAt: string // ISO 8601 timestamp
	calls: JsonExportCall[] // One entry per API request
	totalTokens: {
		// Aggregate across all calls
		input: number
		output: number
		cacheWrite: number
		cacheRead: number
	}
	totalCostUsd: number // Aggregate cost
	totalCalls: number
	totalToolCalls: number
}

interface JsonExportCall {
	index: number // 1-based call number
	apiProtocol?: string // "anthropic", "openai"
	model?: string // Model ID
	inputTokens: number // Input tokens (estimated if `_tokensEstimated`)
	outputTokens: number // Output tokens (estimated if `_tokensEstimated`)
	cacheWriteTokens: number
	cacheReadTokens: number
	costUsd: number
	cancelled?: boolean
	cancelReason?: string
	streamingFailedMessage?: string
	messages: MessageParam[] // Anthropic-format messages for this call (empty `[]` for error-only calls)
	toolCalls: JsonExportToolCall[] // Tool calls from the assistant message
	reasoning?: string // Extracted reasoning / thinking
	retryAttempt?: number // Number of retries before this attempt (0 = first try)
	durationMs?: number // Whole ms, request open → stream end; absent when no stream end was reached
	firstChunkMs?: number // Whole ms, request open → first stream chunk; absent when the stream produced none
	thinkingMs?: number // Whole ms the model spent reasoning, summed; ABSENT means no reasoning phase, never 0
	reasoningIntervalsMs?: Array<[number, number]> // Every reasoning window as [startMs, endMs] from request open; ABSENT means none, never []
	error?: {
		// Structured error info when this call failed
		message: string
		type?: string // e.g. "rate_limit_error", "invalid_request_error"
		statusCode?: number // HTTP status code
		stack?: string // Stack trace
	}
	wireRequest?: string // Serialised wire-level request metadata (JSON)
	_tokensEstimated?: true // Present when tokens are char/4 heuristic
}

interface JsonExportToolCall {
	name: string
	id: string
	input: Record<string, unknown>
	result?: {
		content: unknown
		isError: boolean
	}
}

Example

{
	"version": 1,
	"taskId": "019e127e-e412-75c6-b811-087dd1126e83",
	"task": "1+1",
	"mode": "code",
	"createdAt": "2026-05-10T15:26:18.085Z",
	"calls": [
		{
			"index": 1,
			"apiProtocol": "openai",
			"inputTokens": 450,
			"outputTokens": 12,
			"cacheWriteTokens": 0,
			"cacheReadTokens": 0,
			"costUsd": 0.0005,
			"messages": [
				{ "role": "user", "content": [{ "type": "text", "text": "1+1" }] },
				{
					"role": "assistant",
					"content": [
						{ "type": "reasoning", "text": "The user asked..." },
						{ "type": "text", "text": "2" }
					]
				}
			],
			"toolCalls": [],
			"reasoning": "The user asked \"1+1\" which equals 2..."
		}
	],
	"totalTokens": { "input": 450, "output": 12, "cacheWrite": 0, "cacheRead": 0 },
	"totalCostUsd": 0.0005,
	"totalCalls": 1,
	"totalToolCalls": 0
}

Token Estimation

When the LLM provider does not emit usage chunks in streaming mode (e.g., some OpenAI-compatible providers), token counts are estimated using a character/4 heuristic and the call is marked with "_tokensEstimated": true. When usage data is available from the provider, real values are used without this flag.

Error Calls

API calls that fail entirely (connection errors, rate limits, empty streams) are still included as call entries. These entries have:

  • Empty messages: [] and toolCalls: []
  • Structured error object with message, type, statusCode, and stack
  • wireRequest showing what was about to be sent
  • retryAttempt indicating how many retries preceded this attempt

This ensures the trace shows every attempted API call, not just the successful ones.

Call Duration

Each call may include durationMs — whole milliseconds from request open to stream end, measured with a monotonic clock (see api-req-timing.ts). It is present exactly when the request reached a stream end (including cancelled and failed streams); a call still in flight when the task was persisted, or one the host died holding, carries no durationMs at all — absence means no end is known, never "took zero time".

Call Phases

The same stream-end rewrites split that duration into the phases a reader can act on, using the marks the streaming loop already records for the api_req_finished span — nothing is measured twice:

  • firstChunkMs — request open → the first stream chunk of any kind. Absent when the stream produced nothing at all (an immediate provider failure): there was no first byte, and zero would claim there was one. This is the HOST's measurement. It is deliberately not called ttfbMs, because the api_req_started payload already carries a ttfbMs that shofer-router reports about its own request to the provider — a different observer.
  • reasoningIntervalsMs — EVERY window the model spent reasoning, as [startMs, endMs] pairs on the same open-relative basis as firstChunkMs. A model that interleaves reasoning with output produces several, in arrival order; a window the stream ended inside is closed at durationMs, because that time genuinely was reasoning and an interval must never run past the call's own end. Absence means no reasoning was observed — an empty array is never written.
  • thinkingMs — the SUM of those intervals: how much of the call was reasoning, not where. Present exactly when reasoningIntervalsMs is, and never zero, so a consumer that segments a call into waiting / thinking / output reads "no thinking phase" straight off the field being missing.

The rest of the call — durationMs - firstChunkMs - thinkingMs — is output. Drawn as a bar it is firstChunkMs of waiting, then the intervals alternating with output, then whatever remains. Each figure is rounded independently, so a consumer must clamp the segments to the duration rather than assume they fit exactly.

Wire Request

Each call may include a wireRequest field — a serialised JSON snapshot captured just before the HTTP call. It contains:

  • The model ID and API protocol
  • System prompt length and a truncated head (first 500 chars)
  • Number of messages and tools sent
  • The full normalised message payload and tool definitions

This is useful for diagnosing what data was actually transmitted to the provider, especially when comparing against error responses.

Design & Implementation

Data Flow

flowchart TB
    WV["Webview — TaskActions.tsx<br/>exportCurrentTask / exportCurrentTaskJson"]
    WMH["webviewMessageHandler.ts<br/>routes to provider.exportTaskWithId(id)<br/>and provider.exportTaskWithIdJson(id)"]
    SP["ShoferProvider.ts<br/>getTaskWithId(id) returns historyItem,<br/>apiConversationHistory, uiMessagesFilePath"]
    MDP["exportTaskWithId()<br/>downloadTask(ts, apiConversationHistory, defaultUri)"]
    JSONP["exportTaskWithIdJson()<br/>reads ui_messages.jsonl via readTaskMessages()<br/>buildJsonTrace(...)<br/>downloadJsonTask(ts, trace, defaultUri)"]
    MD["export-markdown.ts<br/>formatContentBlockToMd()<br/>downloadTask()"]
    JS["export-json.ts<br/>buildJsonTrace()<br/>getJsonExportFileName()<br/>downloadJsonTask()<br/>estimateTokens()<br/>estimateMessageTokens()"]

    WV -->|postMessage| WMH --> SP
    SP --> MDP --> MD
    SP --> JSONP --> JS
Loading

The JSON exporter resolves the default save filename before calling downloadJsonTask() (via getTaskFileName()), and reads ui_messages.jsonl through the JSONL reader readTaskMessages() wrapped in a try/catch that falls back to an empty uiMessages array — a defensive guard against missing or unreadable data. (There is no separate fs.stat() pre-check; the reader returns [] when the file is absent.)

Serialization (off the main thread)

A JSON export is the entire descendant task tree — the exported task plus every sub-task, each contributing its full apiConversationHistory. That trace can reach many megabytes, so two steps that were once inline on the extension-host thread are now handled carefully:

  1. Serialize + write run in a worker thread. downloadJsonTask() calls stringifyJsonToFile(), which hands the trace to a workerpool worker (workers/exportJson.ts) that does the JSON.stringify(…, null, 2) and the file write, returning only a byte count. This keeps the heavy serialization and the big-string round-trip off the event loop so the webview stays responsive (mirrors how countTokens offloads tokenization). If the worker can't be spawned or errors, it falls back to an in-process write. The write runs inside a withProgress("Writing JSON export…") notification — indeterminate (JSON.stringify is atomic), but it actually animates now that the main thread is free.
  2. Large exports are not auto-opened. Opening a multi-MB JSON document makes VS Code tokenize/fold it on the UI thread, which is itself a freeze. Files at or below LARGE_EXPORT_BYTES (5 MB) still auto-open in a preview tab; above it, an information message offers Open / Reveal in File Explorer instead.

One residual main-thread cost remains: structured-cloning the trace object into the worker is O(n). It is far cheaper than the former stringify + pretty-print + Buffer.from + editor-open chain, but not zero.

Data Sources

All three source files live under {globalStorageUri}/tasks/{taskId}/:

File Written by Contents
history_item.json TaskHistoryStore HistoryItem — task metadata (id, task, mode, ts, tokens, cost, size)
api_conversation_history.jsonl Task.addToApiConversationHistory() → appendApiMessage() (append-only JSONL; compacted via saveApiConversationHistory()) Anthropic.Messages.MessageParam[] — full message history with tool_use, tool_result, reasoning, thinking, thoughtSignature blocks
ui_messages.jsonl Task.addToShoferMessages() / updateShoferMessage() → appendTaskMessage() (append-only JSONL; compacted via saveTaskMessages()) ShoferMessage[] — UI-level messages including api_req_started entries with per-call metadata

Assembly of the JSON trace, with the guards that keep a partial export alive rather than losing the whole thing (dashed = fallback path):

flowchart TB
    subgraph DISK["{globalStorageUri}/tasks/{taskId}/"]
        direction TB
        F1["history_item.json<br/>HistoryItem — TaskHistoryStore"]
        F2["api_conversation_history.jsonl<br/>MessageParam[]"]
        F3["ui_messages.jsonl<br/>ShoferMessage[] incl. api_req_started"]
    end

    GET["getTaskWithId(id)<br/>called without skipApiHistory,<br/>so the full history is returned"]
    RA["readApiMessages()<br/>append-only log deduped by ts"]
    RT["readTaskMessages()<br/>inside a try/catch"]
    EMPTY["uiMessages = []<br/>partial export survives"]

    BUILD["buildJsonTrace(...)<br/>JsonExportTrace"]
    NAME["getTaskFileName() + resolveDefaultSaveUri()"]
    DL["downloadJsonTask(ts, trace, defaultUri)"]
    WORK["stringifyJsonToFile()<br/>workerpool worker — stringify + write"]
    INPROC["in-process write"]
    SIZE{"bytes &gt; LARGE_EXPORT_BYTES<br/>5 MB?"}
    OPEN["auto-open in a preview tab"]
    MSG["information message:<br/>Open / Reveal in File Explorer"]

    F1 --> GET
    F2 --> RA --> GET
    F3 --> RT
    RT -->|"read ok"| BUILD
    RT -.->|"missing or unreadable"| EMPTY
    EMPTY -.-> BUILD
    GET --> BUILD
    BUILD --> NAME --> DL --> WORK
    WORK -.->|"worker unavailable or errors"| INPROC
    WORK --> SIZE
    INPROC --> SIZE
    SIZE -->|"no"| OPEN
    SIZE -->|"yes"| MSG
Loading

The exporters receive these via ShoferProvider.getTaskWithId(), which constructs the task-directory file paths using constants from GlobalFileNames and reads api_conversation_history.jsonl via readApiMessages() (the append-only log is deduped by ts on read). Note the skipApiHistory option exists for cold task-switch but the export paths call getTaskWithId(id) without it, so they always receive the full apiConversationHistory. The JSON export additionally reads ui_messages.jsonl via readTaskMessages(). Export path resolution and last-path persistence is handled by resolveDefaultSaveUri / saveLastExportPath.

Call Partitioning

buildJsonTrace() partitions apiConversationHistory by assistant message boundaries:

  1. Walk the message array sequentially
  2. Each assistant message closes an API call
  3. Collect all messages from currentCallStart through the assistant message
  4. Match with the api_req_started entry at the same callIndex

Error-Only Calls: API calls that never received an assistant response (connection failures, rate limits, empty streams) are handled by a post-loop while block that catches any unmatched api_req_started entries. These get call entries with messages: [], toolCalls: [] but still carry their error, wire request, and metadata.

flowchart TB
    START["walk apiConversationHistory sequentially"]
    MSG{"assistant message?"}
    ACC["accumulate into the current call<br/>from currentCallStart"]
    CLOSE["close the call<br/>collect currentCallStart..assistant"]
    MATCH["match the api_req_started entry<br/>at the same callIndex"]
    CALL["JsonExportCall<br/>messages, toolCalls, reasoning, metadata"]
    END{"more messages?"}
    POST["post-loop while block:<br/>unmatched api_req_started entries"]
    ERRC["error-only JsonExportCall<br/>messages: [] · toolCalls: []<br/>keeps error, wireRequest, metadata"]
    TRACE["JsonExportTrace.calls"]

    START --> MSG
    MSG -->|"no"| ACC --> END
    MSG -->|"yes"| CLOSE --> MATCH --> CALL --> END
    END -->|"yes"| MSG
    END -->|"no"| POST --> ERRC
    CALL --> TRACE
    ERRC --> TRACE
Loading

How Per-Call Metadata Is Captured

The JSON exporter reads per-call metadata from api_req_started ShoferMessages (ShoferApiReqInfo written in Task.ts), parsed as UiApiReqStartedPayload. The three representations — ShoferApiReqInfo (write), UiApiReqStartedPayload (read/parse), JsonExportCall (export output) — must stay in lock-step; adding a field to one without the others silently drops that field from exports.

flowchart LR
    W["ShoferApiReqInfo — write<br/>Task.ts emits api_req_started"]
    SW["snapshotWireRequest()<br/>snapshotApiReqError()<br/>merge into the last api_req_started"]
    P["ui_messages.jsonl<br/>persisted ShoferMessage"]
    R["UiApiReqStartedPayload — read/parse<br/>export-json.ts"]
    O["JsonExportCall — output<br/>a field missing here is silently dropped"]

    W --> SW --> P --> R --> O
Loading

Token Estimation Trigger: The char/4 fallback in estimateTokens() fires when calls.every(c => c.inputTokens === 0 && c.outputTokens === 0) — i.e., whenever ALL calls have zero tokens, regardless of cause (all error-only calls, or a provider that emits usage but the capture failed).

Wire Traffic Capture

Before each this.api.createMessage() call in attemptApiRequest(), a JSON snapshot is captured containing:

  • Model ID and API protocol
  • System prompt length + truncated head (first 500 chars)
  • Number of messages and tools sent
  • Full normalised message payload
  • Tool definitions (if any)

The snapshot is stored via:

  1. snapshotWireRequest() merges the wire request JSON into the last api_req_started ShoferMessage
  2. The message is persisted to ui_messages.jsonl via the append/compaction save path
  3. The JSON export reads it as wireRequest and surfaces it as-is

Trade-offs:

  • Pro: Captures what was actually sent without modifying provider handlers
  • Con: Does not capture raw HTTP headers, status codes, or provider-specific wire format (messages are always in normalised Anthropic format)

Error Capture

Structured errors use ApiReqError (defined in vscode-extension-host.ts):

interface ApiReqError {
	message: string // Human-readable error message
	type?: string // e.g. "rate_limit_error", "invalid_request_error"
	statusCode?: number // HTTP status code
	stack?: string // Stack trace at the point of error
}

Capture points:

Location When Method
attemptApiRequest first-chunk catch Provider errors, context window exceeded, connection failures snapshotApiReqError(this.buildApiReqError(error))
recursivelyMakeShoferRequests mid-stream catch Stream interruption, tool execution failures snapshotApiReqError(this.buildApiReqError(error)) — only for non-user-cancelled

Both capture points call snapshotApiReqError() which merges the structured error into the last api_req_started ShoferMessage, then persisted to ui_messages.jsonl.

Implementation Files

File Role
export-markdown.ts Markdown formatter
export-json.ts JSON formatter, call partitioning, token estimation
ShoferProvider.ts exportTaskWithId() and exportTaskWithIdJson() entry points
webviewMessageHandler.ts exportCurrentTask, exportCurrentTaskJson, exportTaskWithId, exportTaskWithIdJson handlers
TaskActions.tsx Export buttons in the task header
export.ts Export path resolution and last-path persistence
workers/exportJson.ts Worker thread — off-main-thread JSON.stringify + file write
utils/exportJsonWorker.ts stringifyJsonToFile() — worker-pool wrapper with synchronous in-process fallback

Version History

Version Date Changes
2.73.0 2026-08-26 Added reasoningIntervalsMs (every reasoning window, not just the first); thinkingMs is now their sum
2.72.0 2026-08-26 Added firstChunkMs and thinkingMs (the phase split inside durationMs; thinkingMs absent = no reasoning)
2.71.0 2026-08-25 Added durationMs (request open → stream end; absent when no stream end was reached)
0.11.7 2026-05-16 Added error, retryAttempt, wireRequest fields; fixed field name mismatch; added error-only call handling
0.11.6 2026-05-14 Initial JSON export with api_req_started-based metadata

Comparison with Chrome Extension Trace

The JSON export mirrors the chrome-extension's in-task trace format (sidepanel.js). Key differences:

Shofer JSON Chrome Extension JSON
Trigger Manual export from UI Manual button in task panel
Data source Persisted files (offline-safe) In-memory accumulation during task
Token source Provider usage chunks or char/4 estimate Provider usage chunks
Cost Calculated from tokens × model pricing Calculated from tokens × model pricing
Tool results Extracted from next user message's tool_result blocks Captured in real-time as tool calls complete
LLM call metadata From api_req_started entries From live event stream

Gaps and Known Issues

Source comment inaccuracy — ✅ fixed

The apiProtocol field comment in export-json.ts said (e.g. "anthropic", "openai-native"). Runtime values from getApiProtocol() are only "anthropic" and "openai"; "openai-native" is a provider name, never an apiProtocol. Comment corrected.

Markdown example accuracy

The Markdown example (§Format) wraps user messages in <user_message> XML tags. The actual exporter writes raw content without XML wrapping. The example is illustrative; consider replacing it with verbatim sample output.

Missing test coverage

export-markdown.ts has export-markdown.spec.ts covering the formatContentBlockToMarkdown formatter, but export-json.ts has no dedicated tests — the JSON trace builder (buildJsonTrace) and the higher-level downloadTask() / exportTaskWithIdJson() paths are untested. buildJsonTrace is a pure function amenable to unit testing.

Token estimation heuristic

The char/4 heuristic in estimateTokens() is not calibrated against any provider's tokeniser and may drift significantly for non-English text or code-heavy messages. The _tokensEstimated flag is a reasonable signal but consumers should treat estimated counts as diagnostic, not authoritative.

Tool result extraction fragility

Tool results are extracted from the next user message's tool_result blocks (export-json.ts:261). If the conversation has been truncated or the tool-result ordering is non-standard, results may be silently missing from the export.

Concurrent read/write

When exportTaskWithIdJson reads ui_messages.jsonl (via readTaskMessages()) while the live task is appending to it, there is a race between the export read and the live-task append/compaction. No locking or snapshot mechanism prevents this. Because the read is line-oriented and deduped by ts, a concurrent append is more forgiving than the old full-file JSON.parse (a half-written trailing line is tolerated and dropped), but a concurrent tmp → rename compaction can still surface a transient read error — caught by the try/catch around the read, which falls back to an empty uiMessages array.

Schema version mismatch

The persisted source files (api_conversation_history.jsonl, ui_messages.jsonl, history_item.json) have no embedded version field. Exporting a task persisted by an older Shofer version may produce a trace whose shape the current buildJsonTrace() cannot parse correctly. (Legacy pre-JSONL *.json snapshots are unlinked on first read per the hard cutover in GlobalFileNames, so such tasks read as empty rather than mis-parsing.)

Missing cross-file dependencies

The ExtendedContentBlock union type used by export-json.ts is imported from export-markdown.ts. This cross-file dependency is implicit and undocumented.