Jack position management - #17
Open
SunriseSunsetCoder wants to merge 68 commits into
Open
Conversation
added jack database
Jack v1.2 backfill
Session A: persist each JACK validation run to SQLite (better-sqlite3) on
the VPS while keeping the Vercel deploy green.
- lib/db/: schema.sql (setups/validation_runs/decisions/outcomes + view),
init.ts (WAL singleton), write.ts, read.ts, env.ts (persistence gate)
- route.ts: extract Claude's JSON block, persist run+setups+decisions,
banner shows "Persistence: run #N · N setups tracked · N decisions".
Vercel guard: type-only imports + lazy require("@/lib/db/write") inside
persistRun, skipped entirely when !isPersistenceAvailable() (VERCEL=1 or
JACK_DISABLE_PERSISTENCE=1) -> banner "disabled (running on Vercel)"
- prompt: append v1.3 MACHINE-READABLE JSON block spec (schema_version 1.3)
- next.config: serverExternalPackages ["better-sqlite3"] (native, unbundled)
- package.json: better-sqlite3 -> optionalDependencies (Vercel install-tolerant)
Note: JSON contract is schema_version "1.3" but the HTTP wrapper
JackValidationResponse.schemaVersion stays "1.2" by design (Session B bumps it).
Verified on Win10: db layer exercised against real data/jack.db (schema +
write/read + both guard paths + exact banner strings); tsc clean for all new
files. Live endpoint click-test deferred to VPS (no ANTHROPIC_API_KEY here).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016S2RunhHGBuQkRWVCYHQJK
Jack v1.3 persistence
…8000 The validation route emitted the structured JSON decisions block AFTER the verbose markdown tables. On an 18-setup batch the output hit the token cap mid-response, cutting off the JSON, so extractJsonBlock() returned null and 0 decisions were persisted (setups still upserted pre-call). Two fixes: - Raise max_tokens floor 4000 -> 8000. The dynamic formula computed 3160 for 18 setups and floored to 4000; 8000 gives both JSON and markdown headroom. - Reorder the prompt so the model emits the fenced JSON decisions block FIRST, then the human-readable markdown. Persistence no longer depends on the markdown fitting under the cap. The parser (extractJsonBlock) and markdown-strip both match the fenced json block by content, not position, so JSON-first requires no parser change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CygESnzP53BuypDFD7NeSk
Jack output truncation fix
Every setup came back "Tiingo unavailable" (0/18) even though the Tiingo API, key, and network were confirmed working via a direct call. Root cause was internal: the EOD proxy route lived at app/api/tiingo/eod/route.ts (static path /api/tiingo/eod) but its handler reads a `ticker` dynamic param, and jack-validation calls /api/tiingo/eod/<TICKER>. That path never matched the static route, so Next returned a 404 HTML page; the enrichment code then did fetch(...).then(r => r.json()), which threw an opaque SyntaxError on the HTML and was swallowed into eodError -> unavailable for all setups. Fixes: - Move the EOD route to app/api/tiingo/eod/[ticker]/route.ts (pure git mv, matching the news route convention and the caller's URL). The handler already reads params.ticker, so no code change to the route itself. - Harden + instrument enrichSetup: fetch the raw Response and check res.ok / status instead of blindly parsing JSON, so a non-2xx no longer surfaces as a cryptic SyntaxError. Added TEMP console.warn logging of the real HTTP status/body per ticker (tagged jack-tiingo-eod-route-fix) so any remaining failure (e.g. Tiingo news paid-tier 403) is visible in the dev-server log. Env var name confirmed: both routes read process.env.TIINGO_API_KEY, matching the working manual call. No schema or persistence-guard changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CygESnzP53BuypDFD7NeSk
jack: fix Tiingo enrichment 0/N — relocate EOD route to [ticker] segment
… insertOutcome, updateUserFills, geometry upsert, decision ids
…ckward-compatible)
… timeout=mark-to-market
… decisions + persists CSV geometry
…(React state only)
Jack session b outcomes
…ure) patch() merged onto the render-snapshot `rows` closure instead of `prev` inside the setRows updater, so when handleAction fired two patches in one handler (userAction, then actionSave) the second read stale state and reverted userAction to null. That single bug produced all three symptoms: 1. clicked T/P/W button didn't stay highlighted (userAction reverted) 2. no indication which action was recorded (only a generic green check) 3. Entry/Exit/Exit-date stayed disabled on TRADED rows (isTraded was false) Fix: patch() now merges onto prev. Also add per-action color labels (TRADED green / PASSED gray / WATCHED blue) and fix a SaveSave double-render. React state only. Verified: three rows with distinct actions persist TRADED/PASSED/WATCHED, and a typed entry on a TRADED row lands in outcomes.
jack(session-b): fix decision-table T/P/W action controls (stale-clos…
Bug: Entry/Exit price fields accepted only the number-stepper, not typed input.
Two causes, both fixed:
- the global KeyboardShortcuts window keydown listener matches digit keys to
panel shortcuts and preventDefault()s them, swallowing keystrokes in the
input → add onKeyDown stopPropagation on the fill inputs.
- controlled type=number mangles partial decimals (e.g. "48.") → switch to
type=text inputMode=decimal with a sanitizePrice() filter (digits + one dot),
so "48.48" types cleanly while staying numeric.
Feature: user entry-date fill so actual holding period (entry_date → exit_date)
can be compared against the theoretical replay's timing.
- additive schema: outcomes.user_entry_date TEXT (CREATE + ensureColumns migration)
- updateUserFills() now takes entryDate; route + interactive row wire it through
- date picker mirrors the exit-date field, enabled on TRADED
React state only, no storage. Verified: sanitizePrice keeps "48.48" typable;
user_entry_price + user_entry_date both persist to outcomes on data/jack.db.
jack(session-b): fix Entry/Exit price typing + add user entry-date fill
Surface fields the expandable rows need — note (reasoning), news_class, sector_rs, cross_asset, earnings_flag, pct_to_breakout, breakout level, and current price (Tiingo eodClose) — from the already-parsed JSON decision + enriched data into JackDecisionClient / buildClientDecisions. Presentation plumbing only: no new fetch, no change to the validation pipeline, Claude call, or determinism.
…down)
Collapse the interactive decisions table AND the wide markdown Table 1/2 into
ONE expandable row per setup, in two preserved groups: LIVE (n) / PENDING (n).
Progressive disclosure removes the width problem at the root — no wide tables,
no horizontal scroll, no char-wrapping.
- Collapsed row (scannable): chevron · TICKER · JACK verdict pill
(TRADE=green/SKIP=red/WATCH=amber/FIRED=gray) · stop→target · R/R
(≥1.5 green/≥1 amber/<1 red) · action badge (✓TRADED/✓PASSED/✓WATCHED/
unmarked, re-hydrated from DB).
- Expanded row: price ladder (stop·entry·now·target) · full-width readable
reasoning + chips · T/P/W buttons · fill panel (renders ONLY on TRADED,
any section) with the existing Save fills / ✓Saved / Saving… / Retry states.
- All rows collapsed by default. jack-view keeps the collapsible input panel;
raw markdown moved behind a collapsed "Raw analysis" toggle (Copy retained).
Write paths REUSED, not rewritten: T/P/W → decisions.user_action (upsert per
setup); fills → updateUserFills (unchanged); mount re-hydration → existing
/api/jack-decisions GET → getUserMarksForSetups. React state only.
Plumb `shares` from the already-parsed JSON decision (ExtractedDecision.shares, Claude output) into JackDecisionClient / buildClientDecisions so the row can show position size + computed notional. Presentation plumbing only — no validation, fetch, or data-contract change.
…shares
- Verdict pills are now filled (black/white text) for high contrast in BOTH
light and dark themes; the previous /20 tints on colored text were unreadable.
- Sub-1.0 R/R renders as an alarming red bordered pill with ⚠ (was plain red
text) — a bad reward:risk setup now stands out.
- Collapsed row packed left (removed ml-auto) so the action badge isn't stranded
across dead horizontal space; tighter, more scannable.
- Shares shown compactly in the collapsed row ("697 sh") and, in the expanded
row near the price ladder, Shares + Notional ($ = shares × entry) so position
size vs the individual/session caps in the header reads at decision time.
Jack UI v2 followups
JACK re-validates fresh each run, so a marked setup's live verdict can flip
(GLNG TRADE→SKIP as price moves). Capture the decision-time context so Session C
(and the UI) can show it frozen:
- additive column decisions.jack_decision_at_mark (CREATE + ensureColumns
migration for existing DBs). No existing-column change.
- markDecisionUserAction now sets jack_decision_at_mark = the row's live
`decision` at the moment of marking (still an UPDATE, per-setup upsert).
- getUserMarksForSetups surfaces the frozen verdict + the marked decision's
shares (sharesAtMark) from the marked row, so marked rows show mark-time
context, not the current re-assessment.
Data contract intact: user_action / fills unchanged; jack_decision_at_mark is
new + additive. NULL≠PASSED and re-mark=UPDATE preserved. Fill-save path,
determinism, and replay untouched.
…st fixes
- Marked rows show JACK's FROZEN verdict (de-emphasized outline pill) so the
user's action badge dominates, not JACK's call. Unmarked rows keep the live
verdict pill.
- JACK's CURRENT re-assessment (when it changed post-mark) is shown as framed
position info — "⚠ changed" in the collapsed row and an amber re-assessment
banner in the expanded row ("now SKIP, was TRADE — $49.76 below entry") — NOT
as a contradictory verdict pill.
- Shares fix: use the FROZEN mark-time shares on marked rows (the current run
reports 0 for a now-SKIP setup — that's why it read "0 sh"). Moved shares to
the EXPANDED row only; hidden when 0/unknown.
- Expanded row shows notional as a fill-bar vs the individual cap (passed from
the header), e.g. "17% of indiv cap", colored green/amber/red near the cap.
- Light-mode contrast: sub-1 R/R alarm pill and verdict pills darkened so they're
legible on white (were pale-red-on-white).
Presentation + shares surfacing only. React state; no localStorage.
Jack fixes batch
Empirical root cause of "Table 1/2 render inline and char-wrap": the committed v2 source already gates the markdown behind showRaw (default false), verified on origin/main and PR #13. The symptom of v2 ROWS + inline char-wrapping TABLES can only occur when jack-view.tsx is a STALE pre-v2 file (which renders <ReactMarkdown> ungated/inline) while jack-decisions-table.tsx is the updated v2 rows file — a partial source update on the running machine. `rm -rf .next` rebuilds from that stale file, so it survives. Bulletproof fix so the failure mode cannot exist regardless of file state: remove ReactMarkdown from the JACK view entirely. The raw analysis now renders as PLAIN monospace <pre> text (whitespace-pre-wrap + break-words), still gated behind the collapsed "Raw analysis" toggle. There is no <table> in the JACK view anymore, so the wide Table 1/2 can never render as an HTML table and char-wrap. Rows are the sole default surface; Copy still yields the raw text. Presentation only. React state; no localStorage.
jack(view): render raw analysis as plain <pre>, never an HTML table
jack: backfill script for 6 reconstructed historical trades + replay …
WAL is already enabled; add a 5s busy_timeout so a write that hits a momentarily held lock (a second dev-server worker's own better-sqlite3 handle, an external DBeaver session, or a WAL checkpoint) waits and QUEUES instead of throwing SQLITE_BUSY immediately — which previously could drop a JACK fill-save landing mid validation-write. Additive one-line pragma; no schema or logic change.
…ut-v2 Jack sqlite busy timeout v2
lib/db/analytics.ts — getAnalyticsRows(): read-only join of setups+decisions+ outcomes (one row per setup with an outcome), carrying theoretical R, actual user_R, marks, jack_decision_at_mark (frozen verdict), geometry, MFE/MAE. The decision_outcomes view lacks these columns; no schema change. lib/jack/analytics.ts — pure computeAnalytics() (no DB/React → unit-testable). Encodes the modeling rules: NULL user_action != PASSED (universe yes, selection no); universe = resolved only (excl never_fired + still_open; timeout = MtM); selection keyed on the FROZEN verdict; execution delta = user_R - theoretical R with mean AND median + cause hints + outlier flag; still_open -> open-exposure strip. LOW_SAMPLE_THRESHOLD=30 (named); verdict SUPPRESSED below it.
recharts edge-over-time series + 4 views (edge-decay, universe vs selected, execution quality, decision breakdown) + open-exposure strip. n shown on every stat; n<30 flags "LOW SAMPLE"; the universe-vs-selected verdict renders "VERDICT SUPPRESSED - insufficient data" below threshold. Bloomberg language (mono, orange, v2 pill/R coloring). Wired: view union, useTerminalUI handler, JANLY header button, render branch. No new charting dep (recharts already present).
Jack session c analytics
…laude calls
Large weeks (75+ setups) overran a single Claude call's output-token cap and
truncated the response mid-stream. Fix: run TWO independent, per-section-sized
calls IN PARALLEL — LIVE with full reasoning, PENDING with a compact one-line
treatment (watchlist, not actionable) — instead of one combined call. Each pass
gets its own budget (live ~160 out-tok/setup, pending ~70), so neither hits the
ceiling regardless of week size; parallel Promise.all keeps latency flat.
NOT a filtering/ranking change: the winner/loser and ranking research both
returned NULL (no setup feature separates winners/losers or validates a ranking
beyond the staleness filter). This adds NO quality-sort or priority heuristic —
only status-priority sectioning (live before pending), already prescribed.
Preserved intact:
- Determinism: both calls temp 0 + the day-stable session context.
- JSON contract: the two passes merge into the SAME extracted shape
(schema_version, live_decisions[], pending_decisions[]) that persistRun,
extractedToDecisionRow, and buildClientDecisions already consume — Session C
analytics reads the identical decision columns. persist/replay untouched.
Verified: LIVE/PENDING prompts contain only their section + correct directive and
are deterministic; the 'both' fallback is unchanged; the merged payload keys are
identical to the single-call ExtractedPayload; no JSON fence leaks to the client.
… mode Follow-ups to the truncation split: 1. GATE ON COUNT — don't always split. A single "both" call runs ~160 output tokens/setup, so a normal ~42-setup week fits under an ~8k budget and keeps FULL analysis for BOTH live and pending. Only split (full LIVE + compact PENDING, parallel) when totalFinal > SPLIT_THRESHOLD (=(8000-1000)/160 ≈ 43), i.e. 44+ setups. The compact-pending tradeoff now applies only to the big weeks that force it. Merge is unified (flatMap) so one "both" payload or two split payloads both fold into the same extracted shape. 2. COMPACT != CONTEXT-STRIPPED — the pending directive now requires notes to carry the "why watch this" read (news/catalyst classification + sector context + the breakout trigger) and to keep news_class/sector_rs/cross_asset populated. Only the long multi-paragraph narrative is dropped; the contextual signal that makes a pending setup worth preparing for (and that Session C reads) stays. Determinism (temp 0 + day-stable context) and the JSON contract are unchanged. Verified: threshold=43; 'both' mode gives full pending with no compact directive; compact directive preserves the context fields; merge handles both call shapes.
…-split Jack output truncation split
A live 75-setup run still truncated: the LIVE pass (30 setups) blew past its cap
because the 160 tok/setup budget was far too low (real need >210). Two-layer fix:
1. Budgets sized for the actual worst case (live 500, pending 220, both 380
tok/setup) with a generous 24k ceiling (sonnet-4-5 supports far more).
2. SUB-BATCHING — each pass is chunked (<=15 live/call, <=30 pending/call) so no
single Claude call carries enough setups to approach its cap (live batch caps
at ~9.5k out-tokens). All chunks run in parallel. Truncation becomes
effectively impossible regardless of week size. Gate unchanged: <=45 setups =
one full "both" call; larger = split + sub-batch.
3. When output IS cut, never emit a garbled/partial verdict as valid: any input
setup with no returned decision gets an explicit "INCOMPLETE — RE-RUN"
placeholder (the JSON block is emitted first, so truncation normally only cuts
trailing markdown, not decisions — this is defense in depth).
SIZE DOWN 50% investigation: it is a LEGITIMATE template verdict (a risk-context
size reduction from the earnings / sector-rotation / cross-asset checks), NOT a
truncation artifact and NOT a directive bug. It is orthogonal to R/R (which is
geometry): a REIT like KRC hitting the cross-asset rate check can be SIZE DOWN 50%
at any R/R. The JSON-first ordering means the decisions survived truncation; what
was cut was the trailing markdown reasoning (why it was sized down), which the
batching fix now keeps intact. Determinism (temp 0 + day-stable context) and the
JSON contract are unchanged.
An open trade (marked TRADED, entry filled, no exit) vanished from the JACK view
when its ticker wasn't in the current week's scan — no way to record the exit
(e.g. GLNG). Add a persistent CURRENT POSITIONS section at the TOP of the working
view, populated from the DB regardless of the current run.
- read-only getOpenPositions(): setups JOIN latest-TRADED-decision JOIN outcomes
WHERE user_entry_price present AND user_exit_price NULL. No schema change.
- /api/jack-open-positions (Vercel-guarded) maps them to JackDecisionClient rows
with section "open" (frozen verdict + shares + entry fills).
- jack-view fetches them, dedups vs the current run (a setup that fired again
shows once, in its run section), and renders the table even before a run so
open positions always stay reachable.
- the decisions table renders the "open" rows first as CURRENT POSITIONS — fully
editable TRADED rows (exit price/date + Save). The fill write REUSES
updateUserFills via /api/jack-decisions — no new write path.
So an open position stays closeable across weeks until its exit is recorded.
…d-batching Jack open positions and batching
When a setup is marked TRADED, JACK's analysis TEXT was lost — the Current
Positions row showed "No JACK note". Freeze it alongside the already-frozen
verdict.
- additive column decisions.jack_analysis_at_mark (idempotent migration; no
schema.sql change, no drops).
- markDecisionUserAction now sets jack_analysis_at_mark = notes at mark time,
next to jack_decision_at_mark = decision. Immutable: a later re-VALIDATE that
flips the verdict/reasoning does NOT rewrite it (verified).
- getOpenPositions returns jackAnalysisAtMark as the "why I entered" thesis.
The live position re-read (Part C) is computed separately and never touches this
column or the persisted outcome.
…-read
Every open position now gets a live re-assessment each run (the book is only a few
positions, so per-run analysis is cheap and desired).
PART B (fast, rules-based): fetch Tiingo latest EOD close per open ticker → NOW,
unrealized % vs the entry fill, days held vs a 120-day time stop, and an
at-a-glance flag (at/near stop · at/near target · past time stop · underwater).
Pure helpers in lib/jack/position-mgmt.ts; prices cached per (ticker, ET day).
PART C (live LLM re-read): a NEW position-management prompt — DISTINCT from the
scan prompt. Input: frozen thesis + entry + current + geometry + rules. Question:
"held at [entry], now [current]; has the thesis broken (technically or
contextually)?" Output: HOLD / EXIT / REDUCE + reasoning (a hold/exit/reduce call,
NOT a trade/skip setup verdict) — this resurfaces the failed-breakout "get out"
signal with a why.
- Determinism: temperature 0 + day-stable session context (same guarantee as the
scan pipeline); re-read cached per (ET day + book signature incl. price), so
repeated GETs the same day don't re-call Claude. Batches at 10 positions.
- NEWS HONESTY: Tiingo news is unavailable here, so the directive forbids
inventing specific headlines/dates and requires any context read to be labeled
as the model's own inference, not sourced news.
- Read-only + graceful degrade: if ANTHROPIC_API_KEY is absent the frozen thesis
+ rules still render (reReadAvailable:false). No DB writes from this route.
Never conflate the frozen entry rationale with the live assessment — they are
different information. Each Current Positions row now shows, top to bottom:
1. LIVE RE-READ (LLM, prominent) — HOLD/EXIT/REDUCE badge + reasoning, with an
explicit "no live news feed — inference, not sourced headlines" footnote.
2. PRICE LADDER + NOW + unrealized % + days held (/120d) + rules flag.
3. FROZEN ENTRY THESIS (immutable, de-emphasized) — "why I entered" + the frozen
verdict pill.
Collapsed header leads with the live-read verdict + NOW/unrealized/flag; the frozen
verdict is muted. Exit-fill panel extracted to a shared helper so the open row and
TRADED scan rows use the identical updateUserFills write path. New client-type
fields are optional (open-section only); scan rows and the JSON contract unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.