Detect partner dropout and exit the survivor with a payment path (C&WG) - #17
Open
htsukamoto5 wants to merge 2 commits into
Open
Detect partner dropout and exit the survivor with a payment path (C&WG)#17htsukamoto5 wants to merge 2 commits into
htsukamoto5 wants to merge 2 commits into
Conversation
…t path (#5) Closes #5. The survivor of a dropout did not hang, they ground: round_timeout arms at round start, so when it fires the round ends and the timeline advances to the next one with a fresh timer. Five more rounds at 180s is ~15 minutes staring at a fully interactive board that never responds, with no copy anywhere telling them what happened. Then — per the #14 review — they reached "Game complete!", were told they scored 0 / 0 across 0 trials, and were handed the `complete` code. Two payments burnt and a support ticket, from one dropout. Detection does not use round_timeout alone. It is an unconditional wall-clock bound, so it fires identically for a partner who left and a pair who are merely slow — and trial 1 of a 12-figure board is the longest in the study, so a genuine timeout there is normal. Counting timeouts would abort real dyads at exactly the point the effect is largest. A round is therefore SILENT only if it timed out AND the partner sent nothing during it. The plugin records message_count (every message that round) and messages_sent (mine), on a per-round channel, so the difference is the partner's activity in that round alone. Verified against the bundle rather than assumed. Two consecutive silent rounds trips the abort. That makes the threshold safe at 2. The costs are asymmetric — a false positive ends a live dyad and pays two partial codes, a false negative leaves someone grinding — and requiring partner silence is what buys the low threshold: a slow dyad that is still talking is never aborted. On trip: record the outcome, abort the enclosing timeline (gameLoop's own conditional_function is evaluated once, before the first round, so it cannot end the loop mid-game), flush data, and show an honest screen that says it was not their fault, that payment is unaffected, how many rounds they completed, and offers the partner_dropped code. The submit button is wired before the flush resolves, so the redirect never waits on OSF. Also fixes an existing miscoding this made visible: doneTrial ran unconditionally, so a SPECTATOR — who had just been told the game was full — fell through to "Game complete!" reporting 0 / 0 and received the `complete` code. Both terminal screens are now conditional and mutually exclusive. Records ended_reason, n_trials_completed, n_trials_scheduled and dropout_detected_at_round on every row, so a partial dyad can be filtered on (#8) and so "how did this session end" survives in exactly the partial data most likely to be all we have. Adds tests/dropout.test.mjs: 16 checks driving the real detector, extracted from the file rather than re-implemented. Weighted toward false positives, since that is the direction that silently destroys good data — a clean game, a slow-but-talking dyad, non-consecutive silences, a partner who speaks again, and a partner talking while I am silent must all never abort. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
htsukamoto5
added a commit
that referenced
this pull request
Jul 31, 2026
fix(cwg): guard the dropout detector's silent config dependencies Review of #17 turned up three things worth fixing in place. `recordOutcome` is now first-writer-wins. `addProperties` applies retroactively to every row already collected, so a second call would not add a row -- it would rewrite the first outcome on every existing row and destroy the reason the session actually ended. Only one call site can fire today; this makes that structural rather than a property of the current routing. Two config couplings are now asserted in tests rather than assumed, because neither fails loudly at runtime -- the detector keeps running and quietly stops detecting. `chat_persists` must stay off or the plugin switches the chat channel from per-round to session-wide, `message_count` becomes cumulative, `partnerMessages` never returns to zero, and the survivor grinds through five dead rounds with every other check still green. And `ended_by: "timeout"` means "the round clock ran out" only while `selection_timeout` is unset; the plugin emits the same string for a selection timeout. The CONFIG comment also now states the false positive the silence guard does not close, instead of implying there isn't one. A matcher can be actively placing tangrams and typing nothing, and `save_interaction_history` is matcher-local until submit, so from the director's side that is identical to a partner who has gone. It records the pilot check for it and, more importantly, that the fix if it shows up is a positive-evidence signal (#11), NOT a higher threshold -- which would buy safety with dead minutes and leave the ambiguity intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> @
Review of #17 turned up three things worth fixing in place. `recordOutcome` is now first-writer-wins. `addProperties` applies retroactively to every row already collected, so a second call would not add a row -- it would rewrite the first outcome on every existing row and destroy the reason the session actually ended. Only one call site can fire today; this makes that structural rather than a property of the current routing. Two config couplings are now asserted in tests rather than assumed, because neither fails loudly at runtime -- the detector keeps running and quietly stops detecting. `chat_persists` must stay off or the plugin switches the chat channel from per-round to session-wide, `message_count` becomes cumulative, `partnerMessages` never returns to zero, and the survivor grinds through five dead rounds with every other check still green. And `ended_by: "timeout"` means "the round clock ran out" only while `selection_timeout` is unset; the plugin emits the same string for a selection timeout. The CONFIG comment also now states the false positive the silence guard does not close, instead of implying there isn't one. A matcher can be actively placing tangrams and typing nothing, and `save_interaction_history` is matcher-local until submit, so from the director's side that is identical to a partner who has gone. It records the pilot check for it and, more importantly, that the fix if it shows up is a positive-evidence signal (#11), NOT a higher threshold -- which would buy safety with dead minutes and leave the ambiguity intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
htsukamoto5
force-pushed
the
feat/dropout-abort
branch
from
July 31, 2026 19:54
1853841 to
8b133fc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5.
Stacked on #16 (
feat/datapipe-egress-and-identifiers), which is stacked on #14. It needs #16'sPipeline.flush()and #14'ssubmissionBlockHTML/wireSubmissionButton, so review order is #14 → #16 → this. The diff here is only the dropout work.The problem
The survivor of a dropout did not hang — they ground, which is worse, because it burns their time before they give up.
round_timeoutarms at round start, so when it fires the round ends and the timeline advances to the next one with a fresh timer. Five more rounds at 180s is ~15 minutes in front of a fully interactive board that never responds, with no copy anywhere telling them what happened.And then, as the #14 review pointed out, it got worse: they reached "Game complete!", were told they scored 0 / 0 across 0 trials, and were handed the
completecode. One dropout burnt two payments, miscoded the reconciliation, and generated a support ticket.Detection: silence, not timeouts
round_timeoutalone is not the signal. It is an unconditional wall-clock bound (corrected inc2c69c2on the base branch), so it fires identically for a partner who left and a pair who are simply slow — and trial 1 of a 12-figure board is the longest in the study. Counting timeouts would abort real dyads at exactly the point the effect is largest.So a round is silent only if it timed out and the partner sent nothing during it. The plugin records
message_count(every message that round) andmessages_sent(mine) on a per-round chat channel, so the difference is the partner's activity in that round alone. Verified against the vendored bundle rather than assumed:CONFIG.DROPOUT_SILENT_ROUNDS: 2— suggested, not settled. Requiring partner silence is what makes 2 safe rather than reckless: a slow dyad that is still talking is never aborted, so the threshold does not have to buy its safety with more dead minutes. At 180s that releases a survivor after ~6 minutes instead of 15.The costs are asymmetric and worth stating explicitly: a false positive ends a live dyad and pays two partial codes; a false negative leaves a survivor grinding. The tests are weighted toward the false-positive side, because that is the direction that silently destroys good data.
On trip
jsPsych.abortCurrentTimeline().gameLoop's ownconditional_functionis evaluated once, before the first round, so it cannot end the loop mid-game — this is what actually stops the remaining rounds.partner_droppedcode.Also fixes an existing miscoding
doneTrialran unconditionally, so a spectator — who had just been told the game was full — fell through to "Game complete!" reporting 0 / 0 and received thecompletecode. That predates this PR; the survivor path just made it visible. Both terminal screens are now conditional and mutually exclusive, so exactly one runs and each carries its own code.Data
ended_reason,n_trials_completed,n_trials_scheduled,dropout_detected_at_roundon every row.n_trials_completedcounts onlyended_by: "submit"— a timed-out round has a null assignment and is not a trial. This is what lets a partial dyad be filtered rather than merely present (#8), and it records how a session ended in exactly the partial data most likely to be all we have.Testing
node tests/dropout.test.mjs— 16 checks, no dependencies. The detector is extracted from the experiment file and driven with synthetic round data, so it is tested as written rather than as re-implemented in the test.Never aborts: a clean 6-round game; a slow-but-talking dyad across three timed-out rounds; a single silent round; non-consecutive silences; a partner who speaks again after a silence; a partner talking while I am silent. Does abort: two consecutive silent rounds, mid-game dropout after real trials, and exactly once rather than per subsequent round. Plus the configurable threshold and degenerate data with the chat fields missing entirely.
node tests/pipeline.test.mjsstill passes 39/39.Not in here
partner_droppedcode is still an empty placeholder until Create the DataPipe experiment and a draft Prolific study — critical path #15, so this path warns at startup and shows the configuration-error screen rather than a code.🤖 Generated with Claude Code