Skip to content

Detect partner dropout and exit the survivor with a payment path (C&WG) - #17

Open
htsukamoto5 wants to merge 2 commits into
feat/datapipe-egress-and-identifiersfrom
feat/dropout-abort
Open

Detect partner dropout and exit the survivor with a payment path (C&WG)#17
htsukamoto5 wants to merge 2 commits into
feat/datapipe-egress-and-identifiersfrom
feat/dropout-abort

Conversation

@htsukamoto5

Copy link
Copy Markdown
Member

Closes #5.

Stacked on #16 (feat/datapipe-egress-and-identifiers), which is stacked on #14. It needs #16's Pipeline.flush() and #14's submissionBlockHTML / wireSubmissionButton, so review order is #14#16 → this. The diff here is only the dropout work.

The problem

The survivor of a dropout did not hang — they ground, which is worse, because it burns their time before they give up. round_timeout arms at round start, so when it fires the round ends and the timeline advances to the next one with a fresh timer. Five more rounds at 180s is ~15 minutes in front of a fully interactive board that never responds, with no copy anywhere telling them what happened.

And then, as the #14 review pointed out, it got worse: they reached "Game complete!", were told they scored 0 / 0 across 0 trials, and were handed the complete code. One dropout burnt two payments, miscoded the reconciliation, and generated a support ticket.

Detection: silence, not timeouts

round_timeout alone is not the signal. It is an unconditional wall-clock bound (corrected in c2c69c2 on the base branch), so it fires identically for a partner who left and a pair who are simply slow — and trial 1 of a 12-figure board is the longest in the study. Counting timeouts would abort real dyads at exactly the point the effect is largest.

So a round is silent only if it timed out and the partner sent nothing during it. The plugin records message_count (every message that round) and messages_sent (mine) on a per-round chat channel, so the difference is the partner's activity in that round alone. Verified against the vendored bundle rather than assumed:

message_count: g.length,
messages_sent: g.filter(m => m.senderId === b).length

CONFIG.DROPOUT_SILENT_ROUNDS: 2 — suggested, not settled. Requiring partner silence is what makes 2 safe rather than reckless: a slow dyad that is still talking is never aborted, so the threshold does not have to buy its safety with more dead minutes. At 180s that releases a survivor after ~6 minutes instead of 15.

The costs are asymmetric and worth stating explicitly: a false positive ends a live dyad and pays two partial codes; a false negative leaves a survivor grinding. The tests are weighted toward the false-positive side, because that is the direction that silently destroys good data.

On trip

  1. Record the outcome and the round it fired on.
  2. jsPsych.abortCurrentTimeline(). gameLoop's own conditional_function is evaluated once, before the first round, so it cannot end the loop mid-game — this is what actually stops the remaining rounds.
  3. Show an honest screen: what happened, that it was not their fault, that payment is unaffected, how many rounds they completed, and the partner_dropped code.
  4. Flush data. The submit button is wired before the flush resolves, so the redirect never waits on OSF.

Also fixes an existing miscoding

doneTrial ran unconditionally, so a spectator — who had just been told the game was full — fell through to "Game complete!" reporting 0 / 0 and received the complete code. That predates this PR; the survivor path just made it visible. Both terminal screens are now conditional and mutually exclusive, so exactly one runs and each carries its own code.

Data

ended_reason, n_trials_completed, n_trials_scheduled, dropout_detected_at_round on every row. n_trials_completed counts only ended_by: "submit" — a timed-out round has a null assignment and is not a trial. This is what lets a partial dyad be filtered rather than merely present (#8), and it records how a session ended in exactly the partial data most likely to be all we have.

Testing

node tests/dropout.test.mjs16 checks, no dependencies. The detector is extracted from the experiment file and driven with synthetic round data, so it is tested as written rather than as re-implemented in the test.

Never aborts: a clean 6-round game; a slow-but-talking dyad across three timed-out rounds; a single silent round; non-consecutive silences; a partner who speaks again after a silence; a partner talking while I am silent. Does abort: two consecutive silent rounds, mid-game dropout after real trials, and exactly once rather than per subsequent round. Plus the configurable threshold and degenerate data with the chat fields missing entirely.

node tests/pipeline.test.mjs still passes 39/39.

Not in here

🤖 Generated with Claude Code

…t path (#5)

Closes #5.

The survivor of a dropout did not hang, they ground: round_timeout arms at
round start, so when it fires the round ends and the timeline advances to the
next one with a fresh timer. Five more rounds at 180s is ~15 minutes staring at
a fully interactive board that never responds, with no copy anywhere telling
them what happened. Then — per the #14 review — they reached "Game complete!",
were told they scored 0 / 0 across 0 trials, and were handed the `complete`
code. Two payments burnt and a support ticket, from one dropout.

Detection does not use round_timeout alone. It is an unconditional wall-clock
bound, so it fires identically for a partner who left and a pair who are merely
slow — and trial 1 of a 12-figure board is the longest in the study, so a
genuine timeout there is normal. Counting timeouts would abort real dyads at
exactly the point the effect is largest.

A round is therefore SILENT only if it timed out AND the partner sent nothing
during it. The plugin records message_count (every message that round) and
messages_sent (mine), on a per-round channel, so the difference is the
partner's activity in that round alone. Verified against the bundle rather
than assumed. Two consecutive silent rounds trips the abort.

That makes the threshold safe at 2. The costs are asymmetric — a false positive
ends a live dyad and pays two partial codes, a false negative leaves someone
grinding — and requiring partner silence is what buys the low threshold: a slow
dyad that is still talking is never aborted.

On trip: record the outcome, abort the enclosing timeline (gameLoop's own
conditional_function is evaluated once, before the first round, so it cannot
end the loop mid-game), flush data, and show an honest screen that says it was
not their fault, that payment is unaffected, how many rounds they completed,
and offers the partner_dropped code. The submit button is wired before the
flush resolves, so the redirect never waits on OSF.

Also fixes an existing miscoding this made visible: doneTrial ran
unconditionally, so a SPECTATOR — who had just been told the game was full —
fell through to "Game complete!" reporting 0 / 0 and received the `complete`
code. Both terminal screens are now conditional and mutually exclusive.

Records ended_reason, n_trials_completed, n_trials_scheduled and
dropout_detected_at_round on every row, so a partial dyad can be filtered on
(#8) and so "how did this session end" survives in exactly the partial data
most likely to be all we have.

Adds tests/dropout.test.mjs: 16 checks driving the real detector, extracted
from the file rather than re-implemented. Weighted toward false positives,
since that is the direction that silently destroys good data — a clean game, a
slow-but-talking dyad, non-consecutive silences, a partner who speaks again,
and a partner talking while I am silent must all never abort.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
htsukamoto5 added a commit that referenced this pull request Jul 31, 2026
fix(cwg): guard the dropout detector's silent config dependencies

Review of #17 turned up three things worth fixing in place.

`recordOutcome` is now first-writer-wins. `addProperties` applies retroactively
to every row already collected, so a second call would not add a row -- it would
rewrite the first outcome on every existing row and destroy the reason the
session actually ended. Only one call site can fire today; this makes that
structural rather than a property of the current routing.

Two config couplings are now asserted in tests rather than assumed, because
neither fails loudly at runtime -- the detector keeps running and quietly stops
detecting. `chat_persists` must stay off or the plugin switches the chat channel
from per-round to session-wide, `message_count` becomes cumulative,
`partnerMessages` never returns to zero, and the survivor grinds through five
dead rounds with every other check still green. And `ended_by: "timeout"` means
"the round clock ran out" only while `selection_timeout` is unset; the plugin
emits the same string for a selection timeout.

The CONFIG comment also now states the false positive the silence guard does not
close, instead of implying there isn't one. A matcher can be actively placing
tangrams and typing nothing, and `save_interaction_history` is matcher-local
until submit, so from the director's side that is identical to a partner who
has gone. It records the pilot check for it and, more importantly, that the fix
if it shows up is a positive-evidence signal (#11), NOT a higher threshold --
which would buy safety with dead minutes and leave the ambiguity intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@
Review of #17 turned up three things worth fixing in place.

`recordOutcome` is now first-writer-wins. `addProperties` applies retroactively
to every row already collected, so a second call would not add a row -- it would
rewrite the first outcome on every existing row and destroy the reason the
session actually ended. Only one call site can fire today; this makes that
structural rather than a property of the current routing.

Two config couplings are now asserted in tests rather than assumed, because
neither fails loudly at runtime -- the detector keeps running and quietly stops
detecting. `chat_persists` must stay off or the plugin switches the chat channel
from per-round to session-wide, `message_count` becomes cumulative,
`partnerMessages` never returns to zero, and the survivor grinds through five
dead rounds with every other check still green. And `ended_by: "timeout"` means
"the round clock ran out" only while `selection_timeout` is unset; the plugin
emits the same string for a selection timeout.

The CONFIG comment also now states the false positive the silence guard does not
close, instead of implying there isn't one. A matcher can be actively placing
tangrams and typing nothing, and `save_interaction_history` is matcher-local
until submit, so from the director's side that is identical to a partner who has
gone. It records the pilot check for it and, more importantly, that the fix if
it shows up is a positive-evidence signal (#11), NOT a higher threshold -- which
would buy safety with dead minutes and leave the ambiguity intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant