From cd9a3d352a1358f2aecf2477fc81b41d8b242f9b Mon Sep 17 00:00:00 2001 From: Hannah Tsukamoto Date: Fri, 31 Jul 2026 13:50:08 -0400 Subject: [PATCH 1/2] feat(cwg): detect partner dropout and exit the survivor with a payment path (#5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #5. The survivor of a dropout did not hang, they ground: round_timeout arms at round start, so when it fires the round ends and the timeline advances to the next one with a fresh timer. Five more rounds at 180s is ~15 minutes staring at a fully interactive board that never responds, with no copy anywhere telling them what happened. Then — per the #14 review — they reached "Game complete!", were told they scored 0 / 0 across 0 trials, and were handed the `complete` code. Two payments burnt and a support ticket, from one dropout. Detection does not use round_timeout alone. It is an unconditional wall-clock bound, so it fires identically for a partner who left and a pair who are merely slow — and trial 1 of a 12-figure board is the longest in the study, so a genuine timeout there is normal. Counting timeouts would abort real dyads at exactly the point the effect is largest. A round is therefore SILENT only if it timed out AND the partner sent nothing during it. The plugin records message_count (every message that round) and messages_sent (mine), on a per-round channel, so the difference is the partner's activity in that round alone. Verified against the bundle rather than assumed. Two consecutive silent rounds trips the abort. That makes the threshold safe at 2. The costs are asymmetric — a false positive ends a live dyad and pays two partial codes, a false negative leaves someone grinding — and requiring partner silence is what buys the low threshold: a slow dyad that is still talking is never aborted. On trip: record the outcome, abort the enclosing timeline (gameLoop's own conditional_function is evaluated once, before the first round, so it cannot end the loop mid-game), flush data, and show an honest screen that says it was not their fault, that payment is unaffected, how many rounds they completed, and offers the partner_dropped code. The submit button is wired before the flush resolves, so the redirect never waits on OSF. Also fixes an existing miscoding this made visible: doneTrial ran unconditionally, so a SPECTATOR — who had just been told the game was full — fell through to "Game complete!" reporting 0 / 0 and received the `complete` code. Both terminal screens are now conditional and mutually exclusive. Records ended_reason, n_trials_completed, n_trials_scheduled and dropout_detected_at_round on every row, so a partial dyad can be filtered on (#8) and so "how did this session end" survives in exactly the partial data most likely to be all we have. Adds tests/dropout.test.mjs: 16 checks driving the real detector, extracted from the file rather than re-implemented. Weighted toward false positives, since that is the direction that silently destroys good data — a clean game, a slow-but-talking dyad, non-consecutive silences, a partner who speaks again, and a partner talking while I am silent must all never abort. Co-Authored-By: Claude Opus 5 --- reference-game-cwg.html | 132 +++++++++++++++++++++++++++++++++- tests/dropout.test.mjs | 155 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 tests/dropout.test.mjs diff --git a/reference-game-cwg.html b/reference-game-cwg.html index 7a54a72..816e0cb 100644 --- a/reference-game-cwg.html +++ b/reference-game-cwg.html @@ -103,6 +103,18 @@ // every participant who does not click the download button contributes nothing. DATAPIPE_EXPERIMENT_ID: "", + // --- Dropout detection (#5) --------------------------------------------------------------- + // Consecutive SILENT rounds before concluding the partner is gone. A round counts as silent + // only if it timed out AND the partner sent no chat message during it — see the detector for + // why both conditions are needed. Two is the suggested value: at a 180s round that is ~6 + // minutes of dead time before the survivor is released, against 15 minutes today, while still + // requiring corroboration rather than firing on a single slow round. + // + // The trade is asymmetric and worth stating. A false positive ends a live dyad early and pays + // both partners a partial code; a false negative leaves a survivor grinding. Requiring partner + // silence is what makes 2 safe — a slow dyad that is still talking is never aborted. + DROPOUT_SILENT_ROUNDS: 2, + // A4 — save per-round chunks rather than once at the end. A single end-of-run save loses // everything for any dyad that breaks, which is exactly the population #12 needs to // characterise. Assumes `limitSessions` is OFF (B4): DataPipe's counter increments per SAVE @@ -502,6 +514,51 @@ let myRole; let partnerId; + // =============================================================================================== + // DROPOUT DETECTION (#5) + // + // There is no presence API in any of the four bundles, so the partner's absence has to be + // inferred. The inference has to be good, because acting on it ends a paid session. + // + // `round_timeout` alone is NOT the signal. It is an unconditional wall-clock bound (see CONFIG), + // so it fires identically for a partner who left and a pair who are simply slow — and trial 1 of + // a 12-figure board is the longest in the study, so a genuine timeout there is entirely normal. + // Counting timeouts alone would abort real dyads at exactly the point the effect is largest. + // + // So a round is "silent" only if it timed out AND the partner sent nothing during it. The plugin + // records `message_count` (every message that round) and `messages_sent` (mine), and the chat + // channel is per-round, so the difference is the partner's activity in that round alone. A + // partner who is talking is present, however slow they are; a partner who is neither acting nor + // talking for two consecutive rounds has almost certainly gone. + // + // Upgrade path: when a presence API exists (#11), this becomes a fallback rather than the + // primary signal. It does not need to wait for one. + // =============================================================================================== + let consecutiveSilentRounds = 0; + let partnerDropped = false; + let dropoutRound = null; + + // Trials the dyad actually completed, as opposed to rounds the timeline advanced through. Only + // `ended_by: "submit"` counts — a timed-out round has a null assignment and is not a trial. + // This is what makes a partial dyad usable rather than merely present (#8). + const completedTrials = () => + jsPsych.data + .get() + .filter({ trial_type: "multiplayer-reference-game", ended_by: "submit" }) + .count(); + + // Session-level facts, stamped onto every row at the point the game ends by any route. Recorded + // here rather than left to analysis because "how did this session end" cannot be reconstructed + // reliably from partial data — which is precisely the data most likely to be all we have. + function recordOutcome(reason) { + jsPsych.data.addProperties({ + ended_reason: reason, + n_trials_completed: completedTrials(), + n_trials_scheduled: TRIALS, + dropout_detected_at_round: dropoutRound, + }); + } + const nameTrial = { type: jsPsychSurveyText, questions: [{ prompt: "Choose a display name:", required: true, name: "name" }], @@ -546,6 +603,7 @@ on_finish: () => { // Flush even here. A spectator has no game data, but their arrival and the fact they were // turned away is the raw material for the odd-arrival rate #10's waiting room must handle. + recordOutcome("spectator"); if (CONFIG.FLUSH_ON_ABORT) Pipeline.flush("spectator"); jsPsych.multiplayer.disconnect(); }, @@ -583,6 +641,23 @@ // Deliberately not awaited: the next round must not wait on OSF. on_finish: (data) => { if (CONFIG.SAVE_PER_ROUND) Pipeline.save(`round-${data.round ?? "?"}`); + + // Partner activity in THIS round. `message_count` is every message on the round's channel + // and `messages_sent` is mine, so the difference is theirs. A round that timed out while the + // partner was still talking is a slow round, not a dropout. + const partnerMessages = (data.message_count ?? 0) - (data.messages_sent ?? 0); + const silent = data.ended_by === "timeout" && partnerMessages <= 0; + consecutiveSilentRounds = silent ? consecutiveSilentRounds + 1 : 0; + + if (!partnerDropped && consecutiveSilentRounds >= CONFIG.DROPOUT_SILENT_ROUNDS) { + partnerDropped = true; + dropoutRound = data.round ?? null; + recordOutcome("partner_dropped"); + // `conditional_function` on gameLoop is evaluated ONCE, before the first round, so it + // cannot end the loop mid-game. Aborting the enclosing timeline from here is what actually + // stops the remaining rounds; execution resumes at the screens after gameLoop. + jsPsych.abortCurrentTimeline(); + } }, prompt: (role) => role === "director" @@ -596,6 +671,46 @@ conditional_function: () => myRole === "director" || myRole === "matcher", }; + // The survivor's exit (#5). Honest about what happened, explicit that they are still paid, and + // it never blames them. This screen is the entire difference between a survivor who submits and + // one who abandons and opens a support ticket. + const partnerDroppedScreen = { + timeline: [ + { + type: jsPsychHtmlKeyboardResponse, + stimulus: () => { + const played = completedTrials(); + return `

Your partner seems to have disconnected

+

We waited, but your partner stopped responding, so we have ended the study here. + This is not your fault, and your payment is not affected.

+

You completed ${played} of ${TRIALS} round${played === 1 ? "" : "s"} together, and + ${played > 0 ? "that work is saved and still useful to us" : "your time is still paid"}.

+

Saving your responses…

+ ${submissionBlockHTML("partner_dropped")}`; + }, + choices: "NO_KEYS", + on_load: () => { + // Wire the button FIRST. The participant must be able to submit immediately, whatever + // the save is doing — the whole point of this screen is that they get paid. + wireSubmissionButton("partner_dropped"); + jsPsych.multiplayer.disconnect(); + Pipeline.flush("abort-partner-dropped").then((r) => { + const el = document.getElementById("save-status"); + if (!el) return; + el.textContent = r.skipped + ? "" + : r.ok + ? "Your responses have been saved." + : r.timedOut + ? "Still uploading — you can submit now, this will finish in the background." + : "Your responses could not be uploaded. Please submit anyway and message the researcher."; + }); + }, + }, + ], + conditional_function: () => partnerDropped, + }; + const doneTrial = { type: jsPsychHtmlKeyboardResponse, stimulus: () => { @@ -618,6 +733,7 @@ choices: "NO_KEYS", on_load: () => { wireSubmissionButton("complete"); + recordOutcome("complete"); // Final flush: the last round's rows, plus anything an earlier chunk failed to send. Fired // on_load rather than on_finish because this trial takes no keys and never finishes — the @@ -650,6 +766,17 @@ }, }; + // `doneTrial` used to run unconditionally, which meant two people who never finished the game + // still reached "Game complete!" and were handed the `complete` code: the survivor of a dropout, + // and — already, before this change — any spectator, who saw a 4-second "game is full" screen and + // then a completion screen reporting 0 / 0. Both are paid under the wrong code, which defeats the + // point of having four, and the spectator case also contradicts what the screen before it said. + const completeScreen = { + timeline: [doneTrial], + conditional_function: () => + !partnerDropped && (myRole === "director" || myRole === "matcher"), + }; + jsPsych.multiplayer.connect(localAdapter).then(() => { // Preload FIRST, before pairing: a participant who is still fetching images while their // partner waits in the lobby wastes the partner's time. @@ -660,7 +787,10 @@ roleTrial, spectatorScreen, gameLoop, - doneTrial, + // Exactly one of these two runs, and each carries its own completion code. gameLoop's own + // conditional cannot end it mid-game, so the abort comes from inside the round (#5). + partnerDroppedScreen, + completeScreen, ]); }); diff --git a/tests/dropout.test.mjs b/tests/dropout.test.mjs new file mode 100644 index 0000000..6d8a205 --- /dev/null +++ b/tests/dropout.test.mjs @@ -0,0 +1,155 @@ +// Tests for dropout detection (#5) in reference-game-cwg.html. +// +// node tests/dropout.test.mjs +// +// Extracts the real `on_finish` body from the gameRound definition and drives it with synthetic +// round data, so the detector is tested as written rather than as re-implemented here. +// +// The thing under test is a judgement call with an asymmetric cost: a false positive ends a live +// dyad and pays two partial codes, a false negative leaves a survivor grinding. The cases below are +// mostly about the false-positive side, because that is the one that silently destroys good data. + +import fs from "fs"; + +const html = fs.readFileSync(new URL("../reference-game-cwg.html", import.meta.url), "utf8"); +const scriptMatch = html.match(/