Data: no save target exists — wire up DataPipe with incremental saves

[htsukamoto5]

Phase 1 — unblocked. Highest value per line of code in the audit; without it every other fix produces nothing analyzable.

initJsPsych() is currently called with no options at all — no on_finish, no on_data_update. The only data egress in either experiment is a localSave button on the final screen that the participant must voluntarily click. There is no DataPipe, no OSF, no endpoint, no fetch anywhere in the repo.

Consequences:

• Every participant who does not reach the final screen contributes zero data — all dropouts, all stranded survivors, all lobby timeouts, all spectators. Those are precisely the sessions needed to characterize attrition.
• Even completers only contribute if they click a button labelled "Download data (JSON)" on a screen that already says "You may close this tab."
• We would be paying for data we do not receive.

Work

• Wire up jsPsych DataPipe (pipe.jspsych.org to OSF), roughly five lines.
• Save incrementally, not only on on_finish. With pair-based attrition, on_finish-only saving loses exactly the sessions that matter. Save per-round, or at minimum on the abort path.
• Confirm a partially-saved session is still parseable on the analysis side.

See READINESS.md section 3.

[htsukamoto5]

Incremental saving is viable, but not the way the docs imply — read from DataPipe source

The public docs only describe the jsPsychPipe plugin's save action, a single trial at the end of the experiment, which would defeat the whole point here. Checked against the DataPipe source instead (jspsych/datapipe, origin/main, functions/src/api-data.ts, read 2026-07-30). Incremental saving works, but three properties of the endpoint change the design.

1. sessions increments on every save call, not per participant

await exp_doc_ref.set({ sessions: FieldValue.increment(1) }, { merge: true });

This fires on every successful saveData call. And the limit is checked up front:

if (exp_data.limitSessions) {
  if (exp_data.sessions >= exp_data.maxSessions) → 400 SESSION_LIMIT_REACHED
}

So with per-round saving, one C&WG participant consumes 7 sessions (6 rounds + a final flush), not 1. If limitSessions is enabled with maxSessions set from an expected participant count, we would hit the cap after roughly a seventh of the sample — and every save after that fails with a 400 and the data is lost, silently from the participant's point of view.

• Either leave limitSessions off and control recruitment through Prolific's places, or set maxSessions from the expected save count (participants × (rounds + 2)) with generous headroom.
• Add this to the pre-launch checklist. It is the kind of setting that looks correct and fails a third of the way through a run.

2. Filenames must be globally unique — a collision is unrecoverable

OSF returns 409 on an existing filename, and that path is handled specially:

if (result.errorCode === 409 && result.errorText === "Conflict") → 400 OSF_FILE_EXISTS

Note it returns without calling queueUpload. Every other failure mode gets queued for retry; a filename collision does not. So repeated whole-dataset uploads under a stable filename are not an option, and per-round chunks are the right shape.

• Use per-round chunk files, e.g. {PROLIFIC_PID}_{dyad_id}_r{n}_{nonce}.json.
• Include a nonce or timestamp, not just the round number. If a participant refreshes and replays a round, a bare r{n} filename collides and that round is lost with no retry.
• Analysis side has to reassemble many small files per participant. This is the real cost of incremental saving and it should be confirmed workable before we commit — it makes "is a partially-saved session parseable" a design question rather than a checkbox.

3. The failure story is better than assumed

Genuinely reassuring for the abort path in #5:

• persistPending() writes to Cloud Storage immediately after validation, before the OSF upload, explicitly so data survives a function crash during later processing.
• Network errors, timeouts, and non-409 OSF errors are queued for retry and return 202, not a failure.

So a transient failure at abort time does not lose the round. The only unrecoverable case is a filename collision, which is entirely preventable by (2). That strengthens the argument in #5 that the redirect should never block on the save.

4. Validation interacts with #4

If useValidation is on, each chunk is validated against requiredFields independently. Per-round chunks therefore each need to carry those fields — which works naturally if #4 attaches PROLIFIC_PID / dyad_id via addProperties, since that puts them on every row. Worth confirming the two are configured consistently rather than discovering it mid-run.

Still needs an account to settle

None of the above removes the need for a real DataPipe experiment ID to test against — in particular the round-trip of a per-round chunk and its reassembly. That signup is the thing gating this issue.