Skip to content

test(obs_data): prove per-data_item operation_kwargs actually take effect#116

Merged
FinbarArgus merged 1 commit into
mainfrom
test/operation-kwargs-effect
Jul 24, 2026
Merged

test(obs_data): prove per-data_item operation_kwargs actually take effect#116
FinbarArgus merged 1 commit into
mainfrom
test/operation-kwargs-effect

Conversation

@FinbarArgus

Copy link
Copy Markdown
Contributor

Follow-up to #113. Its tests proved the kwarg value is carried (signature →
schema, UI renders/persists, JSON round-trip) but never that it has an effect
so the feature could break end-to-end with a fully green suite. This adds the three
missing links.

1. The value reaches CA's input file

test_obs_data_upload.pyoperation_kwargs survive the upload into the stored
obs_data.json that CA actually reads (asserted on both the response payload and
the file on disk). parse_obs_data currently passes data_items through verbatim;
a whitelist-style refactor would silently drop the field and every existing
kwargs test would still pass
.

2. The value reaches the operation call

test_local_sensitivity.py_evaluate_features splats operation_kwargs into
func(*operands, **kwargs). Parametrised so an ignored kwarg fails loudly
(factor 1/2/4 → 3/6/12; if kwargs were dropped all three collapse to 3). Also
covers the no-kwargs default and the documented substitution of a string kwarg
naming an earlier observable's feature — two call sites (lines 222, 355) that had
no coverage at all.

3. End-to-end: the value changes the result

test_user_funcs.py (@pytest.mark.integration) — on a CUFLynx-authored custom op
(#104), against the real CA:

  1. the op's kwarg is discoverable via operation_kwargs_schema (and the reserved
    series_output flag is not offered as a tunable),
  2. CA parses our obs_data.json and keeps operation_kwargs through
    parse_obs_data_jsonprocess_obs_info,
  3. invoking the registered op the way CA does gives a different value per kwarg
    (factor 2 → 6, factor 4 → 12, default → 3).

Skips cleanly when CA isn't importable or predates the external-path support (#303).

Tests

Backend 330 unit passed, 360 incl. integration. Tests-only — no production
code changed.

🤖 Generated with Claude Code

…fect

#113 landed the kwargs feature with tests that only prove the value is *carried*
(signature -> schema, UI renders/persists, JSON round-trip). Nothing asserted the
kwarg has an effect, so the feature could break end-to-end with a green suite.
Add the three missing links:

- test_obs_data_upload.py: operation_kwargs survive the upload into the stored
  obs_data.json that CA reads (response payload *and* file on disk). A
  whitelist-style refactor of the data_item parsing would silently drop them and
  every other kwargs test would still pass.
- test_local_sensitivity.py: _evaluate_features passes operation_kwargs into the
  operation call — parametrised so an ignored kwarg fails (factor 1/2/4 ->
  3/6/12), plus the no-kwargs default and the documented substitution of a string
  kwarg naming an earlier observable's feature.
- test_user_funcs.py (integration): end-to-end on a CUFLynx-authored custom op
  (#104) — its kwarg is discoverable via operation_kwargs_schema (and the
  reserved series_output flag is not offered), CA parses our obs_data.json and
  keeps operation_kwargs, and invoking the registered op the way CA does gives a
  different value per kwarg.

Backend 330 unit passed; 360 incl. integration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FinbarArgus
FinbarArgus merged commit 2670dbc into main Jul 24, 2026
3 checks passed
@FinbarArgus
FinbarArgus deleted the test/operation-kwargs-effect branch July 27, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant