test(obs_data): prove per-data_item operation_kwargs actually take effect#116
Merged
Conversation
…fect #113 landed the kwargs feature with tests that only prove the value is *carried* (signature -> schema, UI renders/persists, JSON round-trip). Nothing asserted the kwarg has an effect, so the feature could break end-to-end with a green suite. Add the three missing links: - test_obs_data_upload.py: operation_kwargs survive the upload into the stored obs_data.json that CA reads (response payload *and* file on disk). A whitelist-style refactor of the data_item parsing would silently drop them and every other kwargs test would still pass. - test_local_sensitivity.py: _evaluate_features passes operation_kwargs into the operation call — parametrised so an ignored kwarg fails (factor 1/2/4 -> 3/6/12), plus the no-kwargs default and the documented substitution of a string kwarg naming an earlier observable's feature. - test_user_funcs.py (integration): end-to-end on a CUFLynx-authored custom op (#104) — its kwarg is discoverable via operation_kwargs_schema (and the reserved series_output flag is not offered), CA parses our obs_data.json and keeps operation_kwargs, and invoking the registered op the way CA does gives a different value per kwarg. Backend 330 unit passed; 360 incl. integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #113. Its tests proved the kwarg value is carried (signature →
schema, UI renders/persists, JSON round-trip) but never that it has an effect —
so the feature could break end-to-end with a fully green suite. This adds the three
missing links.
1. The value reaches CA's input file
test_obs_data_upload.py—operation_kwargssurvive the upload into the storedobs_data.jsonthat CA actually reads (asserted on both the response payload andthe file on disk).
parse_obs_datacurrently passesdata_itemsthrough verbatim;a whitelist-style refactor would silently drop the field and every existing
kwargs test would still pass.
2. The value reaches the operation call
test_local_sensitivity.py—_evaluate_featuressplatsoperation_kwargsintofunc(*operands, **kwargs). Parametrised so an ignored kwarg fails loudly(
factor1/2/4 → 3/6/12; if kwargs were dropped all three collapse to 3). Alsocovers the no-kwargs default and the documented substitution of a string kwarg
naming an earlier observable's feature — two call sites (lines 222, 355) that had
no coverage at all.
3. End-to-end: the value changes the result
test_user_funcs.py(@pytest.mark.integration) — on a CUFLynx-authored custom op(#104), against the real CA:
operation_kwargs_schema(and the reservedseries_outputflag is not offered as a tunable),obs_data.jsonand keepsoperation_kwargsthroughparse_obs_data_json→process_obs_info,(factor 2 → 6, factor 4 → 12, default → 3).
Skips cleanly when CA isn't importable or predates the external-path support (#303).
Tests
Backend 330 unit passed, 360 incl. integration. Tests-only — no production
code changed.
🤖 Generated with Claude Code