Skip to content

Benchmark: add one viewer or rubric regression fixture #7

Description

@OthmaneBlial

Goal

Improve one published benchmark signal without widening the product scope. Choose exactly one:

  • add one synthetic viewer-compatibility fixture and record the versioned result; or
  • improve one deterministic rubric assertion with before/after raw evidence.

Allowed files

  • benchmarks/decision-v1/
  • storyboard_studio/benchmark.py
  • tests/test_benchmark.py
  • docs/BENCHMARK.md

Fixture and privacy boundary

Use synthetic or public data only, declare its license, and do not include private decks, customer content, API keys, or remote assets. Provider networking must remain opt-in.

Acceptance criteria

  • The change addresses one viewer or one rubric signal only.
  • A failing-then-passing regression test explains the behavior.
  • make benchmark-fixture-check, make lint, make format-check, and make test pass.
  • make benchmark-check reports no unrelated score regression.
  • Before/after score.json or a versioned viewer result is attached to the pull request.
  • The limitation section is updated if the claim boundary changes.

Maintainer availability

Ask contract questions here before implementing. A maintainer will confirm whether the proposed fixture stays inside the published rubric and privacy boundary.

Out of scope

Private data, new product templates, broad visual redesigns, mandatory cloud providers, and claims of factual or universal presentation quality.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkReproducible benchmark and rubric workhelp wantedStoryboard Studio contribution surface

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions