docs: document the score scales (FineScore, Rating, Rating10, Rating20) - #99
Merged
Conversation
Add a "Score Data Models" API page rendering `backend.pydantic.metrics` (the `Score`/`FineScore`/`Rating*` scales plus `get_score_type` and `normalize_score`), list it in the Data Models API index and nav, and mention the new scales and the `score_type` argument where users meet them: the Data Models guide (Score section, types table, summary), the Modules guide (SelfCritique) and the Rewards guide (LMAsJudge). Recompute the source-snippet line anchors of the three edited guides and regenerate their Colab notebooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The "Source" block of 57 pages used `--8 < --"path:N"`, which pymdownx snippets does not recognize, so the literal marker was rendered instead of the code. Restore `--8<-- "path:N"` (and the `:source` section form), and recompute the three line anchors that had gone stale (RLM agent, Deep Agent, Python Synthesis guides) now that the snippets are live again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #98: user-facing docs for the new score scales and the
score_typeargument, plus a fix for the broken source-snippet syntax across the docs.synalinks.src.backend.pydantic.metrics:Score,FineScore,Rating,Rating10,Rating20, plusget_score_type/normalize_score. Listed in the Data Models API index and the mkdocs nav.Ratingrow in the types table; a paragraph after theScoresection introducing the sibling scales andscore_type; summary bullet updated.SelfCritiquesection explains the defaultScorereward and howscore_typechanges the scale (instructions + schema follow, value normalized to 0..1).LMAsJudgesection gets ascore_type=synalinks.Ratingexample and the normalization note.data_models,modules,rewards). Other notebooks were left untouched even though the generator would reformat them (cell ids / unicode escaping drift vs. what's committed), to keep this PR to the docs change.--8 < --"path:N", which pymdownx snippets does not recognize, so the literal marker rendered instead of the code. Restored--8<-- "path:N"(and the:sourcesection form) everywhere, and recomputed the three anchors that had gone stale (RLM agent, Deep Agent, Python Synthesis guides).Test plan
uv run zensical build --cleansucceeds (withcheck_paths: true, so every snippet path resolves); the new page renders all scales/helpers and the three guide pages show the new text--8<--markersruff🤖 Generated with Claude Code