Repository navigation
test: characterization suite + score-service parity report - #5
Open
beta-devin-ai-integration[bot] wants to merge 3 commits into
Open
beta-devin-ai-integration[bot] wants to merge 3 commits into
beta-devin-ai-integration[bot] wants to merge 3 commits into
Conversation
Co-Authored-By: Beta-Devin AI <248786709+beta-devin-ai-integration[bot]@users.noreply.github.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Co-Authored-By: Beta-Devin AI <248786709+beta-devin-ai-integration[bot]@users.noreply.github.com>
…up in workflow Co-Authored-By: Beta-Devin AI <248786709+beta-devin-ai-integration[bot]@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a black-box HTTP characterization suite (
characterization-tests, JUnit 4 / Java 8) that pins what the legacy estate does today, soscore-service-bootcan be checked for parity before cutover. The tests are skipped in the normal build. Run them with-Dcharacterization=trueagainst running services.-Dscore.urlswitches the score-service under test, and-Dgroups=com.ukcredit.characterization.ScoreApilimits the run to the checks that go through score-service.scripts/db-reset.sh(seed20241231),GET /score/{id}?asOf=2024-12-31for all 256 seeded consumers is recorded ingolden/score-uks3.json, andGET /creditfile/{id}for 100001–100016 ingolden/creditfile-edge-cases.json. Regenerate with-Dgolden.record=true. Each run writes field-level diffs totarget/characterization/<target>/*-golden-diff.json.StatusXCutoffTest,DisputeSlaTest,UtilisationRoundingTest,NameMatchingTest,AddressHistoryMergeTest, plusDisputeScoreFlowTest, which checks forDISPUTE_OPENandDISPUTE_CLOSErows inSCORE_HISTORYand restores its own state.NI) gets no bank holidays at all;.github/workflows/characterization.yml(plain GitHub Actions,pull_request→uk-credit-legacy): set up Java 8 → build → db-reset → run-all → suite. The matrix islegacy, plusbootwhenscore-service-boot/exists on the branch. The boot leg uses Java 21 and starts Boot on :8092 against the same Postgres (SPRING_PROFILES_ACTIVE=postgres,PORT,DB_*). It points dispute-service at Boot and runs the ScoreApi checks. Surefire reports and the parity report are uploaded as artifacts.characterization-tests/parity/parity_report.pybuildsdocs/cutover/score-service-parity-report.mdfrom two runs. Root causes, approved intentional changes and out-of-suite notes are recorded indocs/cutover/parity-findings.json.Parity summary: legacy
score-servicevsscore-service-bootCandidate: #7 (
devin/1792677011-score-service-boot3@8e8b546). It was checked out locally without committing to it and run on JDK 21 against the same PostgreSQL 15 database. Each implementation got a freshdb-resetfirst.Recommendation: GO, subject to sign-off from Engineering, QA, Risk/Model Governance and Platform. Changes that #7 declares intentional but this suite doesn't exercise are listed in the report for explicit acceptance: JSON error bodies instead of Jetty HTML, default port 9082, and the shared
SCR_HIST_SEQ.The Actions run on this PR covers only the
legacytarget, becausescore-service-boot/isn't on this branch. Thebootmatrix leg runs automatically on PRs where both are present, for example #7 once this lands.Link to Devin session: https://cognition.beta.devinenterprise.com/sessions/b69735ce58a04288bfdaa7bb3d35c858
Open in Devin Desktop: https://cognition.beta.devinenterprise.com/desktop/session/b69735ce58a04288bfdaa7bb3d35c858?variant=devin
Requested by: @sofiasepielli1
Devin Review