Skip to content

nvidia-market: compare institution names on normalized form in Sheet 4 check - #67

Open
bleonard wants to merge 1 commit into
hkust-nlp:mainfrom
induction-ai:fix/nvidia-market-name-normalization
Open

nvidia-market: compare institution names on normalized form in Sheet 4 check#67
bleonard wants to merge 1 commit into
hkust-nlp:mainfrom
induction-ai:fix/nvidia-market-name-normalization

Conversation

@bleonard

Copy link
Copy Markdown

Agents were scoring 0 on Sheet 4's institution checks while submitting the correct institutions. Groundtruth names are SEC-filing style (JPMORGAN CHASE & CO, BANK OF AMERICA CORP /DE/, GEODE CAPITAL MANAGEMENT, LLC), while agents write display-case names derived from the same data (JPMorgan Chase & Co., Bank of America Corp.). The check intersected raw strings, so a real failing run produced:

GT list: ['GEODE CAPITAL MANAGEMENT, LLC', 'JPMORGAN CHASE & CO']
ACT list: [..., 'Geode Capital Management, LLC', 'JPMorgan Chase & Co.', ...]
Intersection: [] (total 0)   <- required at least 1

The grader already normalizes shareholder names with normalize_str for Sheets 2–3; this PR applies the same normalization to the Sheet 4 intersection, additionally stripping SEC state-of-incorporation markers (/DE/) and a leading "The".

Replayed against the failing run's data: the "Continuous Increase" check goes 0 → 2 matches (now passes its ≥1 bar); "Top 5 Most Active" goes 1 → 2 (still under its ≥3 bar — those lists genuinely differ, so the check stays meaningful). Distinct institutions remain distinct under the normalization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant