NH: capture version links for bills carried over from prior session year - #5769
Merged
Merged
Conversation
Bills carried over from a prior session (LSR keeps that year's prefix, e.g. "25-0059" while being scraped as part of session "2026") never appear in NH's LsrsOnly.txt bulk-data dump for the current session, so versions_by_lsr had no id to build a billinfo.aspx page from. The scraper fell into a warn-only branch, leaving these bills with zero version/document links even after passing (became_law). scrape_from_web already had a working fallback for this exact case: query the legacy docket page directly by LSR + session year, no numeric id required. Extracted that into scrape_versions_from_docket and wired it into the main bulk-data path's missing-id branch. Verified against NH's live site for the reported became_law examples (HB113, HB565, HB661, SB49, SB189, HB348 all had LSRs absent from LsrsOnly.txt). A full 2026-session scrape before/after the fix drops bills with zero versions and zero documents from 232/1387 to 1/1387 (the remainder, HR48, has no text filed on NH's site at all).
alexkost819
force-pushed
the
fix/nh-bill-text
branch
from
August 24, 2026 21:36
6440e23 to
aa07ab5
Compare
jessemortenson
approved these changes
Aug 26, 2026
jessemortenson
left a comment
Member
There was a problem hiding this comment.
Looks good to me, merging
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
25-0059while being scraped as part of session2026) never appear in NH'sLsrsOnly.txtbulk-data dump for the current session, soversions_by_lsrhad no id to build abillinfo.aspxpage from — the scraper fell into a warn-only branch, leaving these bills with zero version/document links, including bills that became law (HB113, HB565, SB564, HB661, SB522, SB49, SB189, HB348 were the reported examples).scrape_from_webalready had a working fallback for exactly this case: query the legacy docket page directly by LSR + session year (no numeric id required). Extracted that logic intoscrape_versions_from_docketand wired it into the main bulk-data path's missing-id branch, instead of just logging a warning.Verification
Reproduced against NH's live site: all 8 reported became_law examples had LSRs entirely absent from
LsrsOnly.txt(or, for SB564/SB522, already had a version id but that's a separate/already-working path). Confirmedbill_status/legacy/bs2016/bill_docket.aspx?lsr=<lsr>&sy=<session>returns real version links for these bills directly.Ran a full 2026-session scrape before and after the fix (same code otherwise):
All 8 reported example bills have version links after the fix.
Example: HB113
Before —
versions: [], only the generic search-page source:{ "identifier": "HB113", "versions": [], "documents": [], "sources": [ { "url": "https://gc.nh.gov/bill_Status/advanced.aspx", "note": "" } ] }After — introduced, amendment, and engrossed versions all captured, each with HTML/PDF links:
{ "identifier": "HB113", "versions": [ { "note": "Introduced", "links": [ { "url": "https://gc.nh.gov/bill_status/legacy/bs2016/billText.aspx?sy=2026&v=HI&id=65&txtFormat=html", "media_type": "text/html" }, { "url": "https://gc.nh.gov/bill_status/legacy/bs2016/billText.aspx?sy=2026&v=HI&id=65&txtFormat=pdf", "media_type": "application/pdf" } ] }, { "note": "Amendment: #2026-0541h", "links": [ { "url": "https://gc.nh.gov/bill_status/legacy/bs2016/billtext.aspx?sy=2026&txtFormat=amend&id=2026-0541H", "media_type": "application/pdf" } ] }, { "note": "Ought to Pass", "links": [ { "url": "https://gc.nh.gov/bill_status/legacy/bs2016/billText.aspx?sy=2026&v=SP&id=65", "media_type": "text/html" }, { "url": "https://gc.nh.gov/bill_status/legacy/bs2016/billText.aspx?sy=2026&v=SP&id=65&txtFormat=pdf", "media_type": "application/pdf" } ] } ], "documents": [], "sources": [ { "url": "https://gc.nh.gov/bill_Status/advanced.aspx", "note": "" } ] }Before/after scrape output
Full NH 2026-session scrape output, before and after the fix:
nh_before.zipnh_after.zipTest plan
python3 -m py_compile scrapers/nh/bills.py