Repository navigation
docs: adopt research-center direction (brief, audit, epics 12-19) #131
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| --- | ||
| topic: OpenDiscourse vision: free local political research center | ||
| updated: 2026-10-10T03:01 | ||
| --- | ||
|
|
||
| - (decision) Must-have core tracks: legislation+votes (1), bill/law text (2), outcomes by place (5), report cards/summary sheets (8) | ||
| - (decision) Very important, harder inference: donations/lobbying (3), federal awards/regulation beneficiaries (4), politician trades from STOCK Act disclosures (6) | ||
| - (decision) Track 7 statements vs votes is wanted and is the key differentiator; hard | ||
| - (decision) Outcomes scope: all of them (income, crime, standard of living, housing, immigration, taxes, health...) at district, county, state | ||
| - (decision) Audience: researchers plus ordinary readers; free and local; OpenStates/think-tank/NGO equivalent | ||
| - (assumption) Do not start over: identity, provenance, legislation, Epic 10 district slice are the foundation | ||
| - (assumption) Language rules: timelines and evidence not legal conclusions; consistency check not 'lie'; many explained metrics not one corruption score; misandry needs measurable definition | ||
| - (event) Operator picked option A: write vision brief before data audit; AGENTS.md rule changes only after operator sees brief | ||
| - (decision) First user is the operator as researcher; design so an organization (think tank) could run it; output is published findings (blog/Substack), not a voter-facing app | ||
| - (decision) Point-in-time report cards wanted: store every fact with an as-of (knowledge) date so a card can be regenerated for any past date; design now, build later | ||
| - (decision) Report card contents: party loyalty with dissent reasons where available, attendance, ideology position from voting history, donor breakdown and concentration, tenure and prior offices, district change since election, consistency over tenure (flip-flops), ethics actions, committee memberships, donor links to other candidates | ||
| - (decision) Statements sources: Congressional Record floor speeches first; GDELT news-quoted statements as lower-trust reported statements; social media later and only if legally and cheaply available | ||
| - (decision) Credibility rule: honest metrics; every metric labelled established/estimated/exploratory with method and inputs; deservedly right, not pretending | ||
| - (decision) Do not hand-label members as fascist/communist/etc.; use empirical measures (Voteview DW-NOMINATE already loaded) plus issue-level measures | ||
| - (direction) Operator stated strong prior views and wants report cards to 'prove' them; brief requires the tool to be able to contradict operator priors: hypotheses and metrics defined before results are viewed; null results published | ||
| - (direction) Operator limits: do not dox anyone, do not break laws | ||
| - (assumption) Donor 'country of origin' is mostly not in FEC data (foreign nationals barred); use employer, occupation, state, PAC type, concentration instead; foreign influence via FARA/LDA is a separate source | ||
| - (assumption) Illegal immigration per district is not directly measured; use CBP encounters, modelled unauthorized-population estimates (state level), foreign-born share (ACS), and sanctuary-policy lists with stated definitions | ||
| - (event) Operator approved brief; operator clarified they are nonpartisan, earlier rhetoric not a policy input; objectivity principle accepted; addendum with sources/metrics/methods written | ||
| - (decision) Add: LDA filings link bills to lobbyists; CES for district opinion; CAP topic codes for flip-flops; district change is context not credit; donor grouping is a separate labelled estimated layer | ||
| - (decision) Scope widened: any office-holder (senators, president, governors, state legislators via OpenStates #102), one person-office-term model; local officials need a source; courts out | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| --- | ||
| title: OpenDiscourse vision brief addendum (metric, source and method catalog) | ||
| created: 2026-10-10 | ||
| status: draft | ||
| --- | ||
|
|
||
| Reliability tiers: **E** established (published method, official data), **S** estimated | ||
| (modelled or probabilistic joins), **X** exploratory (needs validation before publishing). | ||
|
|
||
| ## 1. Additional sources (beyond what is loaded) | ||
|
|
||
| | Purpose | Source | Notes | | ||
| |---|---|---| | ||
| | Floor speeches, dissent explanations | Congressional Record (GovInfo CREC) | Official, structured by speaker; start here for track 7 | | ||
| | Hearings, witnesses, markups | GovInfo CHRG, committee sites | Who testified, which industries | | ||
| | Ethics actions | House Ethics Committee, Office of Congressional Conduct, Senate Ethics statements | Small, public, official | | ||
| | Trades and assets | House Clerk periodic transaction reports, Senate disclosures | Values are ranges, not exact | | ||
| | Lobbying | Lobbying Disclosure Act database | **Filings name specific bills**: direct link bill -> lobbyist -> client | | ||
| | Foreign agents | DOJ FARA eFile | The honest route to "foreign influence", instead of donor country | | ||
| | Federal money | USAspending, SAM.gov, grants.gov | Awards by place and recipient | | ||
| | Rules and beneficiaries | Federal Register, regulations.gov (comments) | Who commented, who gained | | ||
| | Waste and fraud | Payment-accuracy (improper payments) and Inspector General reports | The only official measures of waste | | ||
| | Constituent opinion by district | Cooperative Election Study (open, Harvard Dataverse) | Lets us compare votes with what the district says it wants | | ||
| | Elections | MIT Election Data + Science Lab results | Needed for close-election comparisons and tenure | | ||
| | Alternative ideology | Bonica DIME / CFscores (donor-based), Volden-Wiseman legislative effectiveness | Cross-check on DW-NOMINATE | | ||
| | Topic taxonomy | Comparative Agendas Project codes | Standard way to say which bills are the same issue (needed for flip-flops) | | ||
|
|
||
| Outcomes by place (track 5), each by county/district where published: | ||
| income and poverty (Census SAIPE, ACS), jobs and wages (BLS LAUS, QCEW; BEA regional), taxes (IRS SOI by | ||
| ZIP/county), housing cost (HUD fair-market rents, FHFA house price index, ACS rent burden), | ||
| homelessness (HUD count), crime (FBI NIBRS; coverage gaps must be flagged, not zero-filled), | ||
| health (CDC PLACES/WONDER, CMS), education (NCES), environment (EPA), mobility (Opportunity Insights), | ||
| immigration (CBP encounters, DHS yearbook, immigration court backlog, ACS foreign-born share), | ||
| disasters (FEMA). Approval ratings: pollster licences must be checked first. | ||
|
|
||
| ## 2. Additional metrics for the card | ||
| - Bipartisanship (share of cosponsors and votes across parties; published method). | ||
| - Legislative effectiveness (bills advanced per stage versus expected for seniority). | ||
| - Vote alignment with district opinion (CES) and with the president. | ||
| - Absences split by reason where recorded, and by whether they were decisive. | ||
| - Donor concentration: top-donor share, small-dollar share, in-state share, industry mix (E/S). | ||
| - Same donor, many candidates (S, because individual donors have no stable id). | ||
| - Trades timed against committee activity and votes (timeline, not verdict). | ||
| - Money received by the district from federal awards (E). | ||
| - Consistency over time at topic level (X until a human-checked sample passes). | ||
| - Statement-to-vote consistency (X; published only with a measured accuracy rate). | ||
|
|
||
| ## 3. Methods and cautions | ||
| 1. **District change is context, not credit.** A member's effect on local crime or income is small and | ||
| confounded. Compare the district to peer districts and the national trend, label it context, and | ||
| use difference-in-differences or close-election comparisons only for specific, stated questions. | ||
| 2. **Donor matching is probabilistic.** FEC individual donors have no stable id. The rule "never match by | ||
| name" protects politician joins; donor grouping must be a separate, labelled (S) layer with its | ||
| error rate measured. | ||
| 3. **Language models must be measured before they are trusted.** Hand-label a sample (around 200), | ||
| measure agreement and error, and publish the rate. Keep the extracted claim, its source passage and | ||
| the model/version. | ||
| 4. **Versioned methodology.** Every metric has a definition, version and changelog; a card records | ||
| which versions produced it. Hypotheses are committed before results are viewed. | ||
| 5. **Bitemporal facts.** Store when a fact was true and when we learned it, so cards can be rebuilt | ||
| for any date. Redistricting makes district-based tenure comparisons need the Epic 10 boundary years. | ||
| 6. **Corrections policy and legal review** before publishing named-person grades. | ||
| 7. **Scope: every office-holder, one model.** Cards are keyed by person, office and term, with the | ||
| outcome geography taken from the office (district, state or nation): U.S. representatives, senators, | ||
| president and vice president, governors and other statewide officers, and state legislators. Metrics | ||
| are defined per office type (legislators vote; governors sign, veto and appoint). The first card is a | ||
| U.S. House member because that data is ready; state legislators come with the approved OpenStates | ||
| promotion (issue #102), not after everything else. Local officials (mayors, councils) are not in the | ||
| OpenStates snapshot and need a separate source. Courts are out of scope for now. | ||
| Cross-office history (state legislator who became a member of Congress) joins only on reviewed | ||
| identifiers, never on name. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| --- | ||
| title: OpenDiscourse vision brief | ||
| status: approved-by-operator | ||
| created: 2026-10-10 | ||
| updated: 2026-10-10 | ||
| supersedes_in_part: ../brief-opendiscourse-2026-09-14/brief.md | ||
| --- | ||
|
|
||
| # OpenDiscourse: a free, local political research center | ||
|
|
||
| ## What it is | ||
| A research workbench that does for one researcher, or a small organization, what | ||
| OpenStates and think tanks do: join official records on **people, votes, bill text, | ||
| money, places and time**, keep the evidence behind every number, and produce | ||
| **findings and politician report cards that can be published** (blog, Substack). | ||
| The reader is the researcher. The audience is whoever the researcher publishes to. | ||
| This replaces the September line "not a consumer app"; it is still not a voter-facing app. | ||
|
|
||
| ## Why | ||
| Think-tank-grade analysis is paywalled or one-sided. The pieces (votes, text, donors, | ||
| district statistics) are public, but they sit in separate systems with different | ||
| identifiers and dates. Linking them correctly is the product. | ||
|
|
||
| ## Tracks and priority | ||
| - **Core (must):** 1 legislation and votes; 2 bill and law text; 5 outcomes by place | ||
| (income, housing, jobs, crime, taxes, health, population, immigration-related measures); | ||
| 8 report cards and summary sheets. | ||
| - **Important, weaker inference:** 3 donations and lobbying; 4 federal awards and | ||
| regulation beneficiaries; 6 members' trades from required disclosures. | ||
| - **Hard, wanted:** 7 what members say versus how they vote. Sources, in order: floor | ||
| speeches in the Congressional Record; news-reported quotes via GDELT (marked as | ||
| reported, lower trust); social media only if legally and cheaply obtainable. | ||
|
|
||
| ## Who gets a card | ||
| Any office-holder in the data: U.S. representatives and senators, president and vice president, | ||
| governors and other statewide officers, and state legislators (via OpenStates). One card model, with | ||
| metrics chosen by office type. The first published card is a U.S. House member. | ||
|
|
||
| ## Report card (target content) | ||
| Party loyalty (with the member's stated reason for dissent where one exists); attendance; | ||
| position on a voting-derived left-right scale and on issue-specific scales; donor | ||
| breakdown (size mix, top-donor concentration, industry, state, other candidates the same | ||
| donors back); tenure and prior offices; committees; ethics actions; district change since | ||
| first elected (income, jobs, crime, housing, population, spending received); consistency of | ||
| positions over time with flagged reversals; each with its inputs one click away. | ||
|
|
||
| ## Principles | ||
| 1. **Evidence first.** Every number traces to an official file, checksum and run. | ||
| 2. **Honest grading.** Each metric is labelled *established*, *estimated* or *exploratory*, | ||
| with method and known limits shown. Uncertain metrics may be published only as labelled. | ||
| 3. **Findings can go against the researcher.** Hypotheses and metric definitions are written | ||
| before results are viewed; null and contrary results are published too. This is what makes | ||
| a "deservedly right" claim credible. | ||
| 4. **Timelines, not verdicts.** The tool shows "trade, then committee vote" or "donation, | ||
| then bill"; it does not label "bribed", "insider trading" or "lie". It labels | ||
| "statement inconsistent with recorded vote" with both items shown. | ||
| 5. **Many explained metrics, never one opaque corruption score.** | ||
| 6. **Time travel.** Facts carry an as-of date so a card can be regenerated for any past date. | ||
| 7. **People joins by identifier only** (BioGuide), never by name. No doxxing: no home | ||
| addresses, family finances or non-public data. Follow the law and each source's terms. | ||
|
Comment on lines
+59
to
+60
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: set -o pipefail
printf '%s\n' '--- brief identity/scope/source references ---'
rg -n -F -- 'BioGuide' _bmad-output/planning-artifacts/briefs/brief-OpenDiscourse-2026-10-10/brief.md || true
rg -n -i -- 'OpenStates|state legisl|crosswalk|join|#102|promotion' _bmad-output/planning-artifacts/briefs/brief-OpenDiscourse-2026-10-10/brief.md || true
printf '%s\n' '--- brief surrounding source ---'
sed -n '1,180p' _bmad-output/planning-artifacts/briefs/brief-OpenDiscourse-2026-10-10/brief.md
printf '%s\n' '--- repository references to the contract and identifiers ---'
rg -n -i --glob '!node_modules' --glob '!dist' --glob '!build' -- '(^|[^0-9])#?102([^0-9]|$)|OpenStates|PersonNode|BioGuide|crosswalk|promotion contract' . || trueRepository: cbwinslow/opendiscourse Length of output: 45677 🏁 Script executed: printf '%s\n' '--- brief ---'
rg -n -i -C 4 -- 'BioGuide|OpenStates|state legisl|crosswalk|join|#102|promotion' _bmad-output/planning-artifacts/briefs/brief-OpenDiscourse-2026-10-10/brief.md || true
printf '%s\n' '--- repository contract references ---'
rg -n -i -C 3 --glob '!node_modules' --glob '!dist' --glob '!build' -- 'OpenStates|PersonNode|BioGuide|crosswalk|promotion contract|#102' . || trueRepository: cbwinslow/opendiscourse Length of output: 45678 🏁 Script executed: set -o pipefail
printf '%s\n' '--- #102 promotion reference ---'
nl -ba _bmad-output/planning-artifacts/sprint-change-proposal-2026-10-10.md | sed -n '45,80p'
printf '%s\n' '--- Epic 13 and identity contract references ---'
rg -n -i -C 6 -- 'Epic 13|#102|state legisl|person.*(join|identity)|OpenStates.*(person|identifier)|crosswalk|promotion' _bmad-output/planning-artifacts/epics.md _bmad-output/planning-artifacts/briefs _bmad-output/planning-artifacts/specs plans inventory docs 2>/dev/null | sed -n '1,260p'Repository: cbwinslow/opendiscourse Length of output: 29238 Define the OpenStates identity rule in the promotion contract. The brief explicitly includes state legislators through OpenStates, but its only people-join rule names BioGuide. The State that BioGuide applies to federal joins. Define the reviewed mapping from stable OpenStates person identifiers to canonical people. Keep unresolved identifiers as exceptions, and never fall back to name matching. 🤖 Prompt for AI Agents |
||
| 8. **Build the foundations now, decide the analysis later.** Linkages and workflows first; | ||
| language-model extraction is added once its accuracy has been measured. | ||
|
|
||
| ## Non-goals (for now) | ||
| Market-price data; news as an editorial product; automatic "truth" verdicts; voter-facing app; | ||
| claims of intent about a person; local-government officials and courts (until a source is chosen); any ingest of data that cannot be tied to a source. | ||
|
|
||
| ## Success signals | ||
| - A researcher can regenerate any member's card as of any past date, and each figure drills to its sources. | ||
| - For the 119th House: cards exist with at least attendance, party loyalty, vote-based ideology, | ||
| donor concentration and district change, each labelled by reliability. | ||
| - A published finding can be re-derived by another person from the repository alone. | ||
|
|
||
| ## Open | ||
| How to measure dissent reasons and topic-level flip-flops reliably; which approval-rating and | ||
| crime sources are usable and licensed; a measurable definition for any "bias" research; | ||
| legal review before publishing named-person grades. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: cbwinslow/opendiscourse
Length of output: 41878
🏁 Script executed:
Repository: cbwinslow/opendiscourse
Length of output: 16251
Reconcile the Voteview readiness statements.
The repository records Voteview as loaded and relinked on 2026-09-26, with named unmatched and conflicting identifiers. The scope statement that Voteview is not loaded live is stale. Update the dated statements to describe the partial links and avoid implying that Voteview is either absent or fully complete.
Suggested documentation fix
🤖 Prompt for AI Agents