Skip to content

[PLDG Submission]: Data Source - @gitsofaryan #259

Description

@gitsofaryan

PLDG Proof-of-Work Submission

Contributor

  • GitHub Username: @gitsofaryan
  • Discord Username (if applicable): thecosmicnerd

Gist Link

Link to your notebook gist: https://gist.github.com/gitsofaryan/5cb61e41b8effb97aa0ef727134510fb

Data Source Audited

Pipeline audited:

  • Staging: oso.stg_occupational_tasks
  • Intermediate: oso.int_ai_exposure_scores
  • Mart: oso.timeseries_labor_metrics_v0

Key Findings

Provide a 2-3 sentence summary of your analysis:

  • Pipeline Integrity: Data flows correctly through all 3 pipeline stages with expected cardinality changes. Staging layer contains ~3,500 task records that aggregate to ~400 occupation-level records in intermediate, then expand to time-series metrics in mart. No unexpected data loss detected.

  • Date Coverage: Employment projections (2024-2034) and unemployment data (2016-present) provide complete temporal continuity, enabling robust before/after analysis around ChatGPT release (Nov 2022) and other economic events. All expected time periods are represented with no gaps.

  • Data Quality: Primary key uniqueness validation shows zero duplicate (occupation, metric, date) composite keys in the mart layer. Minimal null values in critical columns ensures data reliability for downstream analysis of AI labor market impacts.

Checklist

Before submitting, verify that your notebook:

  • Runs successfully with uv run marimo edit ai_labor_market_anthropic_[username].py
  • Passes structure validation with uv run marimo check ai_labor_market_anthropic_[username].py
  • Includes all 3 required checks (presence, date coverage, duplicates)
  • Audits all 3 pipeline stages (staging → intermediate → mart)
  • Contains visualizations using plotly
  • Has clear markdown documentation explaining analysis
  • Separates concerns (markdown, SQL, and Python in separate cells)
  • Follows naming convention: ai_labor_market_anthropic_{username}.py
  • Pipeline documentation section is complete

Additional Notes

Research Base: This submission validates the data pipeline for Anthropic's March 2026 research: "Labor market impacts of AI: A new measure and early evidence" (https://www.anthropic.com/research/labor-market-impacts)

Data Sources:

  • O*NET Occupational Database (task-level occupational data)
  • Anthropic Economic Index (real-world Claude usage patterns)
  • Eloundou et al. (2023) β scores (theoretical LLM capability assessments)
  • BLS Employment Projections (2024-2034 occupational growth forecasts)
  • Current Population Survey (employment and unemployment outcomes)

Key Methodology: The "observed exposure" metric combines theoretical LLM capability with real-world usage data, differentiating between automation (full weight) and augmentation (half weight) use cases. This goes beyond previous approaches that relied solely on theoretical capability.

Validation Results:

  • All 4 projects/occupations verified present in database ✅
  • Temporal gaps: None detected (2016-2034 full coverage) ✅
  • Duplicate keys: 0 found (primary key integrity solid) ✅
  • Data freshness: Current through March 2026 ✅

Next Steps: OSO maintainers will review your submission. If approved, you'll be asked to submit a PR with your notebook.

Metadata

Metadata

Assignees

Labels

c:communityDatasets and notebook templates

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions