Skip to content

DATA.md

Latest commit

 

History

History
77 lines (60 loc) · 3.48 KB

DATA.md

File metadata and controls

77 lines (60 loc) · 3.48 KB
Raw

Data

Compass ships with everything it needs. The web app and the test suite read two committed files and never touch the network:

file what it is
compass/data/questionnaire.json the 60 O*NET Interest Profiler items and the response scale
compass/data/major_profiles.json the 112 majors, each with a RIASEC profile, example careers, Job Zone, and top knowledge areas
compass/data/majors.yaml the hand-maintained major list: slug, name, category, blurb, CIP codes

major_profiles.json is derived and regenerated by compass/data/refresh.py.

Sources

source URL license used for
O*NET Database 31.0 (text) onetcenter.org/dl_files/database/db_31_0_text.zip CC BY 4.0 RIASEC ratings, occupation titles, Job Zones, knowledge, skills
O*NET Interest Profiler Short Form onetcenter.org/dl_files/IPSF_PP.pdf CC BY 4.0 the 60 questionnaire items
CIP 2020 to SOC 2018 Crosswalk nces.ed.gov/ipeds/cipcode/Files/CIP2020_SOC2018_Crosswalk.xlsx public domain (US gov) mapping majors to occupations

ONET is a registered trademark of the US Department of Labor, Employment and Training Administration. Compass is an independent project and is neither endorsed by nor affiliated with ONET, NCES, or the Department of Labor. See NOTICE.

Rebuilding the profiles

pip install -e ".[data]"          # adds httpx + openpyxl
python -m compass.data.refresh    # download, build, write major_profiles.json
python -m compass.data.refresh --check   # rebuild in memory and diff, do not write

Downloads (about 13 MB) are cached under ~/.cache/compass (override with COMPASS_DATA_CACHE). The build is deterministic: --check in CI catches a stale major_profiles.json.

The pipeline

majors.yaml ── CIP codes
     │
     ▼   CIP to SOC 2018 crosswalk (drop "All Other" catch-alls)
6-digit SOC codes
     │
     ▼   expand to O*NET detailed occupations, keep Job Zone >= 3
O*NET-SOC occupations
     │
     ▼   O*NET "Occupational Interests" scale (1 to 7), mean over occupations
major RIASEC profile  ──►  major_profiles.json

Why "All Other" occupations are dropped

The crosswalk attaches summary SOC codes such as 11-9199 "Managers, All Other" to almost every social-science and business major. O*NET has no interest profile for the summary code, so expanding it pulls in an unrelated grab-bag of detailed occupations (Wind Energy Operations Managers, Loss Prevention Managers, ...) that swamp the real signal. _JUNK_SOC in refresh.py lists the cross-field ones that are dropped. Field-specific "All Other" codes, like 19-1029 "Biological Scientists, All Other" whose details are Biologists and Geneticists, are kept.

Labor-market data

Compass does not include wage or job-growth numbers. BLS blocks automated downloads, and interest fit is a separate question from pay and hiring anyway. The major pages label the "where it leads" list as labor market context and nothing more. Adding wages from a reachable source is on the roadmap.

Test fixtures

There are none as files. tests/test_refresh.py writes a tiny synthetic O*NET slice and a small crosswalk into a temp directory and runs the real parsing and building code against it. That keeps the test suite offline and the repo free of large binary blobs.