Skip to content

Survey-aware light-curve column auto-detection - #7

Merged
tjayasinghe merged 3 commits into
mainfrom
feat/survey-column-detection
Jun 30, 2026
Merged

Survey-aware light-curve column auto-detection#7
tjayasinghe merged 3 commits into
mainfrom
feat/survey-column-detection

Conversation

@tjayasinghe

Copy link
Copy Markdown
Owner

Summary

Makes column auto-detection plug-and-play with real light-curve downloads from the
major time-domain surveys — cup.periodogram(df, "GLS") works with no column hints
for ASAS-SN, ASAS-3, ATLAS, CRTS/CSS, ZTF, Pan-STARRS, LSST, TESS, Kepler, Gaia, and
MACHO. Headers were taken from each survey's authoritative format docs (IRSA, MAST,
ESA Gaia datamodel, LSST sdm_schemas, fallingstar-data, ASAS/OGLE READMEs).

What changed

  • Expanded detection lists (columns.py) with the real spellings each survey uses —
    e.g. magerr (ZTF/CRTS), flux_error (Gaia), uJy/duJy/dm (ATLAS),
    psfFlux/psfFluxErr/obsTime/filterID (Pan-STARRS), midpointMjdTai/band +
    DP0.2 midPointTai/psFlux/filterName (LSST), MAG_0/MER_0 (ASAS-3),
    filtercode/HMJD (ZTF), KSPSAP_FLUX (TESS QLP), bkjd (Kepler).
  • Domain- & measurement-aware error pairing. The error is derived from the resolved
    value column first (PDCSAP_FLUXPDCSAP_FLUX_ERR, uJyduJy), then the
    domain-appropriate list, then neutral err/sigma. Keeps a flux value with a flux
    error and a magnitude value with a magnitude error even when a table has both (ATLAS),
    and avoids the ATLAS F-filter vs flux-f collision.
  • Corrected/detrended fluxes preferred (PDCSAP_FLUX, KSPSAP_FLUX) over raw
    SAP_FLUX; infer_domain recognizes flux units (uJy/mJy/nmgy).

Docs & tests

  • A Supported surveys table in the light-curves guide, with a note on headerless
    products (OGLE .dat) and positional reading.
  • A parametrized test over 13 real survey header sets asserting correct
    time/value/error/band resolution (incl. the ATLAS and TESS pairing subtleties).

Verification

pytest 137 passed / 2 skipped · ruff + mypy clean · docs -W build clean.

Limits (documented)

  • ATLAS's single-char F filter, and OGLE/MACHO headerless or undocumented spellings,
    need an explicit ColumnMap.
  • Gaia's wide TAP table (g_transit_flux…) is out of scope; the standard flat epoch
    product is covered.

🤖 Generated with Claude Code

tjayasinghe and others added 3 commits June 29, 2026 21:54
Expand ColumnMap's detection lists to cover the standard light-curve products of the
major time-domain surveys, so a downloaded table needs no column hints: ASAS-SN, ASAS-3,
ATLAS, CRTS/CSS, ZTF, Pan-STARRS, LSST, TESS, Kepler, Gaia, MACHO.

Headers added from each survey's authoritative format docs: magerr (ZTF/CRTS),
flux_error (Gaia), dm/uJy/duJy (ATLAS), psfFlux/psfFluxErr/obsTime/filterID (Pan-STARRS),
midpointMjdTai/band + DP0.2 midPointTai/psFlux/filterName (LSST), MAG_0/MER_0 (ASAS-3),
filtercode/hmjd (ZTF), bkjd (Kepler), KSPSAP_FLUX (TESS QLP).

Error resolution is now domain- and measurement-aware: the error is derived from the
resolved value column first (PDCSAP_FLUX -> PDCSAP_FLUX_ERR, uJy -> duJy), then the
domain-appropriate list, then neutral spellings. This pairs a flux value with a flux
error and a magnitude value with a magnitude error even when a table carries both (ATLAS
m/dm + uJy/duJy), and avoids the ATLAS F-filter vs flux-f collision. Corrected/detrended
fluxes (PDCSAP, KSPSAP) are preferred over raw SAP; infer_domain recognizes flux-unit
names (uJy/mJy/nmgy).

Docs: a "Supported surveys" table in the light-curves guide, incl. headerless OGLE and
positional reading. Tests: a parametrized check over 13 real survey header sets.

Verified: pytest 137 passed / 2 skipped, ruff + mypy clean, docs -W build clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tjayasinghe
tjayasinghe merged commit 46bca89 into main Jun 30, 2026
6 checks passed
@tjayasinghe
tjayasinghe deleted the feat/survey-column-detection branch June 30, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant