Skip to content

add HEALPix interchange guide (issue #63)#128

Merged
espg merged 5 commits into
mainfrom
claude/63-healpix-interchange-guide
Jul 22, 2026
Merged

add HEALPix interchange guide (issue #63)#128
espg merged 5 commits into
mainfrom
claude/63-healpix-interchange-guide

Conversation

@espg

@espg espg commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Closes #63. Refs #48.

What

Adds docs/healpix_interchange.md, a Tier-2 v1.0 guide for moving a packed
morton_index word to and from the wider HEALPix ecosystem, and links it from
the README next to the specification reference.

The guide covers:

  • The conversion API — mort2healpix (word → (nested_ipix, order)),
    mort2norm / norm2mort (split/join around base_cell * nside**2 + normed),
    and infer_order_from_morton — grounded in the real signatures in
    mortie/tools.py.
  • Round-trip against cdshealpix (lonlat_to_healpix / healpix_to_lonlat),
    the same oracle the suite uses in test_morton_index.py /
    test_coverage_hemisphere.py, plus importing a foreign NESTED id back into a
    word.
  • Round-trip against healpy (ang2pix / pix2ang, nside = 2**order,
    nest=True, lonlat=True).
  • The three boundary gotchas: (1) the 4-bit prefix is base_cell + 1 and base
    cells 7–11 set bit 63, so words must stay uint64 (reading int64 corrupts
    southern cells); (2) order is self-encoded — never carry it out of band, but
    every ecosystem call needs it explicitly; (3) a bare geo2mort yields an
    order-29 point, and the null sentinel is the all-zero word.

Approach

Every code block was executed against the built extension and the installed
ecosystem libs, and the pasted outputs are the real values. Snippet argument
order is called out where it differs (geo2mort is (lat, lon); cdshealpix and
healpy are (lon, lat)). numpy stays the only runtime dep — cdshealpix is in
the test extra (the suite's oracle); healpy is not in any mortie extra, and
the guide says so and shows pip install healpy.

Phases

  • Phase 1: write the guide, verify every snippet, wire into README.

How tested

  • maturin develop --release (rebuilt the Rust ext), then installed the test
    extra (cdshealpix==0.8.1, astropy==8.0.1) and healpy==1.19.0 separately.
  • Ran every doc code block verbatim as assertions — all pass (cell ids match
    cdshealpix and healpy exactly; centers agree to ~2e-9 deg).
  • flake8 mortie --select=E9,F63,F7,F82 — clean.
  • pytest -q — 683 passed, 11 skipped, 1 failed. The single failure is the
    pre-existing test_spec_page.py::TestDecimalParseTieBreak::test_order29_string_parses_to_area_word
    (issue main is red: test_spec_page tie-break test predates #120's p-suffix rendering #123), unrelated to this docs-only change.

Questions for review

  • README placement: I linked the guide from the specification paragraph;
    docs/arrow_interchange.md currently isn't linked from the README at all, so
    there's no single docs index to slot into — happy to move the link or add a
    proper docs index if preferred.
  • healpy is shown but is not in any mortie extra (only cdshealpix is, via
    test). Kept it as an optional pip install healpy block. Fine as-is, or
    should the healpy snippet be dropped to keep every snippet runnable purely
    from the declared extras?

🤖 Generated with Claude Code


Generated by Claude Code

@espg espg added the implement label Jul 21, 2026

@espg espg left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

Fresh-context adversarial pass on the HEALPix interchange guide. Overall this is accurate and well-scoped — I verified every claimed signature and worked example against mortie/tools.py and the section refs against docs/specification.md, and they hold up:

  • API table is correct: mort2healpix (scalar → (int, int), array → (ndarray, order)), mort2norm(normed, base_cell, order), norm2mort(normed, parent, order) → scalar uint64, geo2mort(lat, lon) → length-1 array for scalar input. The [0] indexing convention is right.
  • The "raises on mixed orders" claim for mort2healpix is correct — mort2norm raises first (the array-path order branch in mort2healpix is effectively unreachable for mixed orders), so the guide describes real behavior.
  • Arithmetic checks out: 37010 == 9*4096 + 146 (order 6), the int64 wrap -6881500230622117885, prefix nibble base+1, and bit-63 set for base cells 7–11. Operator precedence in int(w) >> 63 & 1 parses as intended.
  • Argument-order pitfall (geo2mort = (lat, lon) vs cds/healpy = (lon, lat)) is called out; healpy keying on nside and nest=True/lonlat=True is correct.
  • healpy being outside the extras is clearly disclosed; cdshealpix correctly identified as the test-extra oracle. Relative doc links and README link resolve correctly. Style/tables match arrow_interchange.md.

Only minor nits, none blocking (COMMENT, not request-changes) — three inline: (1) repr() expected-outputs are numpy-2-only; (2) confirm the pinned numeric outputs (esp. the lon # 0.0) were captured from a real run so they don't drift; (3) a precision nit on "southern base cells" vs the bit-63 set 7–11.


Generated by Claude Code

Comment thread docs/healpix_interchange.md Outdated
# geo2mort takes (lat, lon) and returns a length-1 array for scalar input,
# so index [0] for a single word.
m = mortie.geo2mort(-80.0, 120.0, order=6)[0]
print(repr(m)) # np.uint64(11570310392668225542)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

The # np.uint64(...) expected-output comments here (and at line 151, print(repr(w))) are numpy-2-only. On numpy 1.x repr(np.uint64(x)) prints the bare integer (11570310392668225542), so a reader on that stack sees a "mismatch" against the guide. numpy is the sole pinned runtime dep and no floor is stated in pyproject, so both are in scope. Cheap fixes: use print(int(m)) (stable across versions) or note the outputs assume numpy ≥ 2. The scalar mort2healpix/mort2norm prints below (# 37010 6, # 146 9 6) are already version-stable since those return Python ints — only the two repr lines need it.


Generated by Claude Code

clon, clat = healpix_to_lonlat(cell_ids, depth=order)
mlat, mlon = mortie.mort2geo(words)
print(float(np.max(np.abs(mlat - clat.to_value(u.deg))))) # ~1.98e-09
print(float(np.max(np.abs(mlon - clon.to_value(u.deg))))) # 0.0

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

Please confirm the pinned numeric outputs in this file were captured from an actual run, since they'll silently drift if not: the cell_ids == [...] list (line 84-85), the # ~1.98e-09 lat tolerances, and especially this # 0.0 for longitude. The lon-diff-of-exactly-zero relies on mort2geo returning longitude in [0, 360) (it does not wrap — only mort2bbox does), matching cdshealpix's [0, 360). If mort2geo ever returned [-180, 180), the eastern test points (lons 200/305/270.2) would diff by ~360 and this would not be 0.0. Worth a one-line note that these were generated from the current build (as the arrow guide implicitly does), so a future reviewer can regenerate rather than guess. Not blocking — the snippets run regardless of the comment values.


Generated by Claude Code

Comment thread docs/healpix_interchange.md Outdated
base cells occupy `1..=12`, and `0` is the empty/null sentinel). Base cells
**7–11 set bit 63**, making the word a large unsigned value. Store and exchange
the word as `uint64`; reinterpreting it as signed `int64` reads southern base
cells as negative and corrupts the id.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

Minor precision nit (optional): "reads southern base cells as negative" is slightly loose. The set that reads negative as int64 is exactly the bit-63-set cells 7–11 (stated correctly one sentence up), but base cell 7 is equatorial, not south-polar (HEALPix base cells: 0–3 north, 4–7 equatorial, 8–11 south). The int64-negative property is purely "prefix ≥ 8 ⇒ bit 63 set", independent of hemisphere — that's a different mapping from the decimal -1..-6 southern-sign in §2. Suggest "reads those base cells as negative" (or "base cells 7–11") to match the spec's own §1 wording ("base cells 7–11 set bit 63 and would read back negative"), which pointedly avoids the "southern" label here.


Generated by Claude Code

@codspeed-hq

codspeed-hq Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 1 skipped benchmark1


Comparing claude/63-healpix-interchange-guide (c4d598e) with main (f773873)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

@espg espg left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 from Claude (review)

Adversarial review of the HEALPix-interchange guide (issue #63). Verified every snippet against mortie/tools.py, mortie/__init__.py, pyproject.toml, and the test suite. No blocking defects — the API names, signatures, and gotchas are all correct. Notes below.

Verified correct

  • All referenced functions are real and exported: geo2mort, mort2healpix, mort2norm, norm2mort, infer_order_from_morton, mort2geo. Argument orders match: geo2mort(lat, lon, order=) is lat-first, mort2norm(normed, parent, order), norm2mort(normed, base_cell, order).
  • docs/healpix_interchange.md:89 / :132mlat, mlon = mortie.mort2geo(words) correctly handles mortie's unusual lat-first return order (mort2geo returns lat, lon). Easy to get wrong; got it right.
  • The scalar worked example (order 6 → cell 37010, word 11570310392668225542, base 9, normed 146) matches the mort2healpix docstring in tools.py and is internally consistent (9*4096+146 == 37010).
  • The cdshealpix block reuses the exact order-14 lat/lon arrays and lonlat_to_healpix(lons*u.deg, lats*u.deg, depth=order) oracle call from test_morton_index.py::test_latlon_matches_cdshealpix_nested — a faithful mirror of the test-suite oracle pattern.
  • Extras claims accurate: cdshealpix>=0.7 is in the test extra; healpy is in no extra (guide says install separately). numpy-only runtime respected (§7).
  • healpy calls correct: ang2pix(nside, lons, lats, nest=True, lonlat=True) and pix2ang(..., lonlat=True) both use the nside/lonlat convention properly.
  • Gotcha 1 arithmetic checks out: prefix nibble = base+1 (int(w)>>60 == 10 for base 9), bit 63 set, and np.int64(w) == -6881500230622117885 (= w - 2**64) is exactly the signed-misread value shown. Gotchas 2 (order self-encoding, mixed-order raise via mort2norm) and 3 (bare call → order-29 point) are accurate.

Minor (non-blocking)

  • docs/healpix_interchange.md:103 — the "Importing a foreign nested id" block uses oracle, order, and nside_sq from the previous code block, so it only runs as a continuation. A one-line "(continues the previous snippet)" note, or inlining order = 14, would keep it copy-run-safe.
  • Wiring: there is no docs index/toctree in the repo (docs/ is flat markdown linked ad hoc from README), so linking the guide from README is the right pattern. Note docs/arrow_interchange.md is not linked from README either — out of scope here, just flagging the inconsistency.
  • The exact array cell-ids and the ~1.98e-09 / 0.0 tolerance comments couldn't be re-run here (review-only, no build), but they mirror the test inputs and the shared cds-healpix core, so they're plausible; worth a final pytest/manual run before un-drafting.

Generated by Claude Code

@espg espg added the waiting label Jul 21, 2026
@espg espg mentioned this pull request Jul 21, 2026
@espg

espg commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

this doc mentions cdshealpix as verification, but doesn't mention or track https://github.com/matt-cornell/healpix-rs -- which is the actual crate we use in production for transforms to/from NESTED and other healpix structures and wrapped code functions. please update and add.

@espg espg removed the waiting label Jul 22, 2026
@espg

espg commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

🤖 from Claude

Folded the three non-blocking self-review nits in a35253c:

(1) The two print(repr(...)) demonstrations (the geo2mort(...)[0] word print in the conversion-API block and in gotcha 1) were numpy-2-only — repr(np.uint64(x)) prints np.uint64(x) on numpy 2.x but the bare integer on numpy 1.x. Switched both to print(int(...)) and updated the expected-output comments to the bare integer, so they're version-stable. The scalar mort2healpix / mort2norm prints were already stable and were left as-is.

(2) Added a one-line note after the cdshealpix round-trip block that the cell_ids list and the tolerance figures are captured from the current build and should be regenerated by rerunning the snippet rather than hand-edited. Numbers unchanged.

(3) Reworded the gotcha-1 sentence from "reinterpreting it as signed int64 reads southern base cells as negative" to "makes base cells 7–11 read back negative" — the int64-negative set is exactly base cells 7–11 (bit 63 set), and base cell 7 is equatorial, so the "southern" label there was imprecise (matches specification.md §1 wording).

Docs-only, no build needed. Leaving the review threads for the reviewer to resolve.


Generated by Claude Code

@espg espg added the waiting label Jul 22, 2026
@espg

espg commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

🤖 from Claude

Good catch — the earlier run only folded the self-review nits and missed this. Addressed in 0f7d602: the guide now names and tracks matt-cornell/healpix-rs (the healpix crate, pinned in Cargo.toml) as the crate mortie actually wraps for the NESTED transforms in production, and clarifies that cdshealpix / healpy appear here only as independent external oracles for cross-checking exactness, not as the implementation. Docs-only, no build needed.


Generated by Claude Code

@espg
espg marked this pull request as ready for review July 22, 2026 19:22
@espg

espg commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

🤖 from Claude

Tacked the main-greening fix into this PR as you asked. The red on main (run 29847408029) was test_mixed_order.py::TestResolutionCrossCheck::test_orders_of_indexes_order2res — a pre-existing bug introduced with the order2res rewrite in #119, not by anything in this PR (it was surfaced by the #65 run). This branch was 9 commits behind main and still carried the old order2res, so the fix had to land against current main.

Two commits added:

  1. eeebd7b merge main — brought this branch up to current main (no conflicts; the docs file here is new, main didn't touch it). No rebase/force-push.
  2. c4d598eorder2res now accepts array input and exponentiates in float:
order = np.asarray(order, dtype=np.float64)
area = 4 * np.pi * EARTH_RADIUS_KM**2 / (12 * 4.0**order)  # km2
res = np.sqrt(area)
return float(res) if res.ndim == 0 else res

Two bugs, one fix: (a) float(np.sqrt(area)) threw TypeError: only 0-dimensional arrays can be converted to Python scalars on the array orders_of(words) passes; (b) 4**order on a uint8 order array wrapped 4**29 to 0 (the divide by zero encountered in divide RuntimeWarning in the same run). Float exponentiation of integer orders ≤29 is exact (powers of two), so scalar results are byte-identical — res2display and the existing test_tools order2res assertions are unaffected. The failing test is itself the coverage for the array path.

Green locally (numpy 2.4.6): pytest -q709 passed, 11 skipped, 0 failed (was 708 + 1); the targeted test passes under -W error::RuntimeWarning (no div-by-zero warning); flake8 mortie --select=E9,F63,F7,F82 clean. No Rust changed. CI should now go green on the synchronize.

Kept as draft — it's yours to mark ready and merge whenever CI confirms. Left waiting.


Generated by Claude Code

@espg
espg merged commit 18cdd4f into main Jul 22, 2026
18 checks passed
@espg
espg deleted the claude/63-healpix-interchange-guide branch July 22, 2026 19:33
@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.60%. Comparing base (8a67647) to head (c4d598e).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #128      +/-   ##
==========================================
+ Coverage   94.38%   94.60%   +0.21%     
==========================================
  Files           9        9              
  Lines        1318     1371      +53     
==========================================
+ Hits         1244     1297      +53     
  Misses         74       74              
Flag Coverage Δ
unittests 94.60% <100.00%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
mortie/tools.py 98.37% <100.00%> (+0.33%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f773873...c4d598e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v1.0 docs: HEALPix-ecosystem interchange guide

2 participants