From c5e729f50cfcfaadaa14001e064f7d72d2e829fc Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Mon, 6 Jul 2026 18:23:11 -0300 Subject: [PATCH 01/16] Adding a technical note on the design of the pipeline's crossmatching and deduplication stages. --- .../crossmatch-deduplication-design.md | 409 ++++++++++++++++++ 1 file changed, 409 insertions(+) create mode 100644 docs/architecture/crossmatch-deduplication-design.md diff --git a/docs/architecture/crossmatch-deduplication-design.md b/docs/architecture/crossmatch-deduplication-design.md new file mode 100644 index 0000000..df54eab --- /dev/null +++ b/docs/architecture/crossmatch-deduplication-design.md @@ -0,0 +1,409 @@ +# Crossmatching and Deduplication Architecture + +## Purpose + +This document describes and justifies the execution design used in the CRC +pipeline for crossmatching, graph construction, deduplication, and final catalog +generation. + +The central design choice is deliberate: + +- LSDB/HATS is used where spatial semantics are required; +- Dask DataFrame is used for distributed relational operations; +- Parquet is used as a distributed materialization boundary; +- pandas is used only inside bounded, per-partition computations or small-data + fast paths. + +This is not a replacement of HATS with a generic tabular pipeline. It is a +separation of responsibilities intended to preserve the HATS spatial model +while avoiding unnecessarily large Dask/NestedFrame graphs in operations that +do not require spatial semantics. + +## Executive summary + +The pipeline does not abandon LSDB or HATS. For this pipeline and workload +profile, it avoids using `LSDB Catalog.concat()` followed by +`write_catalog()` on large, accumulated distributed graphs. + +In our production-scale runs, that path kept the full LSDB/NestedFrame lineage +attached through crossmatching, neighbor aggregation, catalog updates, +concatenation, and writing. In this workload, that was associated with higher +scheduler and driver pressure and made serialization and worker recovery more +expensive. + +The large-catalog path therefore uses the following strategy: + +1. Open and spatially partition catalogs with LSDB/HATS. +2. Project each catalog to the columns required by the spatial crossmatch. +3. Execute the crossmatch with LSDB, preserving HATS pixels and margins. +4. Materialize only the resulting pair identifiers as narrow Parquet datasets. +5. Use Dask to aggregate neighbors and join them back to the complete rows. +6. Concatenate the complete updated rows with Dask. +7. Write a distributed Parquet checkpoint. +8. Rebuild the HATS catalog and margins with `hats-import`. +9. Use LSDB/HATS pixel alignment again for margin-aware deduplication. + +The scientific semantics are unchanged. The design only changes how intermediate +data is represented, scheduled, and materialized. + +## Responsibilities by technology + +| Technology | Responsibility | +| --- | --- | +| LSDB/HATS | Catalog opening, spatial partitioning, margins, crossmatching, and main/margin pixel alignment | +| Dask DataFrame | Distributed projection, filtering, grouping, joins by `CRD_ID`, tabular concatenation, and lazy output filtering | +| Parquet | Distributed checkpoints that cut prior task-graph lineage and provide a stable input to HATS import | +| `hats-import` | Reconstruction of large HATS collections, spatial metadata, pixel layout, and margins from staged rows | +| pandas | Local graph solving inside one aligned main+margin partition and explicitly bounded small-data fast paths | + +## The three data representations used during crossmatching + +### 1. Complete HATS catalog + +The complete catalog contains all scientific and provenance columns, for +example: + +```text +CRD_ID, ra, dec, z, z_err, source, survey, +z_flag_homogenized, instrument_type_homogenized, +compared_to, tie_result, ... +``` + +It is represented as an LSDB catalog backed by a distributed +Dask/NestedFrame collection. It remains the authoritative source for complete +rows throughout the pipeline. + +The complete catalog is used to: + +- preserve every output column; +- receive the updated `compared_to` values; +- supply tie-breaking attributes during deduplication; +- produce the final consolidated catalog. + +### 2. Spatially projected HATS catalog + +The angular crossmatch does not require redshift, flags, instrument type, or +other scientific columns. Before calling LSDB crossmatch, the pipeline projects +each input catalog to: + +```text +CRD_ID, ra, dec +``` + +`source` is additionally retained on the left side when neighbor-saturation +diagnostics are enabled. + +This projection is still an LSDB/HATS catalog. Its HATS partitioning and +projected margin are preserved. Consequently, LSDB still controls the spatial +search and boundary handling. + +Projecting before the crossmatch is important. Projecting the output after a +wide crossmatch is too late: the underlying graph can still reference and +serialize the wide input NestedFrames. Early projection prevents that wide +crossmatch result from being constructed. + +### 3. Narrow pair table + +The useful output of the spatial crossmatch is an edge list: + +```text +CRD_IDleft, CRD_IDright +``` + +Optional diagnostic columns are: + +```text +_dist_arcsec, sourceleft +``` + +Inside each worker, the selected NestedFrame partition is converted to a plain +`pandas.DataFrame`. The partitions are then written as a distributed Parquet +dataset and reopened as a Dask DataFrame. + +This step has two purposes: + +1. Only the required columns cross the materialization boundary. +2. Downstream graph operations no longer retain the LSDB crossmatch graph. + +The pair table is used to remove self-matches and duplicate pairs, monitor +neighbor saturation, aggregate adjacency information, and update +`compared_to`. + +## End-to-end crossmatch flow + +```text +Complete HATS catalog A Complete HATS catalog B + | | + | project CRD_ID, ra, dec, [source] | project CRD_ID, ra, dec + v v +Projected HATS catalog A ---- LSDB crossmatch ---- Projected HATS catalog B + | + v + Narrow distributed pair table + CRD_IDleft, CRD_IDright + | + Dask groupby / aggregation + | + v + CRD_ID -> new compared_to values + / \ + v v + Dask join into complete A Dask join into complete B + \ / + v v + Dask concat of complete rows + | + v + Distributed Parquet checkpoint + | + v + hats-import rebuilds HATS + margins +``` + +The complete catalogs are not needed to calculate angular distances. They are +used immediately afterward to restore the complete scientific rows and attach +the graph edges identified by the spatial operation. + +## Why the neighbor update uses Dask joins + +After crossmatching, the pipeline constructs a table similar to: + +```text +CRD_ID _new_compared_to +A B, C +B A +C A +``` + +This is a relational join by `CRD_ID`, not a spatial join. The neighbor table +has no coordinates, HATS pixels, margins, or spatial metadata. Converting it +into an artificial HATS catalog would add work without adding spatial +correctness. + +The appropriate operation is therefore a distributed Dask merge: + +```text +complete catalog rows LEFT JOIN neighbor table ON CRD_ID +``` + +The merge updates `compared_to` while preserving all other columns from the +complete catalog. + +## Why the final crossmatch concatenation uses Dask + +After neighbor values have been joined back, the pipeline has two complete +distributed dataframes. At this stage the required operation is row-wise +tabular concatenation. There is no new spatial relationship to calculate. + +Using `LSDB Catalog.concat()` here would require wrapping the updated +dataframes back into catalog objects while retaining the lineage of: + +- the LSDB crossmatch; +- pair projection and filtering; +- distributed neighbor aggregation; +- joins into both complete catalogs; +- catalog concatenation; +- final catalog writing. + +Calling `write_catalog()` on that result would submit or retain a large, +coupled graph through the writing stage. In our production-scale runs, this +path was associated with high scheduler/driver pressure, large serialization +costs, and avoidable worker instability. + +Dask concatenation followed by Parquet staging is intentionally simpler: + +```text +Dask joins -> Dask concat -> distributed Parquet +``` + +The output remains complete; only the execution representation changes. + +## Why Parquet is an architectural boundary + +Parquet is not used merely as a file-format bridge. It is a checkpoint +between two distributed phases. + +Before the checkpoint, the graph describes how rows were produced from prior +crossmatches, aggregations, and joins. After reopening the Parquet dataset, the +graph describes only how to read the materialized partitions. + +This boundary provides: + +- shorter Dask graph lineage; +- lower scheduler bookkeeping and serialization pressure; +- release of references to earlier NestedFrame graphs; +- independent retry and inspection of staged data; +- a stable, distributed input for `hats-import`; +- explicit schema normalization before rebuilding HATS. + +## Why `hats-import` is used after staging + +The Dask concat produces complete tabular rows but does not, by itself, create a +HATS collection. `hats-import` is then responsible for reconstructing: + +- spatial pixel organization; +- HATS catalog metadata; +- catalog properties; +- the configured margin catalog. + +The result of this staging step is therefore not a permanent non-HATS catalog. +Parquet is an intermediate boundary, and the next persistent processing +artifact is again a valid HATS collection. + +## Deduplication architecture + +Crossmatching records graph edges in `compared_to`. Deduplication then solves +the connected components and assigns `tie_result` and canonical `group_id` +values. + +This stage returns to LSDB/HATS because partition boundaries and margins are +spatially meaningful. + +### HATS main/margin alignment + +The pipeline uses LSDB/HATS pixel-tree alignment through +`concat_align_catalogs`, `get_aligned_pixels_from_alignment`, and +`align_and_apply`. + +This aligns each main partition with the corresponding margin support, +including pixels represented only in the margin. It avoids assuming that the +raw Dask divisions of the main and margin dataframes are identical. + +### Local bounded solver + +For each aligned pixel, the worker receives: + +```text +main partition + aligned margin partition +``` + +Only columns required by the graph solver are retained, including: + +- `CRD_ID`; +- `compared_to`; +- redshift; +- configured tie-breaking priorities; +- star classification; +- coordinates required by optional geometry diagnostics. + +The local partitions are converted to pandas because the graph algorithm is a +bounded per-pixel computation. This does not materialize the full catalog on +the driver. Many independent partition solvers execute across Dask workers. + +The solver labels main and margin rows together but emits labels only for main +rows. Margin rows supply boundary context and are not duplicated in the final +catalog. + +### Merge of labels into complete rows + +The per-partition outputs contain only the identifiers and labels required by +the complete catalog, principally: + +```text +CRD_ID, tie_result, group_id +``` + +These labels are merged back into the complete distributed dataframe. The +complete dataframe remains lazy for HATS, Parquet, and CSV output paths. + +## Scientific semantics preserved by this design + +These representation changes do not alter the intended scientific decisions: + +- angular edges are still generated by LSDB crossmatch at the configured + radius; +- HATS margins still provide cross-partition spatial context; +- graph components are still formed from `compared_to` edges; +- stars remain outside ordinary winner selection; +- configured priority columns still determine winners and hard ties; +- redshift disambiguation and missing-redshift policy remain unchanged; +- canonical groups and tie-result invariants are applied after the same edge + construction. + +The pair table contains identifiers rather than scientific values because its +role is only to represent graph edges. Scientific values remain in the complete +catalog and are read by the deduplication solver. + +## Fast paths intentionally retained + +The pipeline does not ban `concat()` or `write_catalog()`. + +They remain appropriate when the graph is small and simple: + +- when a crossmatch finds no new pairs, the pipeline can use LSDB concat and + `write_catalog()` directly because no neighbor aggregation or join graph was + added; +- small in-memory HATS outputs can use LSDB `from_dataframe()` and + `write_catalog()`; +- large and lazy HATS outputs use Parquet staging and `hats-import`. + +The choice is based on execution scale and graph complexity, not on any claim +that the LSDB APIs are generally unsuitable. It should be read as a +workload-specific execution tradeoff. + +## Why HATS is not used for every intermediate table + +HATS provides value when a dataset needs spatial indexing or spatial boundary +semantics. Some intermediate datasets do not: + +- a `CRD_IDleft`/`CRD_IDright` edge list; +- a `CRD_ID`/neighbor aggregation table; +- per-object tie labels; +- a tabular concatenation waiting to be reimported. + +Forcing these structures into HATS would require artificial coordinates, +unnecessary indexing, extra metadata, and additional import/write cycles. It +would not improve the correctness of a key-based groupby or join. + +The design principle is therefore: + +> Use HATS whenever spatial organization affects correctness or performance; +> use distributed tabular structures when the operation is purely relational. + +## Operational trade-offs + +The checkpoint-based path has costs: + +- additional temporary disk I/O; +- temporary Parquet storage requirements; +- an explicit HATS reimport step; +- cleanup and retry logic for intermediate artifacts. + +These costs are accepted because, in this workload, they replace less +predictable scheduler and worker memory pressure with bounded distributed I/O. +On production-scale catalogs, stability and recoverability are more important +than avoiding a single intermediate write. + +## Future opportunities for a more direct LSDB path + +The current design should not prevent future LSDB optimizations. If LSDB gains +a large-catalog concat/write path that materializes partitions incrementally, +cuts lineage before writing, or accepts narrow key-based updates without +retaining the complete preceding graph, this pipeline design can be +reevaluated and potentially simplified. + +Any comparison should verify: + +- peak scheduler memory; +- peak driver and worker memory; +- serialized graph size; +- retry behavior after worker loss; +- total runtime and temporary storage; +- equivalence of output rows, HATS metadata, margins, and graph labels. + +## Final design rule + +The pipeline uses the narrowest representation that preserves the semantics of +each stage: + +```text +Spatial search and boundary alignment -> LSDB/HATS +Key-based graph operations -> Dask DataFrame +Bounded per-pixel graph solving -> pandas inside workers +Distributed execution checkpoint -> Parquet +Large spatial catalog reconstruction -> hats-import +``` + +This separation keeps HATS at the spatial core of the pipeline while allowing +large non-spatial transformations to execute with simpler, more observable, +and more recoverable distributed graphs. From c34eaac824bdb29cb6a1891a6da75423d793a1e1 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Mon, 6 Jul 2026 18:31:25 -0300 Subject: [PATCH 02/16] Minor changes in the design document --- .../crossmatch-deduplication-design.md | 57 +++++++++---------- 1 file changed, 27 insertions(+), 30 deletions(-) diff --git a/docs/architecture/crossmatch-deduplication-design.md b/docs/architecture/crossmatch-deduplication-design.md index df54eab..1ed41c3 100644 --- a/docs/architecture/crossmatch-deduplication-design.md +++ b/docs/architecture/crossmatch-deduplication-design.md @@ -14,16 +14,16 @@ The central design choice is deliberate: - pandas is used only inside bounded, per-partition computations or small-data fast paths. -This is not a replacement of HATS with a generic tabular pipeline. It is a -separation of responsibilities intended to preserve the HATS spatial model -while avoiding unnecessarily large Dask/NestedFrame graphs in operations that -do not require spatial semantics. +This architecture separates spatial and relational responsibilities. It keeps +the HATS spatial model at the core of the pipeline while using simpler +distributed tabular representations for stages where spatial semantics are not +required. ## Executive summary -The pipeline does not abandon LSDB or HATS. For this pipeline and workload -profile, it avoids using `LSDB Catalog.concat()` followed by -`write_catalog()` on large, accumulated distributed graphs. +For this pipeline and workload profile, the large-catalog path uses +`LSDB Catalog.concat()` and `write_catalog()` selectively, and stages the +heavier non-spatial transformations through Dask and Parquet boundaries. In our production-scale runs, that path kept the full LSDB/NestedFrame lineage attached through crossmatching, neighbor aggregation, catalog updates, @@ -97,10 +97,9 @@ This projection is still an LSDB/HATS catalog. Its HATS partitioning and projected margin are preserved. Consequently, LSDB still controls the spatial search and boundary handling. -Projecting before the crossmatch is important. Projecting the output after a -wide crossmatch is too late: the underlying graph can still reference and -serialize the wide input NestedFrames. Early projection prevents that wide -crossmatch result from being constructed. +Projecting before the crossmatch is important. It keeps the spatial stage narrow +from the start, so the underlying graph does not need to reference and +serialize wide input NestedFrames. ### 3. Narrow pair table @@ -160,9 +159,9 @@ Projected HATS catalog A ---- LSDB crossmatch ---- Projected HATS catalog B hats-import rebuilds HATS + margins ``` -The complete catalogs are not needed to calculate angular distances. They are -used immediately afterward to restore the complete scientific rows and attach -the graph edges identified by the spatial operation. +The complete catalogs are used immediately afterward to restore the complete +scientific rows and attach the graph edges identified by the spatial +operation. ## Why the neighbor update uses Dask joins @@ -175,12 +174,12 @@ B A C A ``` -This is a relational join by `CRD_ID`, not a spatial join. The neighbor table +This is a relational join by `CRD_ID`, rather than a spatial join. The neighbor table has no coordinates, HATS pixels, margins, or spatial metadata. Converting it into an artificial HATS catalog would add work without adding spatial correctness. -The appropriate operation is therefore a distributed Dask merge: +At this stage, the natural operation is a distributed Dask merge: ```text complete catalog rows LEFT JOIN neighbor table ON CRD_ID @@ -220,8 +219,7 @@ The output remains complete; only the execution representation changes. ## Why Parquet is an architectural boundary -Parquet is not used merely as a file-format bridge. It is a checkpoint -between two distributed phases. +Parquet serves as a checkpoint between two distributed phases. Before the checkpoint, the graph describes how rows were produced from prior crossmatches, aggregations, and joins. After reopening the Parquet dataset, the @@ -238,7 +236,7 @@ This boundary provides: ## Why `hats-import` is used after staging -The Dask concat produces complete tabular rows but does not, by itself, create a +The Dask concat produces complete tabular rows, but it does not, by itself, create a HATS collection. `hats-import` is then responsible for reconstructing: - spatial pixel organization; @@ -246,9 +244,8 @@ HATS collection. `hats-import` is then responsible for reconstructing: - catalog properties; - the configured margin catalog. -The result of this staging step is therefore not a permanent non-HATS catalog. -Parquet is an intermediate boundary, and the next persistent processing -artifact is again a valid HATS collection. +This staging step produces an intermediate boundary, after which the next +persistent processing artifact is again a valid HATS collection. ## Deduplication architecture @@ -326,7 +323,8 @@ catalog and are read by the deduplication solver. ## Fast paths intentionally retained -The pipeline does not ban `concat()` or `write_catalog()`. +The pipeline retains `concat()` and `write_catalog()` where they are a good +fit. They remain appropriate when the graph is small and simple: @@ -337,9 +335,8 @@ They remain appropriate when the graph is small and simple: `write_catalog()`; - large and lazy HATS outputs use Parquet staging and `hats-import`. -The choice is based on execution scale and graph complexity, not on any claim -that the LSDB APIs are generally unsuitable. It should be read as a -workload-specific execution tradeoff. +The choice is based on execution scale and graph complexity. It should be read +as a workload-specific execution tradeoff. ## Why HATS is not used for every intermediate table @@ -351,7 +348,7 @@ semantics. Some intermediate datasets do not: - per-object tie labels; - a tabular concatenation waiting to be reimported. -Forcing these structures into HATS would require artificial coordinates, +Representing these structures as HATS datasets would require artificial coordinates, unnecessary indexing, extra metadata, and additional import/write cycles. It would not improve the correctness of a key-based groupby or join. @@ -371,12 +368,12 @@ The checkpoint-based path has costs: These costs are accepted because, in this workload, they replace less predictable scheduler and worker memory pressure with bounded distributed I/O. -On production-scale catalogs, stability and recoverability are more important -than avoiding a single intermediate write. +At production scale, this tradeoff favors bounded execution, recoverability, +and operational clarity. ## Future opportunities for a more direct LSDB path -The current design should not prevent future LSDB optimizations. If LSDB gains +The current design remains compatible with future LSDB optimizations. If LSDB gains a large-catalog concat/write path that materializes partitions incrementally, cuts lineage before writing, or accepts narrow key-based updates without retaining the complete preceding graph, this pipeline design can be From 1799f1ff966ff87eef1aa57328c8c1dde7c2e117 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 20:05:05 -0300 Subject: [PATCH 03/16] feat: add conservative object type homogenization - add object_type_homogenized to all outputs - support survey-specific star, galaxy, and QSO translations - allow optional raw source columns without frontend mappings - preserve nulls for unknown or ambiguous classifications --- flags_translation.yaml | 208 +++++++++++++++++- packages/specz.py | 6 +- packages/specz_homogenization.py | 75 ++++++- tests/test_column_mapping.py | 1 + tests/test_homogenization.py | 364 +++++++++++++++++++++++++++++++ 5 files changed, 648 insertions(+), 6 deletions(-) diff --git a/flags_translation.yaml b/flags_translation.yaml index ba8ab7c..e877b13 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -40,7 +40,7 @@ instrument_type_priority: g: 2 # Grism p: 1 # Photometric (least reliable) -# === Survey-specific translation rules for z_flag and type === +# === Survey-specific translation rules for z_flag, instrument, and object type === # These are our internal definitions used to harmonize redshift quality and instrument types across surveys. # The quality flag system is inspired by the VVDS survey but adapted to our needs: # @@ -55,6 +55,10 @@ instrument_type_priority: # Each block corresponds to a survey name and includes: # - z_flag_translation: how to map original quality flags or based on conditions into a standardized quality scale (0-6) # - instrument_type_translation: how to map original types into a standard label (s, g, p) +# - object_type_translation: how to map classifications into star, qso, or galaxy. +# Its default is null; catalogs without object classifications are valid. +# A rule may declare `source: ` and `optional_source: true` to +# discover a non-standard input column without a frontend columns mapping. # You can mix value-based mappings and conditional rules (using Python expressions) save_expr_columns: false @@ -84,6 +88,11 @@ translation_rules: default: "s" 2DFLENS: + object_type_translation: + conditions: + - expr: "z_flag == 6" + value: "star" + default: null z_flag_translation: 1: 0 2: 1 @@ -94,6 +103,12 @@ translation_rules: default: "s" 2MRS: + object_type_translation: + conditions: + # ZCAT type -9 combines QSO and generic AGN, so it is intentionally unknown. + - expr: "str(TYPE)[:2] == '-9'" + value: null + default: "galaxy" z_flag_translation: conditions: - expr: "z_err == 0" @@ -107,6 +122,11 @@ translation_rules: default: "s" 3D-HST: + object_type_translation: + conditions: + - expr: "z_best_s == 0" + value: "star" + default: null z_flag_translation: conditions: - expr: "z_best_s == 0" @@ -129,6 +149,11 @@ translation_rules: default: "g" 6DFGS: + object_type_translation: + conditions: + - expr: "z_flag == 6" + value: "star" + default: null z_flag_translation: 1: 0 2: 1 @@ -179,6 +204,13 @@ translation_rules: default: "s" COSMOS2020_CLASSIC: + object_type_translation: + conditions: + - expr: "lp_type == 0" + value: "galaxy" + - expr: "lp_type == 1" + value: "star" + default: null z_flag_translation: conditions: - expr: "lp_type == -9 or not np.isfinite(z)" @@ -196,6 +228,11 @@ translation_rules: default: "p" COSMOS_SRC: + object_type_translation: + conditions: + - expr: "10 <= z_flag < 20" + value: "qso" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" @@ -215,6 +252,15 @@ translation_rules: default: "s" COSMOS_Web: + object_type_translation: + conditions: + - expr: "type == '0'" + value: "galaxy" + - expr: "type == '1'" + value: "star" + - expr: "type == '2'" + value: "qso" + default: null z_flag_translation: conditions: - expr: "type != 1 and z_flag != 0" @@ -232,6 +278,17 @@ translation_rules: default: "p" DEIMOS_10K: + object_type_translation: + conditions: + - expr: "Remarks.str.strip().str.lower().isin(['star', 'm star'])" + value: "star" + - expr: "Remarks.str.contains(',star$', case=False, na=False, regex=True)" + value: "star" + - expr: "Remarks.str.contains('[(]QSO[)]|,QSO$', case=False, na=False, regex=True)" + value: "qso" + - expr: "10 <= z_flag < 20" + value: "qso" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" @@ -251,6 +308,15 @@ translation_rules: default: "s" DESI: + object_type_translation: + conditions: + - expr: "SPECTYPE == 'STAR'" + value: "star" + - expr: "SPECTYPE == 'QSO'" + value: "qso" + - expr: "SPECTYPE == 'GALAXY'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "SPECTYPE != 'STAR' and z_flag != 0 and ZCAT_PRIMARY != True" @@ -268,6 +334,15 @@ translation_rules: default: "s" DESI_DR1: + object_type_translation: + conditions: + - expr: "SPECTYPE == 'STAR'" + value: "star" + - expr: "SPECTYPE == 'QSO'" + value: "qso" + - expr: "SPECTYPE == 'GALAXY'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "SPECTYPE != 'STAR' and z_flag != 0 and ZCAT_PRIMARY != True" @@ -285,6 +360,15 @@ translation_rules: default: "s" DESI_DEEP_PILOT: + object_type_translation: + conditions: + - expr: "SPECTYPE == 'STAR'" + value: "star" + - expr: "SPECTYPE == 'QSO'" + value: "qso" + - expr: "SPECTYPE == 'GALAXY'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" @@ -302,6 +386,15 @@ translation_rules: default: "s" ELAISS1OID: + object_type_translation: + conditions: + - expr: "Class == 1" + value: "qso" + - expr: "Class == 2 or Class == 3 or Class == 6" + value: "galaxy" + - expr: "Class == 4" + value: "star" + default: null z_flag_translation: 0: 0 1: 1 @@ -318,6 +411,15 @@ translation_rules: default: "p" ELAISFBMC: + object_type_translation: + conditions: + - expr: "tSp == 1 or tSp == 2 or tSp == 3" + value: "galaxy" + - expr: "tSp == 5" + value: "qso" + - expr: "tSp == 7" + value: "star" + default: null z_flag_translation: 0: 0 1: 1 @@ -334,12 +436,25 @@ translation_rules: default: "p" ELG_FIGS: + object_type_translation: + default: "galaxy" z_flag_translation: default: 3 instrument_type_translation: default: "s" EUCLID_Q1: + # spe_class is discovered directly in the raw dataframe by the backend. + # The current production catalog does not contain it, so its absence must + # safely produce null output without requiring a frontend column mapping. + object_type_translation: + source: spe_class + optional_source: true + STAR: "star" + GALAXY: "galaxy" + QSO: "qso" + UNDEF: null + default: null z_flag_translation: conditions: - expr: "z_flag == 0.0" @@ -357,6 +472,11 @@ translation_rules: default: "s" FMOS_COSMOS: + object_type_translation: + conditions: + - expr: "z_flag == -1" + value: "star" + default: null z_flag_translation: -99: 0 0: 0 @@ -442,6 +562,17 @@ translation_rules: default: "s" OZDES: + object_type_translation: + conditions: + - expr: "Object_types.str.contains('Galaxy|LRG|ELG|SN_host|SN_free_host|RedMaGiC', case=False, na=False)" + value: "galaxy" + - expr: "Object_types.str.contains('QSO', case=False, na=False)" + value: "qso" + - expr: "Object_types.str.contains('BBstar|FStar|RNDstars|BrightStar|WhiteDwarf', case=False, na=False)" + value: "star" + - expr: "z_flag == 6" + value: "star" + default: null z_flag_translation: 1: 0 2: 1 @@ -452,6 +583,13 @@ translation_rules: default: "s" PRIMUS: + object_type_translation: + conditions: + - expr: "CLASS == 'GALAXY'" + value: "galaxy" + - expr: "CLASS == 'STAR'" + value: "star" + default: null z_flag_translation: -1: 0 2: 1 @@ -461,6 +599,15 @@ translation_rules: default: "g" SDSS_DR17: + object_type_translation: + conditions: + - expr: "CLASS == 'STAR'" + value: "star" + - expr: "CLASS == 'QSO'" + value: "qso" + - expr: "CLASS == 'GALAXY'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "CLASS != 'STAR' and z_flag != 0 and SPECPRIMARY != 1" @@ -478,6 +625,15 @@ translation_rules: default: "s" SDSS_DR19: + object_type_translation: + conditions: + - expr: "CLASS == 'STAR'" + value: "star" + - expr: "CLASS == 'QSO'" + value: "qso" + - expr: "CLASS == 'GALAXY'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "CLASS != 'STAR' and z_flag != 0 and SPECPRIMARY != 1" @@ -495,6 +651,16 @@ translation_rules: default: "s" SWIRE-REVISED: + object_type_translation: + conditions: + - expr: "mst < 0" + value: "star" + - expr: "mst > 0" + value: "galaxy" + # Explicit QSO templates take precedence over the morphology scale. + - expr: "(13 <= J1 <= 30) or (13 <= J2 <= 15)" + value: "qso" + default: null z_flag_translation: 0: 0 1: 1 @@ -511,6 +677,11 @@ translation_rules: default: "p" VANDELS: + object_type_translation: + conditions: + - expr: "np.trunc(float(z_flag)) == 13 or np.trunc(float(z_flag)) == 14 or np.trunc(float(z_flag)) == 19 or np.trunc(float(z_flag)) == 213 or np.trunc(float(z_flag)) == 214 or np.trunc(float(z_flag)) == 219" + value: "qso" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" @@ -530,6 +701,13 @@ translation_rules: default: "s" VIMOS: + object_type_translation: + conditions: + - expr: "COMM.str.strip().str.lower() == 'star'" + value: "star" + - expr: "COMM.str.contains('[(]BLAGN[)]', case=False, na=False, regex=True)" + value: "qso" + default: null z_flag_translation: 4: 4 3: 3 @@ -540,6 +718,15 @@ translation_rules: default: "s" VIPERS_PDR2: + object_type_translation: + conditions: + - expr: "classFlag == -1" + value: "star" + # Spectroscopic BLAGN classification takes precedence over the + # photometric stellar-like classification. + - expr: "np.trunc(float(z_flag)) == 13 or np.trunc(float(z_flag)) == 14 or np.trunc(float(z_flag)) == 19 or np.trunc(float(z_flag)) == 213 or np.trunc(float(z_flag)) == 214 or np.trunc(float(z_flag)) == 219" + value: "qso" + default: null z_flag_translation: conditions: - expr: "classFlag == -1" @@ -576,6 +763,11 @@ translation_rules: default: "s" VVDS: + object_type_translation: + conditions: + - expr: "10 <= z_flag < 20" + value: "qso" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" @@ -595,6 +787,15 @@ translation_rules: default: "s" XMM_LSS: + object_type_translation: + conditions: + - expr: "Class == 'STA'" + value: "star" + - expr: "Class == 'QSO'" + value: "qso" + - expr: "Class == 'ALG' or Class == 'ELG'" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "Class == 'STA'" @@ -610,6 +811,11 @@ translation_rules: default: "s" ZCOSMOS: + object_type_translation: + conditions: + - expr: "10 <= z_flag < 20" + value: "qso" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 0" diff --git a/packages/specz.py b/packages/specz.py index c38acce..f632790 100644 --- a/packages/specz.py +++ b/packages/specz.py @@ -1340,6 +1340,7 @@ def _validate_and_rename( "z": DTYPE_FLOAT, "z_flag": DTYPE_FLOAT, "z_err": DTYPE_FLOAT, + "object_type": DTYPE_STR, } for col, pd_dtype in base_schema.items(): if col not in df.columns: @@ -2180,7 +2181,10 @@ def _select_output_columns( "group_id", ] - # Optional homogenized fields. + # Homogenized object type is part of every output, including all-null catalogs. + final_cols.append("object_type_homogenized") + + # Optional homogenized fields used by the legacy quality/ranking logic. if "z_flag_homogenized" in df.columns: final_cols.append("z_flag_homogenized") if "instrument_type_homogenized" in df.columns: diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index 1c2711a..3914e2f 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -6,6 +6,7 @@ homogenized columns: - z_flag_homogenized - instrument_type_homogenized + - object_type_homogenized It is intended to be imported by the main `specz.py` module. Functions are copied verbatim from the original file to avoid behavior changes. @@ -129,6 +130,17 @@ def _ensure_single_series(df: dd.DataFrame, name: str): else: logger.warning(f"{product_name} YAML says instrument_type<-instrument_type_homogenized, but missing.") + if norm(cols_cfg.get("object_type")) == "object_type_homogenized": + if "object_type" in df.columns: + df["object_type_homogenized"] = _ensure_single_series(df, "object_type") + logger.info( + f"{product_name} Using user-provided homogenized object_type via YAML mapping." + ) + else: + logger.warning( + f"{product_name} YAML says object_type<-object_type_homogenized, but missing." + ) + return df @@ -223,8 +235,12 @@ def _homogenize( # ----------------------- def _translate_column_vectorized(df: dd.DataFrame, key: str, out_col: str, out_kind: str) -> dd.DataFrame: """Apply YAML translation rules per partition.""" - assert key in {"z_flag", "instrument_type"} - assert out_col in {"z_flag_homogenized", "instrument_type_homogenized"} + assert key in {"z_flag", "instrument_type", "object_type"} + assert out_col in { + "z_flag_homogenized", + "instrument_type_homogenized", + "object_type_homogenized", + } assert out_kind in {"float", "str"} def _partition(p: pd.DataFrame) -> pd.DataFrame: @@ -320,9 +336,27 @@ def visit_Compare(self, node): fill_vals = pd.Series([default_val] * int(mask_s.sum()), index=out.index[mask_s], dtype=DTYPE_STR) out.loc[mask_s] = out.loc[mask_s].fillna(fill_vals) - direct = {k: v for k, v in rule.items() if k not in {"conditions", "default"}} + source_col = str(rule.get("source", key)) + optional_source = bool(rule.get("optional_source", False)) + direct = { + k: v + for k, v in rule.items() + if k not in { + "conditions", + "default", + "source", + "optional_source", + } + } if direct: - col = s.loc[mask_s, key] + if source_col not in s.columns: + if optional_source: + continue + raise ValueError( + f"Missing source column '{source_col}' for survey " + f"'{sname}' and translation '{out_col}'." + ) + col = s.loc[mask_s, source_col] is_num = pd.api.types.is_numeric_dtype(col) if key == "z_flag": is_num = True @@ -561,6 +595,39 @@ def can_use_type_for_instrument() -> bool: # Keep normalized lower-case values for consistency df["instrument_type_homogenized"] = normed + # object_type_homogenized is an output-schema field, not a ranking field. + # An entirely-null result is valid for catalogs without classification data. + if "object_type_homogenized" not in df.columns: + df = _translate_column_vectorized( + df, + key="object_type", + out_col="object_type_homogenized", + out_kind="str", + ) + + object_types = df["object_type_homogenized"].map_partitions( + _normalize_string_series_to_na, + meta=pd.Series(pd.array([], dtype=DTYPE_STR)), + ).str.lower() + allowed_object_types = {"star", "qso", "galaxy"} + invalid_object_mask = (~dd.isna(object_types)) & ~object_types.isin( + list(allowed_object_types) + ) + invalid_object_count = int(invalid_object_mask.sum().compute()) + if invalid_object_count: + examples = ( + df["object_type_homogenized"] + .loc[invalid_object_mask] + .head(5, compute=True) + .tolist() + ) + raise ValueError( + f"[{product_name}] Invalid values in 'object_type_homogenized'. " + f"Allowed set is {sorted(allowed_object_types)} (NaN allowed). " + f"Examples of invalid values: {examples}" + ) + df["object_type_homogenized"] = object_types + # --- post-homogenization sanity checks (required columns must not be all-NaN) --- if needs_z_flag: if "z_flag_homogenized" not in df.columns: diff --git a/tests/test_column_mapping.py b/tests/test_column_mapping.py index 24689c2..2f67997 100644 --- a/tests/test_column_mapping.py +++ b/tests/test_column_mapping.py @@ -72,6 +72,7 @@ def test_column_mapping_ignores_null_mapping_and_creates_base_schema(): "z", "z_flag", "z_err", + "object_type", "source", } assert expected.issubset(result.columns) diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index ebe14b1..2238da6 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -7,6 +7,7 @@ import dask.dataframe as dd import pandas as pd import pytest +import yaml sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "packages")) if "tables_io" not in sys.modules: @@ -54,6 +55,369 @@ def test_yaml_flag_translation_applies_direct_default_and_condition(): assert result.compute()["z_flag_homogenized"].astype(float).tolist() == [2, 0, 4] +def test_object_type_is_always_present_and_may_be_entirely_null(): + frame = dd.from_pandas( + pd.DataFrame({"survey": ["unknown", "unknown"]}), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, {}, "demo", LOGGER, type_cast_ok=False) + + computed = result.compute() + assert "object_type_homogenized" in computed + assert computed["object_type_homogenized"].isna().all() + + +def test_object_type_translation_uses_canonical_renamed_z_flag(): + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["demo", "demo", "demo"], + "object_type": [pd.NA, pd.NA, pd.NA], + "z_flag": [13, 4, 24], + } + ), + npartitions=1, + sort=False, + ) + config = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "conditions": [ + {"expr": "10 <= z_flag < 20", "value": "qso"} + ], + "default": None, + } + } + } + } + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].tolist() == [ + "qso", + pd.NA, + pd.NA, + ] + + +def test_object_type_rejects_values_outside_domain(): + frame = dd.from_pandas( + pd.DataFrame({"object_type_homogenized": ["STAR", "agn"]}), + npartitions=1, + sort=False, + ) + + with pytest.raises(ValueError, match="Invalid values"): + _homogenize(frame, {}, "demo", LOGGER, type_cast_ok=False) + + +def test_catalog_object_type_rules_use_renamed_flags_and_conservative_defaults(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": [ + "PRIMUS", + "VVDS", + "VUDS", + "VANDELS", + "VIPERS_PDR2", + "OZDES", + "OZDES", + ], + "object_type": [pd.NA] * 7, + "z_flag": [4, 14.5, 23, 214, 213.5, 4, 6], + "CLASS": ["AGN", pd.NA, pd.NA, pd.NA, pd.NA, pd.NA, pd.NA], + "classFlag": [pd.NA, pd.NA, pd.NA, pd.NA, 1, pd.NA, pd.NA], + "Object_types": [ + pd.NA, + pd.NA, + pd.NA, + pd.NA, + pd.NA, + "Photo-z,LRG", + "QSO_faint", + ], + } + ), + npartitions=2, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "missing", + "qso", + "missing", + "qso", + "qso", + "galaxy", + "star", + ] + + +def test_3dhst_and_mosdef_object_type_rules_keep_unknowns_null(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["3D-HST", "3D-HST", "MOSDEF", "MOSDEF", "GAMA_DR4"], + "object_type": [pd.NA] * 5, + "z_best_s": [0.0, 1.0, float("nan"), float("nan"), float("nan")], + "TARGET": [float("nan"), float("nan"), 1.0, 0.0, float("nan")], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "star", + "missing", + "missing", + "missing", + "missing", + ] + + +def test_swire_j2_qso_range_stops_at_15(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["SWIRE-REVISED"] * 7, + "object_type": [pd.NA] * 7, + "mst": [0, 0, -1, 1, -5, 5, -5], + "J1": [1, 1, 1, 1, 1, 1, 13], + "J2": [15, 16, 16, 16, 16, 16, 16], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "qso", + "missing", + "star", + "galaxy", + "star", + "galaxy", + "qso", + ] + + +def test_2df_6df_and_2mrs_use_only_documented_object_classes(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["2DFLENS", "2DFLENS", "6DFGS", "2MRS", "2MRS"], + "object_type": [pd.NA] * 5, + "z_flag": [6, 4, 6, 4, 4], + "TYPE": [pd.NA, pd.NA, pd.NA, "-5A", "-9"], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "star", + "missing", + "star", + "galaxy", + "missing", + ] + + +def test_vimos_uses_only_unambiguous_comm_classifications(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["VIMOS"] * 5, + "object_type": [pd.NA] * 5, + "COMM": [ + "star", + "Star", + "Star?", + "CIV_[CIII]_(BLAGN)", + "Lya(em)_CIV_(BLAGN?)", + ], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "star", + "star", + "missing", + "qso", + "missing", + ] + + +def test_deimos_remarks_use_only_unambiguous_classifications(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["DEIMOS_10K"] * 8, + "object_type": [pd.NA] * 8, + "z_flag": [4] * 8, + "Remarks": [ + "star", + "M star", + "Star?", + "in halo of bright star", + "NaI,TiO,Ha,star", + "MgII,QSO?", + "CIII],NeIV?(br),QSO", + "MgII(br),MgII(abs),[NeV]br,[OII]br(QSO)", + ], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "star", + "star", + "missing", + "missing", + "star", + "missing", + "qso", + "qso", + ] + + +def test_vipers_blagn_takes_precedence_over_photometric_star_like_flag(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["VIPERS_PDR2"] * 3, + "object_type": [pd.NA] * 3, + "z_flag": [4.2, 13.2, 213.2], + "classFlag": [-1, -1, -1], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "star", + "qso", + "qso", + ] + + +def test_ozdes_explicit_stellar_targets_are_stellar(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["OZDES"] * 3, + "object_type": [pd.NA] * 3, + "Object_types": ["RNDstars", "BrightStar", "WhiteDwarf"], + "z_flag": [4] * 3, + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].tolist() == [ + "star", + "star", + "star", + ] + + +def test_euclid_optional_source_is_safe_before_column_arrives(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame_without_source = dd.from_pandas( + pd.DataFrame( + { + "survey": ["EUCLID_Q1"], + "object_type": [pd.NA], + } + ), + npartitions=1, + sort=False, + ) + missing_result, *_ = _homogenize( + frame_without_source, config, "demo", LOGGER, type_cast_ok=False + ) + assert missing_result.compute()["object_type_homogenized"].isna().all() + + frame_with_source = dd.from_pandas( + pd.DataFrame( + { + "survey": ["EUCLID_Q1"] * 5, + "object_type": [pd.NA] * 5, + "spe_class": [pd.NA, "STAR", "GALAXY", "QSO", "UNDEF"], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize( + frame_with_source, config, "demo", LOGGER, type_cast_ok=False + ) + + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "missing", + "star", + "galaxy", + "qso", + "missing", + ] + + def test_homogenization_requires_translation_for_every_survey(): frame = dd.from_pandas( pd.DataFrame({"survey": ["known", "missing"], "z_flag": [1, 1]}), From 0f6122583a17efb0730ab211cf0396d70573e6d2 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 20:11:26 -0300 Subject: [PATCH 04/16] Adding column value 'agn' for generic AGN, in contrast to certain quasars --- flags_translation.yaml | 48 ++++++++++++------ packages/specz_homogenization.py | 2 +- tests/test_homogenization.py | 86 +++++++++++++++++++++++++++----- 3 files changed, 108 insertions(+), 28 deletions(-) diff --git a/flags_translation.yaml b/flags_translation.yaml index e877b13..96d594f 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -55,7 +55,9 @@ instrument_type_priority: # Each block corresponds to a survey name and includes: # - z_flag_translation: how to map original quality flags or based on conditions into a standardized quality scale (0-6) # - instrument_type_translation: how to map original types into a standard label (s, g, p) -# - object_type_translation: how to map classifications into star, qso, or galaxy. +# - object_type_translation: how to map classifications into star, qso, agn, +# or galaxy. `qso` requires an explicit QSO/quasar classification; `agn` is +# used for generic AGN and BLAGN classifications without an explicit QSO label. # Its default is null; catalogs without object classifications are valid. # A rule may declare `source: ` and `optional_source: true` to # discover a non-standard input column without a frontend columns mapping. @@ -105,9 +107,10 @@ translation_rules: 2MRS: object_type_translation: conditions: - # ZCAT type -9 combines QSO and generic AGN, so it is intentionally unknown. + # ZCAT type -9 combines QSO and generic AGN; both belong to the broader + # AGN class, but the information is insufficient for the QSO subtype. - expr: "str(TYPE)[:2] == '-9'" - value: null + value: "agn" default: "galaxy" z_flag_translation: conditions: @@ -231,7 +234,7 @@ translation_rules: object_type_translation: conditions: - expr: "10 <= z_flag < 20" - value: "qso" + value: "agn" default: null z_flag_translation: conditions: @@ -280,14 +283,14 @@ translation_rules: DEIMOS_10K: object_type_translation: conditions: + - expr: "10 <= z_flag < 20" + value: "agn" - expr: "Remarks.str.strip().str.lower().isin(['star', 'm star'])" value: "star" - expr: "Remarks.str.contains(',star$', case=False, na=False, regex=True)" value: "star" - expr: "Remarks.str.contains('[(]QSO[)]|,QSO$', case=False, na=False, regex=True)" value: "qso" - - expr: "10 <= z_flag < 20" - value: "qso" default: null z_flag_translation: conditions: @@ -389,7 +392,9 @@ translation_rules: object_type_translation: conditions: - expr: "Class == 1" - value: "qso" + value: "agn" + - expr: "Class == 5" + value: "agn" - expr: "Class == 2 or Class == 3 or Class == 6" value: "galaxy" - expr: "Class == 4" @@ -415,8 +420,8 @@ translation_rules: conditions: - expr: "tSp == 1 or tSp == 2 or tSp == 3" value: "galaxy" - - expr: "tSp == 5" - value: "qso" + - expr: "tSp == 4 or tSp == 5 or tSp == 6 or tSp == 8" + value: "agn" - expr: "tSp == 7" value: "star" default: null @@ -499,6 +504,12 @@ translation_rules: default: "s" HELP-DMU23: + object_type_translation: + conditions: + # The HELP flag combines QSO and AGN, so only the broader class is safe. + - expr: "agn_flag == 1" + value: "agn" + default: null z_flag_translation: 1: 0 2: 1 @@ -566,6 +577,8 @@ translation_rules: conditions: - expr: "Object_types.str.contains('Galaxy|LRG|ELG|SN_host|SN_free_host|RedMaGiC', case=False, na=False)" value: "galaxy" + - expr: "Object_types.str.contains('AGN_reverberation|AGN_monitoring', case=False, na=False)" + value: "agn" - expr: "Object_types.str.contains('QSO', case=False, na=False)" value: "qso" - expr: "Object_types.str.contains('BBstar|FStar|RNDstars|BrightStar|WhiteDwarf', case=False, na=False)" @@ -589,6 +602,8 @@ translation_rules: value: "galaxy" - expr: "CLASS == 'STAR'" value: "star" + - expr: "CLASS == 'AGN'" + value: "agn" default: null z_flag_translation: -1: 0 @@ -680,7 +695,7 @@ translation_rules: object_type_translation: conditions: - expr: "np.trunc(float(z_flag)) == 13 or np.trunc(float(z_flag)) == 14 or np.trunc(float(z_flag)) == 19 or np.trunc(float(z_flag)) == 213 or np.trunc(float(z_flag)) == 214 or np.trunc(float(z_flag)) == 219" - value: "qso" + value: "agn" default: null z_flag_translation: conditions: @@ -706,7 +721,7 @@ translation_rules: - expr: "COMM.str.strip().str.lower() == 'star'" value: "star" - expr: "COMM.str.contains('[(]BLAGN[)]', case=False, na=False, regex=True)" - value: "qso" + value: "agn" default: null z_flag_translation: 4: 4 @@ -725,7 +740,7 @@ translation_rules: # Spectroscopic BLAGN classification takes precedence over the # photometric stellar-like classification. - expr: "np.trunc(float(z_flag)) == 13 or np.trunc(float(z_flag)) == 14 or np.trunc(float(z_flag)) == 19 or np.trunc(float(z_flag)) == 213 or np.trunc(float(z_flag)) == 214 or np.trunc(float(z_flag)) == 219" - value: "qso" + value: "agn" default: null z_flag_translation: conditions: @@ -746,6 +761,11 @@ translation_rules: default: "s" VUDS: + object_type_translation: + conditions: + - expr: "10 <= z_flag < 20" + value: "agn" + default: null z_flag_translation: conditions: - expr: "np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 1" @@ -766,7 +786,7 @@ translation_rules: object_type_translation: conditions: - expr: "10 <= z_flag < 20" - value: "qso" + value: "agn" default: null z_flag_translation: conditions: @@ -814,7 +834,7 @@ translation_rules: object_type_translation: conditions: - expr: "10 <= z_flag < 20" - value: "qso" + value: "agn" default: null z_flag_translation: conditions: diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index 3914e2f..a33c4c3 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -609,7 +609,7 @@ def can_use_type_for_instrument() -> bool: _normalize_string_series_to_na, meta=pd.Series(pd.array([], dtype=DTYPE_STR)), ).str.lower() - allowed_object_types = {"star", "qso", "galaxy"} + allowed_object_types = {"star", "qso", "agn", "galaxy"} invalid_object_mask = (~dd.isna(object_types)) & ~object_types.isin( list(allowed_object_types) ) diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index 2238da6..bb1c8c8 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -104,14 +104,21 @@ def test_object_type_translation_uses_canonical_renamed_z_flag(): def test_object_type_rejects_values_outside_domain(): - frame = dd.from_pandas( + valid = dd.from_pandas( pd.DataFrame({"object_type_homogenized": ["STAR", "agn"]}), npartitions=1, sort=False, ) + result, *_ = _homogenize(valid, {}, "demo", LOGGER, type_cast_ok=False) + assert result.compute()["object_type_homogenized"].tolist() == ["star", "agn"] + invalid = dd.from_pandas( + pd.DataFrame({"object_type_homogenized": ["unknown"]}), + npartitions=1, + sort=False, + ) with pytest.raises(ValueError, match="Invalid values"): - _homogenize(frame, {}, "demo", LOGGER, type_cast_ok=False) + _homogenize(invalid, {}, "demo", LOGGER, type_cast_ok=False) def test_catalog_object_type_rules_use_renamed_flags_and_conservative_defaults(): @@ -152,11 +159,11 @@ def test_catalog_object_type_rules_use_renamed_flags_and_conservative_defaults() result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "agn", + "agn", "missing", - "qso", - "missing", - "qso", - "qso", + "agn", + "agn", "galaxy", "star", ] @@ -245,7 +252,7 @@ def test_2df_6df_and_2mrs_use_only_documented_object_classes(): "missing", "star", "galaxy", - "missing", + "agn", ] @@ -277,7 +284,7 @@ def test_vimos_uses_only_unambiguous_comm_classifications(): "star", "star", "missing", - "qso", + "agn", "missing", ] @@ -289,9 +296,9 @@ def test_deimos_remarks_use_only_unambiguous_classifications(): frame = dd.from_pandas( pd.DataFrame( { - "survey": ["DEIMOS_10K"] * 8, - "object_type": [pd.NA] * 8, - "z_flag": [4] * 8, + "survey": ["DEIMOS_10K"] * 10, + "object_type": [pd.NA] * 10, + "z_flag": [4] * 8 + [14, 14], "Remarks": [ "star", "M star", @@ -301,6 +308,8 @@ def test_deimos_remarks_use_only_unambiguous_classifications(): "MgII,QSO?", "CIII],NeIV?(br),QSO", "MgII(br),MgII(abs),[NeV]br,[OII]br(QSO)", + "-", + "CIV,QSO", ], } ), @@ -319,6 +328,57 @@ def test_deimos_remarks_use_only_unambiguous_classifications(): "missing", "qso", "qso", + "agn", + "qso", + ] + + +def test_generic_agn_labels_are_not_promoted_to_qso(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = [] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": [ + "HELP-DMU23", + "OZDES", + "OZDES", + "ELAISS1OID", + "ELAISS1OID", + "ELAISFBMC", + "ELAISFBMC", + ], + "object_type": [pd.NA] * 7, + "agn_flag": [1.0] + [float("nan")] * 6, + "Object_types": [ + pd.NA, + "AGN_reverberation", + "AGN_reverberation,XXL_QSO", + pd.NA, + pd.NA, + pd.NA, + pd.NA, + ], + "z_flag": [4.0] * 7, + "Class": [float("nan")] * 3 + [1.0, 5.0] + [float("nan")] * 2, + "tSp": [float("nan")] * 5 + [4.0, 5.0], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["object_type_homogenized"].tolist() == [ + "agn", + "agn", + "qso", + "agn", + "agn", + "agn", + "agn", ] @@ -343,8 +403,8 @@ def test_vipers_blagn_takes_precedence_over_photometric_star_like_flag(): assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ "star", - "qso", - "qso", + "agn", + "agn", ] From 14de10bbaab3c1e5fc47b0ad1dbf0d7dee64bd54 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 20:36:24 -0300 Subject: [PATCH 05/16] feat: harden translation rules and disable Rubin footprint by default - validate user-defined translation schemas and output domains - restrict expressions to a safe AST-based DSL - add configurable fast-path policies - detect conflicting conditional translations - improve optional source handling and validation errors - fix COSMOS Web and JADES translation edge cases - disable Rubin footprint flag generation by default --- config.py | 2 +- config.template.yaml | 2 +- flags_translation.yaml | 23 +- packages/specz.py | 2 +- packages/specz_homogenization.py | 362 +++++++++++++++++++++++++++---- scripts/crd-run.py | 7 +- tests/test_homogenization.py | 259 +++++++++++++++++++++- 7 files changed, 610 insertions(+), 47 deletions(-) diff --git a/config.py b/config.py index b508efb..387e10a 100644 --- a/config.py +++ b/config.py @@ -105,7 +105,7 @@ class Param(BaseModel): z_flag_homogenized_value_to_cut: float = 3.0 flags_translation_file: str = str(Path(MAINDIR, "flags_translation.yaml")) insert_DP1_footprint_flag: bool = False - insert_rubin_footprint_flag: bool = True + insert_rubin_footprint_flag: bool = False class Config(BaseModel): diff --git a/config.template.yaml b/config.template.yaml index dd92a79..572096b 100644 --- a/config.template.yaml +++ b/config.template.yaml @@ -89,5 +89,5 @@ param: # type: str z_flag_homogenized_value_to_cut: 0.0 # 0 disables the cut; valid active cuts are 1, 2, 3, 4, 5, 6. insert_DP1_footprint_flag: false # Adds is_in_DP1_fields (1/0) - insert_rubin_footprint_flag: true # Adds is_in_rubin_footprint (1/0) + insert_rubin_footprint_flag: false # Adds is_in_rubin_footprint (1/0) flags_translation_file: flags_translation.yaml # File with homogenization rules for z_flag and type diff --git a/flags_translation.yaml b/flags_translation.yaml index 96d594f..ae25c5d 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -62,6 +62,12 @@ instrument_type_priority: # A rule may declare `source: ` and `optional_source: true` to # discover a non-standard input column without a frontend columns mapping. # You can mix value-based mappings and conditional rules (using Python expressions) +# Conditions are evaluated in YAML order. When multiple conditions match a row, +# the later condition takes precedence ("last match wins"). Conflicting matches +# emit a warning unless that translation declares `allow_condition_overlap: true`. +# z_flag and instrument translations accept `fast_path: auto|disabled|required`. +# `auto` is the default, `disabled` forces YAML rules, and `required` fails when +# the input is not compatible with the corresponding probability/s-g-p fast path. save_expr_columns: false @@ -215,6 +221,7 @@ translation_rules: value: "star" default: null z_flag_translation: + allow_condition_overlap: true conditions: - expr: "lp_type == -9 or not np.isfinite(z)" value: 0 @@ -266,15 +273,15 @@ translation_rules: default: null z_flag_translation: conditions: - - expr: "type != 1 and z_flag != 0" + - expr: "type != '1' and z_flag != 0" value: 0 - - expr: "type != 1 and z_flag == 0 and np.abs(zpdf_med - zchi2) > 0.001" + - expr: "type != '1' and z_flag == 0 and np.abs(zpdf_med - zchi2) > 0.001" value: 1 - - expr: "type != 1 and z_flag == 0 and np.abs(zpdf_med - zchi2) <= 0.001 and nbfilt < 30" + - expr: "type != '1' and z_flag == 0 and np.abs(zpdf_med - zchi2) <= 0.001 and nbfilt < 30" value: 2 - - expr: "type != 1 and z_flag == 0 and np.abs(zpdf_med - zchi2) <= 0.001 and nbfilt > 30" + - expr: "type != '1' and z_flag == 0 and np.abs(zpdf_med - zchi2) <= 0.001 and nbfilt >= 30" value: 3 - - expr: "type == 1" + - expr: "type == '1'" value: 6 default: 0 instrument_type_translation: @@ -282,6 +289,7 @@ translation_rules: DEIMOS_10K: object_type_translation: + allow_condition_overlap: true conditions: - expr: "10 <= z_flag < 20" value: "agn" @@ -531,6 +539,7 @@ translation_rules: JADES_DR5_PHOTOZ: z_flag_translation: + allow_condition_overlap: true conditions: - expr: "not np.isfinite(z) or not np.isfinite(z_ml) or not np.isfinite(chi_a) or not np.isfinite(nfilt) or nfilt <= 0" value: 0 @@ -546,6 +555,7 @@ translation_rules: JADES_DR5_PHOTOZ_KRON: z_flag_translation: + allow_condition_overlap: true conditions: - expr: "not np.isfinite(z) or not np.isfinite(z_ml) or not np.isfinite(chi_a) or not np.isfinite(nfilt) or nfilt <= 0" value: 0 @@ -574,6 +584,7 @@ translation_rules: OZDES: object_type_translation: + allow_condition_overlap: true conditions: - expr: "Object_types.str.contains('Galaxy|LRG|ELG|SN_host|SN_free_host|RedMaGiC', case=False, na=False)" value: "galaxy" @@ -667,6 +678,7 @@ translation_rules: SWIRE-REVISED: object_type_translation: + allow_condition_overlap: true conditions: - expr: "mst < 0" value: "star" @@ -734,6 +746,7 @@ translation_rules: VIPERS_PDR2: object_type_translation: + allow_condition_overlap: true conditions: - expr: "classFlag == -1" value: "star" diff --git a/packages/specz.py b/packages/specz.py index f632790..f11ecca 100644 --- a/packages/specz.py +++ b/packages/specz.py @@ -2650,7 +2650,7 @@ def prepare_catalog( param_config.get("insert_DP1_footprint_flag", False), default=False ) insert_rubin = _as_bool_config( - param_config.get("insert_rubin_footprint_flag", True), default=True + param_config.get("insert_rubin_footprint_flag", False), default=False ) if insert_dp1: diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index a33c4c3..c160f79 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -21,6 +21,7 @@ # ----------------------- import ast as _ast import builtins +import difflib import logging import math @@ -67,6 +68,186 @@ "LRB_A": 4.0, "MR_A": 4.0, } +_TRANSLATION_KEYS = { + "z_flag_translation": ("float", {0.0, 1.0, 2.0, 3.0, 4.0, 6.0}), + "instrument_type_translation": ("str", {"s", "g", "p"}), + "object_type_translation": ("str", {"star", "qso", "agn", "galaxy"}), +} +_RULE_OPTIONS = { + "conditions", "default", "source", "optional_source", + "allow_condition_overlap", "fast_path", +} +_TOP_LEVEL_KEYS = { + "tiebreaking_priority", "delta_z_threshold", "crossmatch_radius_arcsec", + "margin_threshold_arcsec", "margin_warning_fraction", + "validate_global_graph_edges", "validate_global_tie_invariants", + "validate_crd_id_uniqueness", "repartition_prepared_catalogs", + "prepared_partition_size", "crossmatch_n_neighbors", + "crossmatch_saturation_enabled", "crossmatch_saturation_warn_fraction", + "crossmatch_saturation_fail_fraction", + "crossmatch_geometry_diagnostics_enabled", + "representative_radius_diagnostics_enabled", "dedup_edge_diagnostics_enabled", + "instrument_type_priority", "save_expr_columns", "expr_column_schema", + "translation_rules", +} +_SAFE_FUNCTIONS = {"len", "int", "float", "str"} +_SAFE_NUMPY_FUNCTIONS = {"isfinite", "abs", "trunc"} +_SAFE_SERIES_METHODS = {"isin", "contains", "strip", "lower"} +_SAFE_AST_NODES = ( + _ast.Expression, _ast.BoolOp, _ast.BinOp, _ast.UnaryOp, _ast.Compare, + _ast.Name, _ast.Load, _ast.Constant, _ast.Call, _ast.Attribute, + _ast.Subscript, _ast.Slice, _ast.List, _ast.Tuple, _ast.keyword, + _ast.And, _ast.Or, _ast.Not, _ast.Invert, _ast.UAdd, _ast.USub, + _ast.Add, _ast.Sub, _ast.Mult, _ast.Div, _ast.FloorDiv, _ast.Mod, + _ast.Pow, _ast.BitAnd, _ast.BitOr, _ast.Eq, _ast.NotEq, _ast.Lt, + _ast.LtE, _ast.Gt, _ast.GtE, _ast.In, _ast.NotIn, +) + + +class _SafeExpressionValidator(_ast.NodeVisitor): + """Reject expression syntax outside the small vectorized-rule DSL.""" + + def __init__(self, path: str): + self.path = path + + def fail(self, node: _ast.AST, detail: str) -> None: + raise ValueError( + f"{self.path}: forbidden expression element {type(node).__name__}: {detail}" + ) + + def generic_visit(self, node: _ast.AST) -> None: + if not isinstance(node, _SAFE_AST_NODES): + self.fail(node, "not allowed by the translation expression DSL") + super().generic_visit(node) + + def visit_Name(self, node: _ast.Name) -> None: + if node.id.startswith("_"): + self.fail(node, f"private name {node.id!r} is not allowed") + + def visit_Attribute(self, node: _ast.Attribute) -> None: + if node.attr.startswith("_"): + self.fail(node, f"private attribute {node.attr!r} is not allowed") + if isinstance(node.value, _ast.Name) and node.value.id == "np": + if node.attr not in _SAFE_NUMPY_FUNCTIONS: + self.fail(node, f"np.{node.attr} is not allowed") + elif node.attr not in _SAFE_SERIES_METHODS and node.attr != "str": + self.fail(node, f"attribute or method {node.attr!r} is not allowed") + self.generic_visit(node) + + def visit_Call(self, node: _ast.Call) -> None: + if isinstance(node.func, _ast.Name): + if node.func.id not in _SAFE_FUNCTIONS: + self.fail(node, f"function {node.func.id!r} is not allowed") + elif isinstance(node.func, _ast.Attribute): + self.visit_Attribute(node.func) + else: + self.fail(node, "only whitelisted named functions and methods are allowed") + for arg in node.args: + self.visit(arg) + for keyword in node.keywords: + if keyword.arg is None or keyword.arg.startswith("_"): + self.fail(keyword, "keyword expansion/private keywords are not allowed") + self.visit(keyword.value) + + +def _validate_expression(expr: str, path: str) -> None: + try: + tree = _ast.parse(expr, mode="eval") + except SyntaxError as exc: + raise ValueError(f"{path}: invalid expression syntax: {exc.msg}") from exc + _SafeExpressionValidator(path).visit(tree) + + +def validate_translation_config(config: dict) -> None: + """Validate user-editable translation rules before catalog processing.""" + if not isinstance(config, dict): + raise TypeError("flags translation root must be a mapping") + unknown_top = set(config) - _TOP_LEVEL_KEYS + if unknown_top: + raise ValueError(f"flags translation root: unknown option(s): {sorted(unknown_top)}") + rules = config.get("translation_rules", {}) + if not isinstance(rules, dict): + raise TypeError("translation_rules must be a mapping") + for survey, ruleset in rules.items(): + base = f"translation_rules.{survey}" + if not isinstance(survey, str) or not survey.strip(): + raise ValueError("translation_rules survey names must be non-empty strings") + if not isinstance(ruleset, dict): + raise TypeError(f"{base} must be a mapping") + unknown_blocks = set(ruleset) - set(_TRANSLATION_KEYS) + if unknown_blocks: + raise ValueError(f"{base}: unknown option(s): {sorted(unknown_blocks)}") + for block, (kind, allowed) in _TRANSLATION_KEYS.items(): + if block not in ruleset: + continue + path = f"{base}.{block}" + rule = ruleset[block] + if not isinstance(rule, dict): + raise TypeError(f"{path} must be a mapping") + for option in ("optional_source", "allow_condition_overlap"): + if option in rule and not isinstance(rule[option], bool): + raise TypeError(f"{path}.{option} must be a boolean") + if "source" in rule and ( + not isinstance(rule["source"], str) or not rule["source"].strip() + ): + raise ValueError(f"{path}.source must be a non-empty string") + policy = rule.get("fast_path", "auto") + if policy not in {"auto", "disabled", "required"}: + raise ValueError( + f"{path}.fast_path must be one of ['auto', 'disabled', 'required']" + ) + conditions = rule.get("conditions", []) + if not isinstance(conditions, list): + raise TypeError(f"{path}.conditions must be a list") + for index, condition in enumerate(conditions): + cpath = f"{path}.conditions[{index}]" + if not isinstance(condition, dict): + raise TypeError(f"{cpath} must be a mapping") + unknown = set(condition) - {"expr", "value"} + if unknown: + raise ValueError(f"{cpath}: unknown option(s): {sorted(unknown)}") + if not isinstance(condition.get("expr"), str) or not condition["expr"].strip(): + raise ValueError(f"{cpath}.expr must be a non-empty string") + if "value" not in condition: + raise ValueError(f"{cpath}.value is required") + _validate_expression(condition["expr"], f"{cpath}.expr") + values = [] + if "default" in rule: + values.append((f"{path}.default", rule["default"])) + values.extend( + (f"{path}.conditions[{i}].value", c["value"]) + for i, c in enumerate(conditions) + ) + for key, value in rule.items(): + if key not in _RULE_OPTIONS: + close = difflib.get_close_matches(str(key), _RULE_OPTIONS, n=1, cutoff=0.8) + if close: + raise ValueError( + f"{path}.{key}: unknown option; did you mean {close[0]!r}?" + ) + values.append((f"{path}.{key}", value)) + if block == "z_flag_translation": + try: + float(key) + except (TypeError, ValueError) as exc: + raise ValueError( + f"{path}.{key}: z_flag direct mapping keys must be numeric" + ) from exc + for value_path, value in values: + if value is None: + continue + normalized = str(value).strip().lower() if kind == "str" else value + if kind == "float": + try: + normalized = float(value) + except (TypeError, ValueError) as exc: + raise ValueError(f"{value_path} must be numeric or null") from exc + if normalized not in allowed: + raise ValueError( + f"{value_path}={value!r} is invalid; allowed values are " + f"{sorted(allowed)} or null" + ) + # ----------------------- # Local helper (duplicated to avoid circular dep) # ----------------------- @@ -223,6 +404,7 @@ def _homogenize( Returns: Tuple: (df, used_type_fastpath, tiebreaking_priority, instrument_type_priority, translation_rules_uc) """ + validate_translation_config(translation_config) tiebreaking_priority = translation_config.get("tiebreaking_priority", []) instrument_type_priority = translation_config.get("instrument_type_priority", {}) translation_rules_uc = {k.upper(): v for k, v in translation_config.get("translation_rules", {}).items()} @@ -230,6 +412,52 @@ def _homogenize( z_flag_is_priority = "z_flag_homogenized" in tiebreaking_priority needs_z_flag = z_flag_is_priority or require_z_flag_homogenized + def _fast_path_policy(key: str) -> str: + if "survey" not in df.columns: + return "auto" + surveys = ( + df["survey"].dropna().astype(str).str.upper().unique().compute().tolist() + ) + policies = { + str( + translation_rules_uc.get(survey, {}) + .get(f"{key}_translation", {}) + .get("fast_path", "auto") + ) + for survey in surveys + } + if "disabled" in policies and "required" in policies: + raise ValueError( + f"[{product_name}] Conflicting {key} fast_path policies for surveys " + f"{sorted(surveys)}: cannot mix 'disabled' and 'required' in one catalog" + ) + if "disabled" in policies: + return "disabled" + if "required" in policies: + return "required" + return "auto" + + def _validate_result_domain( + column: str, allowed: set, *, normalize_string: bool = False + ) -> None: + values = df[column] + if normalize_string: + values = values.map_partitions( + _normalize_string_series_to_na, + meta=pd.Series(pd.array([], dtype=DTYPE_STR)), + ).str.lower() + else: + values = dd.to_numeric(values, errors="coerce") + invalid = (~dd.isna(values)) & ~values.isin(list(allowed)) + invalid_count, non_null_count = dask.compute(invalid.sum(), values.count()) + validated_non_null_counts[column] = int(non_null_count) + if int(invalid_count): + examples = df[column].loc[invalid].head(5, compute=True).tolist() + raise ValueError( + f"[{product_name}] Translation produced invalid values in '{column}'. " + f"Allowed set is {sorted(allowed)} (NaN allowed). Examples: {examples}" + ) + # ----------------------- # Vectorized translator # ----------------------- @@ -338,6 +566,9 @@ def visit_Compare(self, node): source_col = str(rule.get("source", key)) optional_source = bool(rule.get("optional_source", False)) + allow_condition_overlap = bool( + rule.get("allow_condition_overlap", False) + ) direct = { k: v for k, v in rule.items() @@ -346,43 +577,45 @@ def visit_Compare(self, node): "default", "source", "optional_source", + "allow_condition_overlap", + "fast_path", } } if direct: if source_col not in s.columns: - if optional_source: - continue - raise ValueError( - f"Missing source column '{source_col}' for survey " - f"'{sname}' and translation '{out_col}'." - ) - col = s.loc[mask_s, source_col] - is_num = pd.api.types.is_numeric_dtype(col) - if key == "z_flag": - is_num = True - if is_num: - col_num = pd.to_numeric(col, errors="coerce") - num_map = {} - for rk, rv in direct.items(): - try: - num_map[float(rk)] = rv - except Exception: - pass - mapped = col_num.map(num_map) - else: - col_str = col.astype(str).str.strip().str.lower() - str_map = {str(k).strip().lower(): v for k, v in direct.items()} - mapped = col_str.map(str_map) - - if out_kind == "float": - out.loc[mask_s] = mapped.fillna(out.loc[mask_s]).astype(DTYPE_FLOAT) + if not optional_source: + raise ValueError( + f"Missing source column '{source_col}' for survey " + f"'{sname}' and translation '{out_col}'." + ) else: - mapped = mapped.astype("object") - mapped_str = pd.Series(pd.array(mapped.where(mapped.notna(), None), dtype=DTYPE_STR), index=mapped.index) - take = mapped_str.notna() - out.loc[take.index] = out.loc[take.index].where(~take, mapped_str) - - for cond in (rule.get("conditions") or []): + col = s.loc[mask_s, source_col] + is_num = pd.api.types.is_numeric_dtype(col) + if key == "z_flag": + is_num = True + if is_num: + col_num = pd.to_numeric(col, errors="coerce") + num_map = {float(rk): rv for rk, rv in direct.items()} + mapped = col_num.map(num_map) + else: + col_str = col.astype(str).str.strip().str.lower() + str_map = {str(k).strip().lower(): v for k, v in direct.items()} + mapped = col_str.map(str_map) + + if out_kind == "float": + out.loc[mask_s] = mapped.fillna(out.loc[mask_s]).astype(DTYPE_FLOAT) + else: + mapped = mapped.astype("object") + mapped_str = pd.Series(pd.array(mapped.where(mapped.notna(), None), dtype=DTYPE_STR), index=mapped.index) + take = mapped_str.notna() + out.loc[take.index] = out.loc[take.index].where(~take, mapped_str) + + condition_values = pd.Series( + pd.array([pd.NA] * len(s), dtype="object"), index=s.index + ) + for condition_index, cond in enumerate( + rule.get("conditions") or [] + ): expr = cond.get("expr") if not expr: continue @@ -423,6 +656,26 @@ def visit_Compare(self, node): mlocal = mlocal & mask_s_aligned val = cond.get("value", default_val) + normalized_val = pd.NA if pd.isna(val) else val + conflicting = ( + mlocal + & condition_values.notna() + & condition_values.ne(normalized_val).fillna(False) + ) + conflict_count = int(conflicting.sum()) + if conflict_count and not allow_condition_overlap: + logger.warning( + "[%s] %s.%s condition %d overwrites a different " + "condition value for %d row(s); later conditions " + "take precedence. Set allow_condition_overlap: true " + "when this is intentional.", + product_name, + sname, + f"{key}_translation", + condition_index + 1, + conflict_count, + ) + condition_values.loc[mlocal] = normalized_val if out_kind == "float": out.loc[mlocal] = pd.to_numeric(val, errors="coerce") else: @@ -478,8 +731,20 @@ def quality_like_to_flag(x): if needs_z_flag: if "z_flag_homogenized" not in df.columns: - if can_use_zflag_as_quality(): - logger.info(f"{product_name} Using 'z_flag' fast path for z_flag_homogenized.") + z_fast_path = _fast_path_policy("z_flag") + z_fast_path_compatible = can_use_zflag_as_quality() + if z_fast_path == "required" and not z_fast_path_compatible: + raise ValueError( + f"[{product_name}] z_flag fast_path is 'required', but z_flag " + "is not a non-empty probability-like column in [0, 1]" + ) + if z_fast_path != "disabled" and z_fast_path_compatible: + logger.info( + "%s Using 'z_flag' fast path for z_flag_homogenized " + "(policy=%s); YAML z_flag rules are bypassed.", + product_name, + z_fast_path, + ) df["z_flag_homogenized"] = df["z_flag"].map_partitions( lambda s: s.apply(quality_like_to_flag).astype(DTYPE_FLOAT), meta=pd.Series(pd.array([], dtype=DTYPE_FLOAT)), @@ -497,6 +762,9 @@ def quality_like_to_flag(x): df = _translate_column_vectorized(df, key="z_flag", out_col="z_flag_homogenized", out_kind="float") + _validate_result_domain( + "z_flag_homogenized", {0.0, 1.0, 2.0, 3.0, 4.0, 6.0} + ) else: # User-provided 'z_flag_homogenized' is present. Validate allowed domain {0,1,2,3,4} (NaN allowed). logger.info(f"{product_name} 'z_flag_homogenized' already exists; validating user-provided values.") @@ -504,7 +772,10 @@ def quality_like_to_flag(x): vals = dd.to_numeric(df["z_flag_homogenized"], errors="coerce") # NaN is allowed; only non-NaN values outside the allowed set are invalid - invalid_mask = (~dd.isna(vals)) & ~vals.isin(list(allowed)) + invalid_mask = ( + ((~dd.isna(df["z_flag_homogenized"])) & dd.isna(vals)) + | ((~dd.isna(vals)) & ~vals.isin(list(allowed))) + ) invalid_count, non_null_count = dask.compute( invalid_mask.sum(), vals.count() ) @@ -543,8 +814,20 @@ def can_use_type_for_instrument() -> bool: if "instrument_type_homogenized" in tiebreaking_priority: if "instrument_type_homogenized" not in df.columns: - if can_use_type_for_instrument(): - logger.info(f"{product_name} Using 'type' fast path for instrument_type_homogenized.") + instrument_fast_path = _fast_path_policy("instrument_type") + instrument_fast_path_compatible = can_use_type_for_instrument() + if instrument_fast_path == "required" and not instrument_fast_path_compatible: + raise ValueError( + f"[{product_name}] instrument_type fast_path is 'required', " + "but type is not a non-empty column containing only s/g/p" + ) + if instrument_fast_path != "disabled" and instrument_fast_path_compatible: + logger.info( + "%s Using 'type' fast path for instrument_type_homogenized " + "(policy=%s); YAML instrument rules are bypassed.", + product_name, + instrument_fast_path, + ) df["instrument_type_homogenized"] = df["type"].map_partitions( _normalize_string_series_to_na, meta=pd.Series(pd.array([], dtype=DTYPE_STR)), @@ -567,6 +850,11 @@ def can_use_type_for_instrument() -> bool: _normalize_string_series_to_na, meta=pd.Series(pd.array([], dtype=DTYPE_STR)), ).str.lower() + _validate_result_domain( + "instrument_type_homogenized", + {"s", "g", "p"}, + normalize_string=True, + ) else: # User-provided 'instrument_type_homogenized' is present. Validate allowed domain {"s","p","g"}. logger.info(f"{product_name} 'instrument_type_homogenized' already exists; validating user-provided values.") diff --git a/scripts/crd-run.py b/scripts/crd-run.py index 7a58c45..adf1832 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -52,7 +52,11 @@ from executor import get_executor from product_handle import save_dataframe from resource_usage import ResourceUsageMonitor -from specz import prepare_catalog, validate_combine_configuration +from specz import ( + prepare_catalog, + validate_combine_configuration, + validate_translation_config, +) from utils import ( configure_exception_hook, configure_warning_handler, @@ -725,6 +729,7 @@ def main( try: translation_config = load_yml(path_to_translation_file) + validate_translation_config(translation_config) except Exception as e: log_init.error("Failed to parse flags_translation_file: %s", e, exc_info=True) raise diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index bb1c8c8..bdb1735 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -20,11 +20,83 @@ _requires_z_flag_homogenization, validate_combine_configuration, ) -from specz_homogenization import _homogenize # noqa: E402 +from specz_homogenization import ( + _homogenize, # noqa: E402 + validate_translation_config, # noqa: E402 +) LOGGER = logging.getLogger("test.homogenization") +def test_translation_schema_rejects_unsafe_expression_with_exact_path(): + config = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "conditions": [ + {"expr": "__import__('os').system('id')", "value": "star"} + ] + } + } + } + } + + with pytest.raises( + ValueError, + match=r"translation_rules\.DEMO\.object_type_translation\.conditions\[0\]\.expr", + ): + validate_translation_config(config) + + +def test_translation_schema_rejects_private_attributes_and_typos(): + private = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "conditions": [{"expr": "value.__class__", "value": "star"}] + } + } + } + } + with pytest.raises(ValueError, match="private attribute"): + validate_translation_config(private) + + typo = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "allow_condition_overlaps": True, + "default": None, + } + } + } + } + with pytest.raises(ValueError, match="did you mean 'allow_condition_overlap'"): + validate_translation_config(typo) + + +def test_translation_schema_validates_output_domains_and_condition_shape(): + invalid_value = { + "translation_rules": { + "DEMO": {"z_flag_translation": {"default": 9}} + } + } + with pytest.raises(ValueError, match=r"z_flag_translation\.default=9"): + validate_translation_config(invalid_value) + + invalid_condition = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "conditions": [{"expr": "value == 1", "vale": "star"}] + } + } + } + } + with pytest.raises(ValueError, match=r"conditions\[0\].*unknown option"): + validate_translation_config(invalid_condition) + + def test_yaml_flag_translation_applies_direct_default_and_condition(): frame = dd.from_pandas( pd.DataFrame( @@ -55,6 +127,191 @@ def test_yaml_flag_translation_applies_direct_default_and_condition(): assert result.compute()["z_flag_homogenized"].astype(float).tolist() == [2, 0, 4] +def test_z_flag_fast_path_policy_can_be_disabled_or_required(): + frame = dd.from_pandas( + pd.DataFrame({"survey": ["demo", "demo"], "z_flag": [0.2, 0.95]}), + npartitions=1, + sort=False, + ) + disabled = { + "tiebreaking_priority": ["z_flag_homogenized"], + "translation_rules": { + "DEMO": { + "z_flag_translation": {"fast_path": "disabled", "default": 3} + } + }, + } + result, *_ = _homogenize( + frame, disabled, "demo", LOGGER, type_cast_ok=False + ) + assert result.compute()["z_flag_homogenized"].tolist() == [3.0, 3.0] + + incompatible = dd.from_pandas( + pd.DataFrame({"survey": ["demo"], "z_flag": [4]}), + npartitions=1, + sort=False, + ) + required = { + "tiebreaking_priority": ["z_flag_homogenized"], + "translation_rules": { + "DEMO": { + "z_flag_translation": {"fast_path": "required", "default": 3} + } + }, + } + with pytest.raises(ValueError, match="fast_path is 'required'"): + _homogenize(incompatible, required, "demo", LOGGER, type_cast_ok=False) + + +def test_instrument_fast_path_can_be_disabled(): + frame = dd.from_pandas( + pd.DataFrame( + {"survey": ["demo", "demo"], "instrument_type": [pd.NA] * 2, "type": ["s", "g"]} + ), + npartitions=1, + sort=False, + ) + config = { + "tiebreaking_priority": ["instrument_type_homogenized"], + "translation_rules": { + "DEMO": { + "instrument_type_translation": { + "fast_path": "disabled", + "default": "p", + } + } + }, + } + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=True) + assert result.compute()["instrument_type_homogenized"].tolist() == ["p", "p"] + + +def test_optional_source_absence_keeps_condition_fallback(): + frame = dd.from_pandas( + pd.DataFrame( + {"survey": ["demo", "demo"], "object_type": [pd.NA] * 2, "fallback": [1, 0]} + ), + npartitions=1, + sort=False, + ) + config = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "source": "missing_optional_column", + "optional_source": True, + "STAR": "star", + "conditions": [{"expr": "fallback == 1", "value": "galaxy"}], + "default": None, + } + } + } + } + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ + "galaxy", + "missing", + ] + + +def test_later_condition_wins_and_conflicting_overlap_warns(caplog): + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["demo"], + "object_type": [pd.NA], + "score": [2], + } + ), + npartitions=1, + sort=False, + ) + config = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "conditions": [ + {"expr": "score > 0", "value": "star"}, + {"expr": "score > 1", "value": "galaxy"}, + ], + "default": None, + } + } + } + } + + with caplog.at_level(logging.WARNING, logger=LOGGER.name): + result, *_ = _homogenize( + frame, config, "demo", LOGGER, type_cast_ok=False + ) + + assert result.compute()["object_type_homogenized"].tolist() == ["galaxy"] + assert "later conditions take precedence" in caplog.text + + +def test_declared_condition_overlap_does_not_warn(caplog): + frame = dd.from_pandas( + pd.DataFrame( + {"survey": ["demo"], "object_type": [pd.NA], "score": [2]} + ), + npartitions=1, + sort=False, + ) + config = { + "translation_rules": { + "DEMO": { + "object_type_translation": { + "allow_condition_overlap": True, + "conditions": [ + {"expr": "score > 0", "value": "star"}, + {"expr": "score > 1", "value": "galaxy"}, + ], + "default": None, + } + } + } + } + + with caplog.at_level(logging.WARNING, logger=LOGGER.name): + result, *_ = _homogenize( + frame, config, "demo", LOGGER, type_cast_ok=False + ) + + assert result.compute()["object_type_homogenized"].tolist() == ["galaxy"] + assert "later conditions take precedence" not in caplog.text + + +def test_cosmos_web_flag_translation_compares_normalized_string_type(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = ["z_flag_homogenized"] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": ["COSMOS_Web"] * 5, + "object_type": [pd.NA] * 5, + "type": ["1", "0", "0", "0", "0"], + "z_flag": [0, 1, 0, 0, 0], + "zpdf_med": [1.0, 1.0, 1.01, 1.0, 1.0], + "zchi2": [1.0, 1.0, 1.0, 1.0, 1.0], + "nbfilt": [40, 40, 40, 20, 30], + } + ), + npartitions=1, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["z_flag_homogenized"].tolist() == [ + 6.0, + 0.0, + 1.0, + 2.0, + 3.0, + ] + + def test_object_type_is_always_present_and_may_be_entirely_null(): frame = dd.from_pandas( pd.DataFrame({"survey": ["unknown", "unknown"]}), From 42982abb7ab2cae18016b06d02cd750d3e0cb872 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 20:42:33 -0300 Subject: [PATCH 06/16] Correcting validate_translation_config import error --- scripts/crd-run.py | 7 ++----- tests/test_entrypoint_imports.py | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+), 5 deletions(-) create mode 100644 tests/test_entrypoint_imports.py diff --git a/scripts/crd-run.py b/scripts/crd-run.py index adf1832..de3c58e 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -52,11 +52,8 @@ from executor import get_executor from product_handle import save_dataframe from resource_usage import ResourceUsageMonitor -from specz import ( - prepare_catalog, - validate_combine_configuration, - validate_translation_config, -) +from specz import prepare_catalog, validate_combine_configuration +from specz_homogenization import validate_translation_config from utils import ( configure_exception_hook, configure_warning_handler, diff --git a/tests/test_entrypoint_imports.py b/tests/test_entrypoint_imports.py new file mode 100644 index 0000000..95ad749 --- /dev/null +++ b/tests/test_entrypoint_imports.py @@ -0,0 +1,18 @@ +"""Smoke tests for executable entrypoint imports.""" + +import importlib.util +import sys +from pathlib import Path + + +def test_crd_run_module_imports_successfully(): + root = Path(__file__).resolve().parents[1] + packages = str(root / "packages") + if packages not in sys.path: + sys.path.insert(0, packages) + + script = root / "scripts" / "crd-run.py" + spec = importlib.util.spec_from_file_location("crd_run_import_smoke", script) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) From 0933e0655379692afa5da2ed39d034ac54ba5325 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 21:37:28 -0300 Subject: [PATCH 07/16] fix: normalize runtime schemas across crossmatch rounds - normalize object_type_homogenized before LSDB concatenation - propagate schemas for extra and expression columns - preserve custom ranking and enabled footprint dtypes - validate runtime schema hints - add regression tests for Python and PyArrow strings --- packages/crossmatch_cross.py | 9 +- packages/specz.py | 40 +++++++++ packages/specz_homogenization.py | 13 ++- scripts/crd-run.py | 9 +- tests/test_crossmatch_dtype_normalization.py | 90 ++++++++++++++++++++ 5 files changed, 156 insertions(+), 5 deletions(-) create mode 100644 tests/test_crossmatch_dtype_normalization.py diff --git a/packages/crossmatch_cross.py b/packages/crossmatch_cross.py index 78018db..c1dcc8b 100644 --- a/packages/crossmatch_cross.py +++ b/packages/crossmatch_cross.py @@ -511,9 +511,11 @@ def _get_expr_schema_hints(translation_config: dict | None) -> dict: dict: Expr column schema hints or empty dict. """ cfg = translation_config or {} - if cfg.get("save_expr_columns") is False: - return {} - return cfg.get("expr_column_schema", {}) or {} + hints = {} + if cfg.get("save_expr_columns") is not False: + hints.update(cfg.get("expr_column_schema", {}) or {}) + hints.update(cfg.get("runtime_schema_hints", {}) or {}) + return hints def _ensure_compared_to(cat): @@ -546,6 +548,7 @@ def _ensure_compared_to(cat): "tie_result": DTYPE_INT8, "z_flag_homogenized": DTYPE_FLOAT, "instrument_type_homogenized": DTYPE_STR, + "object_type_homogenized": DTYPE_STR, "compared_to": DTYPE_STR, } diff --git a/packages/specz.py b/packages/specz.py index f11ecca..289af22 100644 --- a/packages/specz.py +++ b/packages/specz.py @@ -1627,6 +1627,7 @@ def _normalize_extra_columns_config(value: Any) -> dict[str, dict[str, str]]: "group_id", "z_flag_homogenized", "instrument_type_homogenized", + "object_type_homogenized", "is_in_DP1_fields", "is_in_rubin_footprint", } @@ -1638,6 +1639,45 @@ def _normalize_extra_columns_config(value: Any) -> dict[str, dict[str, str]]: return normalized +def build_runtime_schema_hints( + param_config: dict, translation_config: dict +) -> dict[str, str]: + """Build schema hints that must survive every crossmatch round. + + Args: + param_config: Pipeline ``param`` configuration. + translation_config: Validated flags translation configuration. + + Returns: + Mapping of output column names to ``str``, ``float``, ``int`` or ``bool``. + """ + hints: dict[str, str] = {} + if bool(translation_config.get("save_expr_columns", False)): + hints.update( + _normalize_schema_hints(translation_config.get("expr_column_schema")) + ) + + extra_columns = _normalize_extra_columns_config(param_config.get("extra_columns")) + hints.update({output: spec["type"] for output, spec in extra_columns.items()}) + + if _as_bool_config(param_config.get("insert_DP1_footprint_flag"), default=False): + hints["is_in_DP1_fields"] = "int" + if _as_bool_config( + param_config.get("insert_rubin_footprint_flag"), default=False + ): + hints["is_in_rubin_footprint"] = "int" + + standard_priorities = { + "z_flag_homogenized", + "instrument_type_homogenized", + } + for column in translation_config.get("tiebreaking_priority", []) or []: + name = str(column).strip() + if name and name not in standard_priorities: + hints[name] = "float" + return hints + + def _copy_extra_columns_from_sources( df: dd.DataFrame, columns: dict[str, dict[str, str]], diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index c160f79..36cb036 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -88,7 +88,7 @@ "crossmatch_geometry_diagnostics_enabled", "representative_radius_diagnostics_enabled", "dedup_edge_diagnostics_enabled", "instrument_type_priority", "save_expr_columns", "expr_column_schema", - "translation_rules", + "runtime_schema_hints", "translation_rules", } _SAFE_FUNCTIONS = {"len", "int", "float", "str"} _SAFE_NUMPY_FUNCTIONS = {"isfinite", "abs", "trunc"} @@ -166,6 +166,17 @@ def validate_translation_config(config: dict) -> None: if unknown_top: raise ValueError(f"flags translation root: unknown option(s): {sorted(unknown_top)}") rules = config.get("translation_rules", {}) + runtime_hints = config.get("runtime_schema_hints", {}) + if not isinstance(runtime_hints, dict): + raise TypeError("runtime_schema_hints must be a mapping") + for column, kind in runtime_hints.items(): + if not isinstance(column, str) or not column.strip(): + raise ValueError("runtime_schema_hints column names must be non-empty strings") + if kind not in {"str", "float", "int", "bool"}: + raise ValueError( + f"runtime_schema_hints.{column}={kind!r} is invalid; " + "expected str, float, int or bool" + ) if not isinstance(rules, dict): raise TypeError("translation_rules must be a mapping") for survey, ruleset in rules.items(): diff --git a/scripts/crd-run.py b/scripts/crd-run.py index de3c58e..f7e2717 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -52,7 +52,11 @@ from executor import get_executor from product_handle import save_dataframe from resource_usage import ResourceUsageMonitor -from specz import prepare_catalog, validate_combine_configuration +from specz import ( + build_runtime_schema_hints, + prepare_catalog, + validate_combine_configuration, +) from specz_homogenization import validate_translation_config from utils import ( configure_exception_hook, @@ -782,6 +786,9 @@ def main( log_init, ) translation_config["tiebreaking_priority"] = validated_priorities + translation_config["runtime_schema_hints"] = build_runtime_schema_hints( + param_config, translation_config + ) completed = read_completed_steps(os.path.join(temp_dir, "process_resume.log")) # --- Dask cluster/client --- diff --git a/tests/test_crossmatch_dtype_normalization.py b/tests/test_crossmatch_dtype_normalization.py new file mode 100644 index 0000000..7186d81 --- /dev/null +++ b/tests/test_crossmatch_dtype_normalization.py @@ -0,0 +1,90 @@ +"""Regression tests for strict crossmatch schema normalization.""" + +import sys +import types +from pathlib import Path + +import pandas as pd + +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "packages")) +if "tables_io" not in sys.modules: + tables_io = types.ModuleType("tables_io") + tables_io.types = types.SimpleNamespace(PD_DATAFRAME="PD_DATAFRAME") + sys.modules["tables_io"] = tables_io + +from crossmatch_cross import ( # noqa: E402 + _EXPECTED_TYPES, + _cast_partition_expected, + _get_expr_schema_hints, +) +from specz import DTYPE_STR, build_runtime_schema_hints # noqa: E402 + + +def test_object_type_string_backends_normalize_to_same_dtype(): + python_string = pd.DataFrame( + {"object_type_homogenized": pd.Series(["star"], dtype="string")} + ) + pyarrow_string = pd.DataFrame( + {"object_type_homogenized": pd.Series(["agn"], dtype="string[pyarrow]")} + ) + + left = _cast_partition_expected(python_string, _EXPECTED_TYPES, None) + right = _cast_partition_expected(pyarrow_string, _EXPECTED_TYPES, None) + + assert left["object_type_homogenized"].dtype == DTYPE_STR + assert right["object_type_homogenized"].dtype == DTYPE_STR + assert left["object_type_homogenized"].dtype == right["object_type_homogenized"].dtype + + +def test_runtime_schema_combines_expr_extra_priority_and_enabled_flags(): + param_config = { + "extra_columns": { + "label": "str", + "external_score": {"source": "RAW_SCORE", "type": "float"}, + }, + "insert_DP1_footprint_flag": True, + "insert_rubin_footprint_flag": False, + } + translation_config = { + "save_expr_columns": True, + "expr_column_schema": {"expr_value": "int", "label": "float"}, + "tiebreaking_priority": ["z_flag_homogenized", "custom_rank"], + } + + hints = build_runtime_schema_hints(param_config, translation_config) + + assert hints == { + "expr_value": "int", + "label": "str", + "external_score": "float", + "is_in_DP1_fields": "int", + "custom_rank": "float", + } + + +def test_crossmatch_runtime_hints_override_expr_hints_and_normalize_extra_columns(): + config = { + "save_expr_columns": True, + "expr_column_schema": {"label": "float", "expr_value": "int"}, + "runtime_schema_hints": {"label": "str", "extra_text": "str"}, + } + hints = _get_expr_schema_hints(config) + assert hints == {"label": "str", "expr_value": "int", "extra_text": "str"} + + left = pd.DataFrame( + { + "label": pd.Series(["left"], dtype="string"), + "extra_text": pd.Series(["a"], dtype="string"), + } + ) + right = pd.DataFrame( + { + "label": pd.Series(["right"], dtype="string[pyarrow]"), + "extra_text": pd.Series(["b"], dtype="string[pyarrow]"), + } + ) + left_fixed = _cast_partition_expected(left, _EXPECTED_TYPES, hints) + right_fixed = _cast_partition_expected(right, _EXPECTED_TYPES, hints) + + assert left_fixed["label"].dtype == right_fixed["label"].dtype == DTYPE_STR + assert left_fixed["extra_text"].dtype == right_fixed["extra_text"].dtype == DTYPE_STR From ddeda3ae6093bbd57a6569638143f3296bb3b1e6 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 22:27:57 -0300 Subject: [PATCH 08/16] feat: add galactic object type and tighten stellar classification rules --- flags_translation.yaml | 58 +++++++++--------- packages/specz_homogenization.py | 7 ++- tests/test_homogenization.py | 100 ++++++++++++++++++++++++------- 3 files changed, 112 insertions(+), 53 deletions(-) diff --git a/flags_translation.yaml b/flags_translation.yaml index ae25c5d..1070a82 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -50,14 +50,16 @@ instrument_type_priority: # - 2: 70% < Confidence < 90% # - 3: 90% < Confidence < 99% # - 4: Confidence > 99% -# - 6: Object identified as non-extragalactic (e.g., stars) +# - 6: Object explicitly identified as a star # Each block corresponds to a survey name and includes: # - z_flag_translation: how to map original quality flags or based on conditions into a standardized quality scale (0-6) # - instrument_type_translation: how to map original types into a standard label (s, g, p) -# - object_type_translation: how to map classifications into star, qso, agn, -# or galaxy. `qso` requires an explicit QSO/quasar classification; `agn` is -# used for generic AGN and BLAGN classifications without an explicit QSO label. +# - object_type_translation: how to map classifications into star, galactic, +# qso, agn, or galaxy. `galactic` means a confirmed Galactic/non-extragalactic +# source without a secure stellar subtype. `qso` requires an explicit +# QSO/quasar classification; `agn` is used for generic AGN and BLAGN +# classifications without an explicit QSO label. # Its default is null; catalogs without object classifications are valid. # A rule may declare `source: ` and `optional_source: true` to # discover a non-standard input column without a frontend columns mapping. @@ -99,14 +101,13 @@ translation_rules: object_type_translation: conditions: - expr: "z_flag == 6" - value: "star" + value: "galactic" default: null z_flag_translation: 1: 0 2: 1 3: 3 4: 4 - 6: 6 instrument_type_translation: default: "s" @@ -117,7 +118,11 @@ translation_rules: # AGN class, but the information is insufficient for the QSO subtype. - expr: "str(TYPE)[:2] == '-9'" value: "agn" - default: "galaxy" + # Use only documented ZCAT galaxy morphology codes. Type 98 means that + # the source was never visually examined and is intentionally omitted. + - expr: "TYPE.str.contains(r'^(?:-[1-7]|20|19|16|15|11|10|[0-9])(?:[^0-9]|$)', regex=True, na=False)" + value: "galaxy" + default: null z_flag_translation: conditions: - expr: "z_err == 0" @@ -161,14 +166,13 @@ translation_rules: object_type_translation: conditions: - expr: "z_flag == 6" - value: "star" + value: "galactic" default: null z_flag_translation: 1: 0 2: 1 3: 3 4: 4 - 6: 6 instrument_type_translation: default: "s" @@ -414,7 +418,9 @@ translation_rules: 2: 2 3: 3 4: 4 - 6: 6 + conditions: + - expr: "Class == 4" + value: 6 instrument_type_translation: conditions: - expr: "zfinal_type == 's'" @@ -439,7 +445,9 @@ translation_rules: 2: 2 3: 3 4: 4 - 6: 6 + conditions: + - expr: "tSp == 7" + value: 6 instrument_type_translation: conditions: - expr: "zbest_type == 's'" @@ -584,16 +592,9 @@ translation_rules: OZDES: object_type_translation: - allow_condition_overlap: true conditions: - - expr: "Object_types.str.contains('Galaxy|LRG|ELG|SN_host|SN_free_host|RedMaGiC', case=False, na=False)" - value: "galaxy" - - expr: "Object_types.str.contains('AGN_reverberation|AGN_monitoring', case=False, na=False)" - value: "agn" - - expr: "Object_types.str.contains('QSO', case=False, na=False)" - value: "qso" - - expr: "Object_types.str.contains('BBstar|FStar|RNDstars|BrightStar|WhiteDwarf', case=False, na=False)" - value: "star" + # Object_types contains targeting categories, not confirmed object + # classifications. qop=6 is explicitly a securely classified star. - expr: "z_flag == 6" value: "star" default: null @@ -680,9 +681,9 @@ translation_rules: object_type_translation: allow_condition_overlap: true conditions: - - expr: "mst < 0" + - expr: "mst == -1" value: "star" - - expr: "mst > 0" + - expr: "mst == 1" value: "galaxy" # Explicit QSO templates take precedence over the morphology scale. - expr: "(13 <= J1 <= 30) or (13 <= J2 <= 15)" @@ -694,7 +695,9 @@ translation_rules: 2: 2 3: 3 4: 4 - 6: 6 + conditions: + - expr: "mst == -1 and not ((13 <= J1 <= 30) or (13 <= J2 <= 15))" + value: 6 instrument_type_translation: conditions: - expr: "zfinal_type == 's'" @@ -746,19 +749,14 @@ translation_rules: VIPERS_PDR2: object_type_translation: - allow_condition_overlap: true conditions: - - expr: "classFlag == -1" - value: "star" - # Spectroscopic BLAGN classification takes precedence over the - # photometric stellar-like classification. + # classFlag=-1 is only a photometric "stellar-like" classification and + # is intentionally left null. Only spectroscopic BLAGN is retained. - expr: "np.trunc(float(z_flag)) == 13 or np.trunc(float(z_flag)) == 14 or np.trunc(float(z_flag)) == 19 or np.trunc(float(z_flag)) == 213 or np.trunc(float(z_flag)) == 214 or np.trunc(float(z_flag)) == 219" value: "agn" default: null z_flag_translation: conditions: - - expr: "classFlag == -1" - value: 6 - expr: "classFlag != -1 and np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 1" value: 1 - expr: "classFlag != -1 and np.trunc(float(z_flag)) - 10*np.trunc(np.trunc(float(z_flag))/10) == 2" diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index 36cb036..c71cbf0 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -71,7 +71,10 @@ _TRANSLATION_KEYS = { "z_flag_translation": ("float", {0.0, 1.0, 2.0, 3.0, 4.0, 6.0}), "instrument_type_translation": ("str", {"s", "g", "p"}), - "object_type_translation": ("str", {"star", "qso", "agn", "galaxy"}), + "object_type_translation": ( + "str", + {"star", "galactic", "qso", "agn", "galaxy"}, + ), } _RULE_OPTIONS = { "conditions", "default", "source", "optional_source", @@ -908,7 +911,7 @@ def can_use_type_for_instrument() -> bool: _normalize_string_series_to_na, meta=pd.Series(pd.array([], dtype=DTYPE_STR)), ).str.lower() - allowed_object_types = {"star", "qso", "agn", "galaxy"} + allowed_object_types = {"star", "galactic", "qso", "agn", "galaxy"} invalid_object_mask = (~dd.isna(object_types)) & ~object_types.isin( list(allowed_object_types) ) diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index bdb1735..631ffd9 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -362,12 +362,16 @@ def test_object_type_translation_uses_canonical_renamed_z_flag(): def test_object_type_rejects_values_outside_domain(): valid = dd.from_pandas( - pd.DataFrame({"object_type_homogenized": ["STAR", "agn"]}), + pd.DataFrame({"object_type_homogenized": ["STAR", "agn", "GALACTIC"]}), npartitions=1, sort=False, ) result, *_ = _homogenize(valid, {}, "demo", LOGGER, type_cast_ok=False) - assert result.compute()["object_type_homogenized"].tolist() == ["star", "agn"] + assert result.compute()["object_type_homogenized"].tolist() == [ + "star", + "agn", + "galactic", + ] invalid = dd.from_pandas( pd.DataFrame({"object_type_homogenized": ["unknown"]}), @@ -421,7 +425,7 @@ def test_catalog_object_type_rules_use_renamed_flags_and_conservative_defaults() "missing", "agn", "agn", - "galaxy", + "missing", "star", ] @@ -479,8 +483,8 @@ def test_swire_j2_qso_range_stops_at_15(): "missing", "star", "galaxy", - "star", - "galaxy", + "missing", + "missing", "qso", ] @@ -492,10 +496,17 @@ def test_2df_6df_and_2mrs_use_only_documented_object_classes(): frame = dd.from_pandas( pd.DataFrame( { - "survey": ["2DFLENS", "2DFLENS", "6DFGS", "2MRS", "2MRS"], - "object_type": [pd.NA] * 5, - "z_flag": [6, 4, 6, 4, 4], - "TYPE": [pd.NA, pd.NA, pd.NA, "-5A", "-9"], + "survey": [ + "2DFLENS", + "2DFLENS", + "6DFGS", + "2MRS", + "2MRS", + "2MRS", + ], + "object_type": [pd.NA] * 6, + "z_flag": [6, 4, 6, 4, 4, 4], + "TYPE": [pd.NA, pd.NA, pd.NA, "-5A", "-9", "98"], } ), npartitions=1, @@ -505,11 +516,62 @@ def test_2df_6df_and_2mrs_use_only_documented_object_classes(): result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ - "star", + "galactic", "missing", - "star", + "galactic", "galaxy", "agn", + "missing", + ] + + +def test_z_flag_6_requires_an_explicit_stellar_classification(): + config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + config["tiebreaking_priority"] = ["z_flag_homogenized"] + frame = dd.from_pandas( + pd.DataFrame( + { + "survey": [ + "2DFLENS", + "6DFGS", + "VIPERS_PDR2", + "ELAISS1OID", + "ELAISS1OID", + "ELAISFBMC", + "ELAISFBMC", + "SWIRE-REVISED", + "SWIRE-REVISED", + "SWIRE-REVISED", + "OZDES", + ], + "z_flag": [6, 6, 4.2, 6, 4, 6, 4, 6, 4, 4, 6], + "classFlag": [0, 0, -1, 0, 0, 0, 0, 0, 0, 0, 0], + "Class": [0, 0, 0, 2, 4, 0, 0, 0, 0, 0, 0], + "tSp": [0, 0, 0, 0, 0, 1, 7, 0, 0, 0, 0], + "mst": [0, 0, 0, 0, 0, 0, 0, 1, -1, -1, 0], + "J1": [1, 1, 1, 1, 1, 1, 1, 1, 1, 13, 1], + "J2": [1] * 11, + } + ), + npartitions=2, + sort=False, + ) + + result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) + + assert result.compute()["z_flag_homogenized"].fillna(-1).tolist() == [ + -1, + -1, + 0, + -1, + 6, + -1, + 6, + -1, + 6, + 4, + 6, ] @@ -630,8 +692,8 @@ def test_generic_agn_labels_are_not_promoted_to_qso(): assert result.compute()["object_type_homogenized"].tolist() == [ "agn", - "agn", - "qso", + pd.NA, + pd.NA, "agn", "agn", "agn", @@ -639,7 +701,7 @@ def test_generic_agn_labels_are_not_promoted_to_qso(): ] -def test_vipers_blagn_takes_precedence_over_photometric_star_like_flag(): +def test_vipers_keeps_photometric_star_like_null_but_retains_spectroscopic_blagn(): config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" config = yaml.safe_load(config_path.read_text(encoding="utf-8")) config["tiebreaking_priority"] = [] @@ -659,13 +721,13 @@ def test_vipers_blagn_takes_precedence_over_photometric_star_like_flag(): result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) assert result.compute()["object_type_homogenized"].fillna("missing").tolist() == [ - "star", + "missing", "agn", "agn", ] -def test_ozdes_explicit_stellar_targets_are_stellar(): +def test_ozdes_targeting_labels_are_not_treated_as_object_classifications(): config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" config = yaml.safe_load(config_path.read_text(encoding="utf-8")) config["tiebreaking_priority"] = [] @@ -684,11 +746,7 @@ def test_ozdes_explicit_stellar_targets_are_stellar(): result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) - assert result.compute()["object_type_homogenized"].tolist() == [ - "star", - "star", - "star", - ] + assert result.compute()["object_type_homogenized"].isna().all() def test_euclid_optional_source_is_safe_before_column_arrives(): From 9042bfeaef42e3ed7fa0806d087c538306f0587e Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Tue, 7 Jul 2026 23:34:10 -0300 Subject: [PATCH 09/16] feat: filter deduplication by object type and retire z-flag 6 - add configurable object-type graph inclusion - use internal 0.5 ranking fallback for stellar objects without quality - redefine tie_result=3 for graph-excluded rows - add global tie and label-merge diagnostics --- config.py | 22 +- config.template.yaml | 10 +- .../crossmatch-deduplication-design.md | 21 +- flags_translation.yaml | 39 ++- packages/deduplication.py | 227 +++++++++++++----- packages/specz.py | 20 +- packages/specz_homogenization.py | 46 ++-- scripts/crd-run.py | 161 +++++++++++-- tests/test_dedup_partition_safety.py | 18 +- tests/test_deduplication_core.py | 106 +++++++- tests/test_homogenization.py | 65 +++-- tests/test_tie_treatment.py | 6 +- 12 files changed, 574 insertions(+), 167 deletions(-) diff --git a/config.py b/config.py index 387e10a..4388e2c 100644 --- a/config.py +++ b/config.py @@ -101,12 +101,32 @@ class Columns(BaseModel): class Param(BaseModel): combine_type: str = "concatenate" extra_columns: dict[str, Any] = Field(default_factory=dict) - # Zero disables the cut; valid active cuts are 1, 2, 3, 4, 5, 6. + # Zero disables the cut; valid active cuts are 1, 2, 3, 4. z_flag_homogenized_value_to_cut: float = 3.0 + include_unclassified: bool = True + include_galaxy: bool = True + include_star: bool = False + include_agn: bool = True + include_qso: bool = True + include_galactic: bool = False flags_translation_file: str = str(Path(MAINDIR, "flags_translation.yaml")) insert_DP1_footprint_flag: bool = False insert_rubin_footprint_flag: bool = False + @model_validator(mode="after") + def validate_object_type_inclusion(self): + inclusion = ( + self.include_unclassified, + self.include_galaxy, + self.include_star, + self.include_agn, + self.include_qso, + self.include_galactic, + ) + if not any(inclusion): + raise ValueError("at least one include_* object-type option must be true") + return self + class Config(BaseModel): output_root_dir: str = "." diff --git a/config.template.yaml b/config.template.yaml index 572096b..276cff4 100644 --- a/config.template.yaml +++ b/config.template.yaml @@ -87,7 +87,15 @@ param: # ORIGINAL_ID: # source: id # type: str - z_flag_homogenized_value_to_cut: 0.0 # 0 disables the cut; valid active cuts are 1, 2, 3, 4, 5, 6. + z_flag_homogenized_value_to_cut: 0.0 # 0 disables the cut; valid active cuts are 1, 2, 3, 4. + # Object types allowed to participate in the deduplication graph. Disabled + # types remain in marked outputs as isolated rows with tie_result=3. + include_unclassified: true + include_galaxy: true + include_star: false + include_agn: true + include_qso: true + include_galactic: false insert_DP1_footprint_flag: false # Adds is_in_DP1_fields (1/0) insert_rubin_footprint_flag: false # Adds is_in_rubin_footprint (1/0) flags_translation_file: flags_translation.yaml # File with homogenization rules for z_flag and type diff --git a/docs/architecture/crossmatch-deduplication-design.md b/docs/architecture/crossmatch-deduplication-design.md index 1ed41c3..2ba5da0 100644 --- a/docs/architecture/crossmatch-deduplication-design.md +++ b/docs/architecture/crossmatch-deduplication-design.md @@ -280,7 +280,7 @@ Only columns required by the graph solver are retained, including: - `compared_to`; - redshift; - configured tie-breaking priorities; -- star classification; +- homogenized object type and graph-inclusion policy; - coordinates required by optional geometry diagnostics. The local partitions are converted to pandas because the graph algorithm is a @@ -303,6 +303,23 @@ CRD_ID, tie_result, group_id These labels are merged back into the complete distributed dataframe. The complete dataframe remains lazy for HATS, Parquet, and CSV output paths. +The default diagnostics count rows missing a newly computed label and classify +invalid tie groups by aggregate reason. Optional detailed diagnostics collect a +bounded sample of offending `group_id` and member `CRD_ID` values; this mode is +disabled by default because it requires an additional distributed scan. + +`tie_result` has the following stable meanings: + +- `0`: participating row that lost within its graph component; +- `1`: selected winner; +- `2`: unresolved hard-tie candidate; +- `3`: row deliberately excluded from the graph by object-type configuration. + +Rows with `tie_result=3` remain visible as isolated records in marked outputs. +`concatenate_and_remove_duplicates` filters them out because that mode retains +only `tie_result=1` (plus hard ties according to `tie_treatment_option`). They +neither create nor receive graph edges. + ## Scientific semantics preserved by this design These representation changes do not alter the intended scientific decisions: @@ -311,7 +328,7 @@ These representation changes do not alter the intended scientific decisions: radius; - HATS margins still provide cross-partition spatial context; - graph components are still formed from `compared_to` edges; -- stars remain outside ordinary winner selection; +- object types disabled by configuration remain isolated from the graph; - configured priority columns still determine winners and hard ties; - redshift disambiguation and missing-redshift policy remain unchanged; - canonical groups and tie-result invariants are applied after the same edge diff --git a/flags_translation.yaml b/flags_translation.yaml index 1070a82..a923a78 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -13,7 +13,7 @@ tiebreaking_priority: # contain at least one valid numeric value in every input catalog. # The columns will be evaluated in the order listed to resolve duplicates. # In deduplication modes, z_flag_homogenized is still created and preserved - # for star classification (value 6), even when omitted from this ranking. + # as a quality field even when omitted from this ranking. delta_z_threshold: 0.001 # Maximum allowed redshift difference (|z1 - z2|) to resolve hard ties # Applied only when initial tiebreaking via columns results in a tie (tie_result == 2) @@ -23,6 +23,11 @@ margin_threshold_arcsec: 5.0 # HATS margin used by independent partition-local margin_warning_fraction: 0.8 # Warn when a boundary component spans this fraction of the margin. validate_global_graph_edges: false # Distributed edge/group consistency check (costs two joins). validate_global_tie_invariants: true # Fail on invalid winner/hard-tie patterns after label merge. +tie_invariant_diagnostics_enabled: true # Cheap aggregate reason counts for invalid tie groups. +tie_invariant_diagnostics_detailed_enabled: false # Expensive sample of group IDs and member CRD_IDs. +tie_invariant_diagnostics_sample_size: 10 # Maximum invalid groups in the detailed sample. +tie_invariant_diagnostics_max_rows: 100 # Maximum member rows collected across sampled groups. +label_merge_diagnostics_enabled: true # Cheap count of rows without a newly computed dedup label. validate_crd_id_uniqueness: true # Global nunique validation for every prepared input catalog. repartition_prepared_catalogs: false # Extra full-data pass to resize prepared partitions. prepared_partition_size: "256MB" # Used only when repartition_prepared_catalogs is enabled. @@ -50,10 +55,11 @@ instrument_type_priority: # - 2: 70% < Confidence < 90% # - 3: 90% < Confidence < 99% # - 4: Confidence > 99% -# - 6: Object explicitly identified as a star +# - null: quality unavailable or not applicable; stellar/galactic rows without +# an independently documented quality remain null # Each block corresponds to a survey name and includes: -# - z_flag_translation: how to map original quality flags or based on conditions into a standardized quality scale (0-6) +# - z_flag_translation: how to map original quality flags or based on conditions into a standardized quality scale (0-4) # - instrument_type_translation: how to map original types into a standard label (s, g, p) # - object_type_translation: how to map classifications into star, galactic, # qso, agn, or galaxy. `galactic` means a confirmed Galactic/non-extragalactic @@ -144,7 +150,7 @@ translation_rules: z_flag_translation: conditions: - expr: "z_best_s == 0" - value: 6 + value: null - expr: "z_best_s == 1 and z_spec != -1" value: 3 - expr: "z_best_s == 2 and use_zgrism == 1 and flag1 == 0 and flag2 == 0" @@ -230,7 +236,7 @@ translation_rules: - expr: "lp_type == -9 or not np.isfinite(z)" value: 0 - expr: "lp_type == 1" - value: 6 + value: null - expr: "(lp_type != 1 and lp_type != -9) and np.isfinite(z)" value: 1 - expr: "(lp_type != 1 and lp_type != -9) and np.isfinite(z) and np.isfinite(lp_zPDF) and np.isfinite(lp_zMinChi2) and np.isfinite(lp_zPDF_l68) and np.isfinite(lp_zPDF_u68) and lp_zPDF_l68 <= z <= lp_zPDF_u68 and lp_zPDF_l68 <= lp_zPDF <= lp_zPDF_u68 and lp_zPDF_l68 <= lp_zMinChi2 <= lp_zPDF_u68" @@ -286,7 +292,7 @@ translation_rules: - expr: "type != '1' and z_flag == 0 and np.abs(zpdf_med - zchi2) <= 0.001 and nbfilt >= 30" value: 3 - expr: "type == '1'" - value: 6 + value: null default: 0 instrument_type_translation: default: "p" @@ -343,7 +349,7 @@ translation_rules: - expr: "SPECTYPE != 'STAR' and z_flag == 0 and ZCAT_PRIMARY == True and z_err < 0.001" value: 4 - expr: "SPECTYPE == 'STAR'" - value: 6 + value: null default: 0 instrument_type_translation: default: "s" @@ -369,7 +375,7 @@ translation_rules: - expr: "SPECTYPE != 'STAR' and z_flag == 0 and ZCAT_PRIMARY == True and z_err < 0.001" value: 4 - expr: "SPECTYPE == 'STAR'" - value: 6 + value: null default: 0 instrument_type_translation: default: "s" @@ -418,9 +424,6 @@ translation_rules: 2: 2 3: 3 4: 4 - conditions: - - expr: "Class == 4" - value: 6 instrument_type_translation: conditions: - expr: "zfinal_type == 's'" @@ -445,9 +448,6 @@ translation_rules: 2: 2 3: 3 4: 4 - conditions: - - expr: "tSp == 7" - value: 6 instrument_type_translation: conditions: - expr: "zbest_type == 's'" @@ -505,7 +505,6 @@ translation_rules: 2: 2 3: 3 4: 4 - -1: 6 instrument_type_translation: default: "s" @@ -603,7 +602,6 @@ translation_rules: 2: 1 3: 3 4: 4 - 6: 6 instrument_type_translation: default: "s" @@ -646,7 +644,7 @@ translation_rules: - expr: "CLASS != 'STAR' and z_flag == 0 and SPECPRIMARY == 1 and z_err < 0.001" value: 4 - expr: "CLASS == 'STAR'" - value: 6 + value: null default: 0 instrument_type_translation: default: "s" @@ -672,7 +670,7 @@ translation_rules: - expr: "CLASS != 'STAR' and z_flag == 0 and SPECPRIMARY == 1 and z_err < 0.001" value: 4 - expr: "CLASS == 'STAR'" - value: 6 + value: null default: 0 instrument_type_translation: default: "s" @@ -695,9 +693,6 @@ translation_rules: 2: 2 3: 3 4: 4 - conditions: - - expr: "mst == -1 and not ((13 <= J1 <= 30) or (13 <= J2 <= 15))" - value: 6 instrument_type_translation: conditions: - expr: "zfinal_type == 's'" @@ -830,7 +825,7 @@ translation_rules: z_flag_translation: conditions: - expr: "Class == 'STA'" - value: 6 + value: null - expr: "Class != 'STA' and z_flag == 3" value: 1 - expr: "Class != 'STA' and z_flag == 2" diff --git a/packages/deduplication.py b/packages/deduplication.py index 77cb952..8de7197 100644 --- a/packages/deduplication.py +++ b/packages/deduplication.py @@ -76,10 +76,71 @@ def _phase_logger() -> logging.LoggerAdapter: "run_dedup_with_lsdb_map_partitions", "count_global_edge_group_mismatches", "count_global_tie_invariant_violations", + "build_global_tie_invariant_diagnostics", "filter_dask_by_tie_treatment", "filter_pandas_by_tie_treatment", + "validate_object_type_inclusion", ] +OBJECT_TYPE_INCLUDE_DEFAULTS = { + "include_unclassified": True, + "include_galaxy": True, + "include_star": False, + "include_agn": True, + "include_qso": True, + "include_galactic": False, +} +_OBJECT_TYPE_TO_INCLUDE_KEY = { + "galaxy": "include_galaxy", + "star": "include_star", + "agn": "include_agn", + "qso": "include_qso", + "galactic": "include_galactic", +} + + +def validate_object_type_inclusion(config: Mapping[str, object] | None) -> dict[str, bool]: + """Return strict, complete object-type inclusion settings.""" + supplied = dict(config or {}) + unknown = sorted(set(supplied) - set(OBJECT_TYPE_INCLUDE_DEFAULTS)) + if unknown: + raise ValueError(f"Unknown object-type inclusion option(s): {unknown}") + result = dict(OBJECT_TYPE_INCLUDE_DEFAULTS) + for key, value in supplied.items(): + if not isinstance(value, bool): + raise TypeError(f"param.{key} must be a boolean, got {type(value).__name__}") + result[key] = value + if not any(result.values()): + raise ValueError( + "At least one object-type inclusion option must be true: " + + ", ".join(OBJECT_TYPE_INCLUDE_DEFAULTS) + ) + return result + + +def _excluded_object_type_mask( + object_types: pd.Series, + inclusion: Mapping[str, bool], +) -> pd.Series: + normalized = object_types.astype("string").str.strip().str.lower() + included = pd.Series(False, index=object_types.index, dtype=bool) + included |= normalized.isna() & bool(inclusion["include_unclassified"]) + for object_type, key in _OBJECT_TYPE_TO_INCLUDE_KEY.items(): + included |= normalized.eq(object_type).fillna(False) & bool(inclusion[key]) + return ~included + + +def _effective_z_flag_score(df: pd.DataFrame) -> pd.Series: + """Quality ranking score; stellar classes without quality receive 0.5.""" + score = _to_numeric( + df.get("z_flag_homogenized", pd.Series(np.nan, index=df.index)) + ).astype("float64") + object_type = df.get( + "object_type_homogenized", pd.Series(pd.NA, index=df.index, dtype="string") + ).astype("string").str.strip().str.lower() + stellar_without_quality = object_type.isin(["star", "galactic"]) & score.isna() + return score.mask(stellar_without_quality, 0.5) + # ----------------------- # Small helpers: string/parse/score @@ -140,24 +201,13 @@ def _validate_local_tie_invariants( *, group_col: str, tie_col: str, - z_flag_col: str = "z_flag_homogenized", ) -> None: """Validate winner/hard-tie semantics for every local component.""" - if z_flag_col not in df.columns: - raise KeyError( - f"Missing required semantic column '{z_flag_col}' for star classification" - ) - zf = pd.to_numeric(df[z_flag_col], errors="coerce") tie = pd.to_numeric(df[tie_col], errors="coerce") - stars = zf.eq(6.0) - if not tie[stars].eq(3).all(): - raise RuntimeError( - "Local tie invariant failed: every star must have tie_result=3" - ) - - nonstars = df.loc[~stars, [group_col]].copy() - nonstars["__tie"] = tie.loc[~stars].to_numpy() - for gid_value, component in nonstars.groupby(group_col, dropna=False): + participants = ~tie.eq(3.0) + participating = df.loc[participants, [group_col]].copy() + participating["__tie"] = tie.loc[participants].to_numpy() + for gid_value, component in participating.groupby(group_col, dropna=False): values = component["__tie"] n_one = int(values.eq(1).sum()) n_two = int(values.eq(2).sum()) @@ -399,11 +449,15 @@ def _edge_rows_partition( part: pd.DataFrame, crd_col: str, compared_col: str, - z_flag_col: str, + tie_col: str, ) -> pd.DataFrame: - """Extract directed non-star graph edges from one dataframe partition.""" - zf = pd.to_numeric(part[z_flag_col], errors="coerce") - work = part.loc[~zf.eq(6.0), [crd_col, compared_col]].copy() + """Extract directed graph edges from participating rows.""" + tie = ( + pd.to_numeric(part[tie_col], errors="coerce") + if tie_col in part.columns + else pd.Series(0, index=part.index, dtype="int8") + ) + work = part.loc[~tie.eq(3.0), [crd_col, compared_col]].copy() if work.empty: return pd.DataFrame( { @@ -429,7 +483,7 @@ def count_global_edge_group_mismatches( *, crd_col: str = "CRD_ID", compared_col: str = "compared_to", - z_flag_col: str = "z_flag_homogenized", + tie_col: str = "tie_result", group_col: str = "group_id", ): """Return lazy counts of cross-group edges and dangling non-star edges.""" @@ -443,14 +497,21 @@ def count_global_edge_group_mismatches( _edge_rows_partition, crd_col, compared_col, - z_flag_col, + tie_col, meta=meta, ).drop_duplicates() - zf = dd.to_numeric(df[z_flag_col], errors="coerce") + tie = ( + dd.to_numeric(df[tie_col], errors="coerce") + if tie_col in df.columns + else df[crd_col].map_partitions( + lambda s: pd.Series(0, index=s.index, dtype="int8"), + meta=pd.Series(dtype="int8"), + ) + ) groups = ( df[[crd_col, group_col]] - .assign(is_star=zf.eq(6.0)) + .assign(is_excluded=tie.eq(3.0)) .rename(columns={crd_col: "node", group_col: "node_group"}) ) groups = groups.assign(node=groups["node"].astype("string[pyarrow]")) @@ -460,17 +521,27 @@ def count_global_edge_group_mismatches( # can lose a left_on/right_on key while lowering consecutive merge/rename # expressions, producing a spurious merge key of None. groups_u = groups.rename( - columns={"node": "u", "node_group": "group_u", "is_star": "is_star_u"} + columns={ + "node": "u", + "node_group": "group_u", + "is_excluded": "is_excluded_u", + } ) groups_v = groups.rename( - columns={"node": "v", "node_group": "group_v", "is_star": "is_star_v"} + columns={ + "node": "v", + "node_group": "group_v", + "is_excluded": "is_excluded_v", + } ) checked = edges.merge(groups_u, on="u", how="left") checked = checked.merge(groups_v, on="v", how="left") dangling = checked["group_u"].isna() | checked["group_v"].isna() - both_nonstar = checked["is_star_u"].eq(False) & checked["is_star_v"].eq(False) - mismatch = (~dangling) & both_nonstar & checked["group_u"].ne(checked["group_v"]) + both_included = checked["is_excluded_u"].eq(False) & checked[ + "is_excluded_v" + ].eq(False) + mismatch = (~dangling) & both_included & checked["group_u"].ne(checked["group_v"]) return mismatch.sum(), dangling.sum() @@ -479,9 +550,26 @@ def count_global_tie_invariant_violations( *, group_col: str = "group_id", tie_col: str = "tie_result", - z_flag_col: str | None = "z_flag_homogenized", + z_flag_col: str | None = None, ): """Return a lazy count of groups violating final tie-result semantics.""" + invalid, missing_group_rows = build_global_tie_invariant_diagnostics( + df, + group_col=group_col, + tie_col=tie_col, + z_flag_col=z_flag_col, + ) + return invalid.map_partitions(len).sum() + missing_group_rows + + +def build_global_tie_invariant_diagnostics( + df: dd.DataFrame, + *, + group_col: str = "group_id", + tie_col: str = "tie_result", + z_flag_col: str | None = None, +) -> tuple[dd.DataFrame, object]: + """Build lazy per-group invariant diagnostics and missing-group count.""" tie = dd.to_numeric(df[tie_col], errors="coerce") if z_flag_col is None: nonstar_mask = ~tie.eq(3) @@ -506,7 +594,14 @@ def count_global_tie_invariant_violations( ) invalid = stats["n_invalid"].gt(0) | ~(valid_single | valid_hard) missing_group_rows = nonstars[group_col].isna().sum() - return invalid.sum() + missing_group_rows + diagnostics = stats.assign( + multiple_winners=stats["n1"].gt(1), + no_survivor=stats["n1"].eq(0) & stats["n2"].eq(0), + mixed_winner_hard_tie=stats["n1"].gt(0) & stats["n2"].gt(0), + single_hard_tie=stats["n1"].eq(0) & stats["n2"].eq(1), + invalid_tie_values=stats["n_invalid"].gt(0), + ) + return diagnostics.loc[invalid], missing_group_rows def _build_edges_fast( @@ -1012,6 +1107,7 @@ def deduplicate_pandas( partition_tag: str | None = None, logger: logging.LoggerAdapter | None = None, group_col: str | None = None, # new + object_type_inclusion: Mapping[str, object] | None = None, ) -> pd.DataFrame: """Graph-based deduplication with vectorized per-group resolution and Dz collapse. @@ -1038,6 +1134,12 @@ def deduplicate_pandas( raise KeyError(f"Missing required columns: {missing}") out = df.copy() + inclusion = validate_object_type_inclusion(object_type_inclusion) + object_types = out.get( + "object_type_homogenized", + pd.Series(pd.NA, index=out.index, dtype="string"), + ) + excluded_mask = _excluded_object_type_mask(object_types, inclusion) tie_col_orig = f"{tie_col}_orig" if tie_col in out.columns: @@ -1049,9 +1151,11 @@ def deduplicate_pandas( crd_norm = out[crd_col].astype("string").str.strip() priority_set = set(tiebreaking_priority) - zf_series: pd.Series | None = None - if "z_flag_homogenized" in out.columns: - zf_series = _to_numeric(out["z_flag_homogenized"]) + # Reuse the mature isolated-node graph path internally. Value 6 is never + # persisted; it is only an implementation marker for configuration-excluded + # rows while the legacy graph code is being generalized. + zf_series = _to_numeric(out["z_flag_homogenized"]).copy() + zf_series.loc[excluded_mask] = 6.0 # Pass edge_log down so diagnostics are computed only when requested. nodes_edge, edges_uv, diag = _build_edges_fast( @@ -1223,9 +1327,7 @@ def deduplicate_pandas( non_star = ~is_star survivors = (is_multi & non_star).copy() - zf_num = _to_numeric( - out.get("z_flag_homogenized", pd.Series(np.nan, index=out.index)) - ).astype("float64") + zf_num = _effective_z_flag_score(out) if "instrument_type_homogenized" in priority_set: if instrument_type_priority is None: @@ -1269,18 +1371,17 @@ def deduplicate_pandas( out[tie_col] = pd.Series(tr, index=out.index).astype("Int8") - if zf_series is not None: - tr_num = pd.to_numeric(out[tie_col], errors="coerce") - eq3_np = tr_num.eq(3.0).to_numpy(dtype=bool, na_value=False) - is_star_np = is_star.to_numpy(dtype=bool, na_value=False) - is_single_np = is_singleton.to_numpy(dtype=bool, na_value=False) - - invalid_3_np = eq3_np & ~is_star_np - if invalid_3_np.any(): - invalid_3 = pd.Series(invalid_3_np, index=out.index) - single = pd.Series(is_single_np, index=out.index) - out.loc[invalid_3 & single, tie_col] = np.int8(1) - out.loc[invalid_3 & ~single, tie_col] = np.int8(0) + tr_num = pd.to_numeric(out[tie_col], errors="coerce") + eq3_np = tr_num.eq(3.0).to_numpy(dtype=bool, na_value=False) + excluded_np = excluded_mask.to_numpy(dtype=bool, na_value=False) + is_single_np = is_singleton.to_numpy(dtype=bool, na_value=False) + + invalid_3_np = eq3_np & ~excluded_np + if invalid_3_np.any(): + invalid_3 = pd.Series(invalid_3_np, index=out.index) + single = pd.Series(is_single_np, index=out.index) + out.loc[invalid_3 & single, tie_col] = np.int8(1) + out.loc[invalid_3 & ~single, tie_col] = np.int8(0) z_num = _to_numeric(out[z_col]).astype("float64") f_num = zf_num.fillna(-np.inf) @@ -1430,15 +1531,10 @@ def deduplicate_pandas( out.drop(columns=drop_cols, inplace=True, errors="ignore") - if zf_series is not None: - out.loc[zf_series.eq(6).fillna(False), tie_col] = np.int8(3) + out.loc[excluded_mask, tie_col] = np.int8(3) if group_col: - _validate_local_tie_invariants( - out, - group_col=group_col, - tie_col=tie_col, - ) + _validate_local_tie_invariants(out, group_col=group_col, tie_col=tie_col) return out @@ -1521,6 +1617,7 @@ def _dedup_local_with_margin( margin_threshold_arcsec: float = 5.0, margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, + object_type_inclusion: Mapping[str, object] | None = None, ) -> pd.DataFrame: """Run dedup on (main + margin) and return labels for main rows only. @@ -1554,6 +1651,7 @@ def _dedup_local_with_margin( "ra", "dec", "z_flag_homogenized", + "object_type_homogenized", } | set( tiebreaking_priority or [] ) @@ -1620,6 +1718,7 @@ def _dedup_local_with_margin( partition_tag=partition_tag, logger=_phase_logger(), group_col=group_col, + object_type_inclusion=object_type_inclusion, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( @@ -1634,15 +1733,18 @@ def _dedup_local_with_margin( ) if group_col and group_col in solved.columns: - # Every non-star edge whose endpoints are present in this local view must + # Every participating edge whose endpoints are present in this local view must # resolve to one canonical component. This catches graph/label drift at # the partition boundary before labels are merged globally. - zf = pd.to_numeric(solved.get("z_flag_homogenized"), errors="coerce") + semantic_exclusion = pd.Series(np.nan, index=solved.index, dtype="float64") + semantic_exclusion.loc[ + pd.to_numeric(solved[tie_col], errors="coerce").eq(3.0) + ] = 6.0 edge_nodes, edge_uv, _ = _build_edges_fast( solved, crd_col=crd_col, compared_col=compared_col, - zf_series=zf, + zf_series=semantic_exclusion, edge_log=False, ) if edge_uv.size: @@ -1651,7 +1753,7 @@ def _dedup_local_with_margin( right = group_by_id.reindex(edge_nodes.take(edge_uv[:, 1])).to_numpy() if np.any(left != right): raise RuntimeError( - f"{partition_tag}: non-star edge endpoints received different group_id values" + f"{partition_tag}: participating edge endpoints received different group_id values" ) if representative_radius_diagnostics_enabled: @@ -1739,6 +1841,7 @@ def _dedup_alignfunc_with_margin( margin_threshold_arcsec: float = 5.0, margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, + object_type_inclusion: Mapping[str, object] | None = None, ) -> pd.DataFrame: """Adapter for LSDB/HATS `align_and_apply`. @@ -1770,6 +1873,7 @@ def _dedup_alignfunc_with_margin( margin_threshold_arcsec=margin_threshold_arcsec, margin_warning_fraction=margin_warning_fraction, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, + object_type_inclusion=object_type_inclusion, ) @@ -1787,6 +1891,7 @@ def _dedup_local_no_margin( group_col: str | None = None, crossmatch_radius_arcsec: float = 0.5, representative_radius_diagnostics_enabled: bool = False, + object_type_inclusion: Mapping[str, object] | None = None, ) -> pd.DataFrame: """Run dedup using only the main partition. @@ -1817,6 +1922,7 @@ def _dedup_local_no_margin( "ra", "dec", "z_flag_homogenized", + "object_type_homogenized", } | set( tiebreaking_priority or [] ) @@ -1857,6 +1963,7 @@ def _dedup_local_no_margin( partition_tag=partition_tag, logger=_phase_logger(), group_col=group_col, + object_type_inclusion=object_type_inclusion, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( @@ -1929,6 +2036,7 @@ def run_dedup_with_lsdb_map_partitions( margin_threshold_arcsec: float = 5.0, margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, + object_type_inclusion: Mapping[str, object] | None = None, ) -> dd.DataFrame: """Compute dedup labels per partition via LSDB; align divisions if margin exists. @@ -1975,6 +2083,7 @@ def run_dedup_with_lsdb_map_partitions( compared_col, z_col, "z_flag_homogenized", + "object_type_homogenized", } _assert_required(main_ddf, required_base, "main") _assert_priorities(main_ddf, list(tiebreaking_priority), "main") @@ -2022,6 +2131,7 @@ def run_dedup_with_lsdb_map_partitions( group_col=group_col, crossmatch_radius_arcsec=crossmatch_radius_arcsec, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, + object_type_inclusion=object_type_inclusion, ) else: # ------------------------------------------------------------------ @@ -2088,6 +2198,7 @@ def run_dedup_with_lsdb_map_partitions( margin_threshold_arcsec=float(margin_threshold_arcsec), margin_warning_fraction=float(margin_warning_fraction), representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, + object_type_inclusion=object_type_inclusion, ) # Build a Dask DataFrame from the delayed per-pixel label frames. diff --git a/packages/specz.py b/packages/specz.py index 289af22..c4dd001 100644 --- a/packages/specz.py +++ b/packages/specz.py @@ -2388,7 +2388,7 @@ def _requires_z_flag_homogenization(combine_mode: str, cut_value: object) -> boo numeric_cut = float(cut_value) except (TypeError, ValueError): return False - return numeric_cut in {1.0, 2.0, 3.0, 4.0, 5.0, 6.0} + return numeric_cut in {1.0, 2.0, 3.0, 4.0} def validate_combine_configuration( @@ -2425,20 +2425,6 @@ def validate_combine_configuration( f"tiebreaking_priority must be non-empty for {normalized_mode}" ) - try: - numeric_cut = float(cut_value) if cut_value is not None else None - except (TypeError, ValueError): - numeric_cut = None - if ( - normalized_mode == "concatenate_and_remove_duplicates" - and numeric_cut == 6.0 - and logger is not None - ): - logger.warning( - "z_flag_homogenized_value_to_cut=6 retains only stars, while " - "concatenate_and_remove_duplicates excludes tie_result=3; the final " - "catalog will normally be empty." - ) return normalized_mode, priorities @@ -2605,10 +2591,10 @@ def prepare_catalog( # Zero is the explicit no-cut sentinel. It is summarized once # by the driver instead of repeated for every input catalog. pass - elif cut_val not in {1.0, 2.0, 3.0, 4.0, 5.0, 6.0}: + elif cut_val not in {1.0, 2.0, 3.0, 4.0}: lg.warning( "Invalid z_flag_homogenized_value_to_cut=%s; use 0 to disable " - "the cut or one of 1, 2, 3, 4, 5, 6. Skipping cut.", + "the cut or one of 1, 2, 3, 4. Skipping cut.", z_flag_homogenized_value_to_cut, ) else: diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index c71cbf0..bfe9dd8 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -69,7 +69,7 @@ } _TRANSLATION_KEYS = { - "z_flag_translation": ("float", {0.0, 1.0, 2.0, 3.0, 4.0, 6.0}), + "z_flag_translation": ("float", {0.0, 1.0, 2.0, 3.0, 4.0}), "instrument_type_translation": ("str", {"s", "g", "p"}), "object_type_translation": ( "str", @@ -90,6 +90,11 @@ "crossmatch_saturation_fail_fraction", "crossmatch_geometry_diagnostics_enabled", "representative_radius_diagnostics_enabled", "dedup_edge_diagnostics_enabled", + "tie_invariant_diagnostics_enabled", + "tie_invariant_diagnostics_detailed_enabled", + "tie_invariant_diagnostics_sample_size", + "tie_invariant_diagnostics_max_rows", + "label_merge_diagnostics_enabled", "instrument_type_priority", "save_expr_columns", "expr_column_schema", "runtime_schema_hints", "translation_rules", } @@ -168,6 +173,23 @@ def validate_translation_config(config: dict) -> None: unknown_top = set(config) - _TOP_LEVEL_KEYS if unknown_top: raise ValueError(f"flags translation root: unknown option(s): {sorted(unknown_top)}") + for key in ( + "tie_invariant_diagnostics_enabled", + "tie_invariant_diagnostics_detailed_enabled", + "label_merge_diagnostics_enabled", + ): + if key in config and not isinstance(config[key], bool): + raise TypeError(f"{key} must be a boolean") + for key in ( + "tie_invariant_diagnostics_sample_size", + "tie_invariant_diagnostics_max_rows", + ): + if key in config and ( + not isinstance(config[key], int) + or isinstance(config[key], bool) + or config[key] < 1 + ): + raise ValueError(f"{key} must be a positive integer") rules = config.get("translation_rules", {}) runtime_hints = config.get("runtime_schema_hints", {}) if not isinstance(runtime_hints, dict): @@ -412,8 +434,8 @@ def _homogenize( product_name: Catalog identifier. logger: Logger. type_cast_ok: Whether `type` was normalized. - require_z_flag_homogenized: Create and validate the flag for semantic - star classification even when it is not a ranking priority. + require_z_flag_homogenized: Create and validate the quality flag even + when it is not a ranking priority. Returns: Tuple: (df, used_type_fastpath, tiebreaking_priority, instrument_type_priority, translation_rules_uc) @@ -777,12 +799,12 @@ def quality_like_to_flag(x): out_col="z_flag_homogenized", out_kind="float") _validate_result_domain( - "z_flag_homogenized", {0.0, 1.0, 2.0, 3.0, 4.0, 6.0} + "z_flag_homogenized", {0.0, 1.0, 2.0, 3.0, 4.0} ) else: # User-provided 'z_flag_homogenized' is present. Validate allowed domain {0,1,2,3,4} (NaN allowed). logger.info(f"{product_name} 'z_flag_homogenized' already exists; validating user-provided values.") - allowed = {0.0, 1.0, 2.0, 3.0, 4.0, 6.0} + allowed = {0.0, 1.0, 2.0, 3.0, 4.0} vals = dd.to_numeric(df["z_flag_homogenized"], errors="coerce") # NaN is allowed; only non-NaN values outside the allowed set are invalid @@ -935,18 +957,10 @@ def can_use_type_for_instrument() -> bool: if "z_flag_homogenized" not in df.columns: raise ValueError( f"[{product_name}] 'z_flag_homogenized' is required for " - "star classification but is missing after homogenization." + "quality ranking or filtering but is missing after homogenization." ) - if z_flag_is_priority: - non_null = validated_non_null_counts.get("z_flag_homogenized") - if non_null is None: - non_null = dask.compute(df["z_flag_homogenized"].count())[0] - if int(non_null) == 0: - raise ValueError( - f"[{product_name}] All values in 'z_flag_homogenized' are NaN. " - "This column is required (in tiebreaking_priority) and must contain at least one non-NaN value. " - "Verify YAML translations / fast-path logic and input columns." - ) + # An all-null quality column is valid: this priority simply cannot + # distinguish candidates. Later priorities or hard-tie handling apply. if "instrument_type_homogenized" in tiebreaking_priority: if "instrument_type_homogenized" not in df.columns: diff --git a/scripts/crd-run.py b/scripts/crd-run.py index f7e2717..8714a59 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -30,7 +30,6 @@ import dask import dask.dataframe as dd import lsdb -import numpy as np import pandas as pd # ----------------------- @@ -42,11 +41,13 @@ from dask.distributed import wait as dask_wait from deduplication import ( REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN, + build_global_tie_invariant_diagnostics, count_global_edge_group_mismatches, count_global_tie_invariant_violations, filter_dask_by_tie_treatment, filter_pandas_by_tie_treatment, run_dedup_with_lsdb_map_partitions, + validate_object_type_inclusion, validate_spatial_safety, ) from executor import get_executor @@ -786,6 +787,33 @@ def main( log_init, ) translation_config["tiebreaking_priority"] = validated_priorities + object_type_inclusion = validate_object_type_inclusion( + { + key: param_config[key] + for key in ( + "include_unclassified", + "include_galaxy", + "include_star", + "include_agn", + "include_qso", + "include_galactic", + ) + if key in param_config + } + ) + log_init.info("Object-type graph inclusion: %s", object_type_inclusion) + if ( + cut_value_numeric in {1.0, 2.0, 3.0, 4.0} + and ( + object_type_inclusion["include_star"] + or object_type_inclusion["include_galactic"] + ) + ): + log_init.warning( + "An active z_flag_homogenized cut removes star/galactic rows whose " + "quality is null before graph inclusion. Their internal 0.5 ranking " + "fallback applies only to rows that survive this cut." + ) translation_config["runtime_schema_hints"] = build_runtime_schema_hints( param_config, translation_config ) @@ -1700,6 +1728,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: margin_threshold_arcsec=margin_threshold_arcsec, margin_warning_fraction=margin_warning_fraction, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, + object_type_inclusion=object_type_inclusion, ) log_dedup.info( "Labels graph built (lazy). Persisting compact labels for " @@ -1788,6 +1817,16 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: # Coalesce tie_result in Dask (still lazy) try: + label_merge_diagnostics = bool( + translation_config.get( + "label_merge_diagnostics_enabled", True + ) + ) + missing_new_labels_lazy = ( + merged["tie_result_new"].isna().sum() + if label_merge_diagnostics + else None + ) if "tie_result" in merged.columns: merged["tie_result"] = merged["tie_result_new"].fillna( merged["tie_result"] @@ -1796,15 +1835,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: merged = merged.assign(tie_result=merged["tie_result_new"]) if "tie_result_new" in merged.columns: merged = merged.drop(columns=["tie_result_new"]) - if "z_flag_homogenized" in merged.columns: - star_mask = dd.to_numeric( - merged["z_flag_homogenized"], errors="coerce" - ).eq(6.0) - merged["tie_result"] = ( - merged["tie_result"] - .mask(star_mask, np.int8(3)) - .astype("Int8") - ) + merged["tie_result"] = merged["tie_result"].astype("Int8") validate_edges = bool( translation_config.get("validate_global_graph_edges", False) @@ -1812,6 +1843,17 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: validate_ties = bool( translation_config.get("validate_global_tie_invariants", False) ) + tie_diagnostics = bool( + translation_config.get( + "tie_invariant_diagnostics_enabled", True + ) + ) + detailed_tie_diagnostics = bool( + translation_config.get( + "tie_invariant_diagnostics_detailed_enabled", False + ) + ) + invalid_stats_lazy = None validation_tasks = [] if representative_radius_diagnostics_enabled: representative_radius = labels_dd[ @@ -1838,13 +1880,31 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: count_global_edge_group_mismatches(merged) ) validation_tasks.extend([mismatch_lazy, dangling_lazy]) + if label_merge_diagnostics: + validation_tasks.append(missing_new_labels_lazy) if validate_ties: - invalid_groups_lazy = count_global_tie_invariant_violations( - labels_dd, - z_flag_col=None, - ) - - validation_tasks.append(invalid_groups_lazy) + if tie_diagnostics or detailed_tie_diagnostics: + invalid_stats_lazy, missing_group_rows_lazy = ( + build_global_tie_invariant_diagnostics(labels_dd) + ) + validation_tasks.extend( + [ + invalid_stats_lazy.map_partitions(len).sum() + + missing_group_rows_lazy, + missing_group_rows_lazy, + invalid_stats_lazy["multiple_winners"].sum(), + invalid_stats_lazy["no_survivor"].sum(), + invalid_stats_lazy[ + "mixed_winner_hard_tie" + ].sum(), + invalid_stats_lazy["single_hard_tie"].sum(), + invalid_stats_lazy["invalid_tie_values"].sum(), + ] + ) + else: + validation_tasks.append( + count_global_tie_invariant_violations(labels_dd) + ) validation_results = iter(dask.compute(*validation_tasks)) if representative_radius_diagnostics_enabled: @@ -1898,12 +1958,81 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: "catalog concatenation." ) + if label_merge_diagnostics: + missing_new_labels = int(next(validation_results)) + log_dedup.info( + "Label merge diagnostics: rows_without_new_label=%d", + missing_new_labels, + ) + if validate_ties: invalid_groups = int(next(validation_results)) + if tie_diagnostics or detailed_tie_diagnostics: + missing_group_rows = int(next(validation_results)) + multiple_winners = int(next(validation_results)) + no_survivor = int(next(validation_results)) + mixed_winner_hard_tie = int(next(validation_results)) + single_hard_tie = int(next(validation_results)) + invalid_tie_values = int(next(validation_results)) + log_dedup.info( + "Global tie invariant diagnostics: " + "missing_group_rows=%d multiple_winners=%d " + "no_survivor=%d mixed_winner_hard_tie=%d " + "single_hard_tie=%d invalid_tie_values=%d", + missing_group_rows, + multiple_winners, + no_survivor, + mixed_winner_hard_tie, + single_hard_tie, + invalid_tie_values, + ) log_dedup.info( "Global tie invariant validation: invalid_groups=%d", invalid_groups, ) + if ( + invalid_groups + and detailed_tie_diagnostics + and invalid_stats_lazy is not None + ): + sample_size = int( + translation_config.get( + "tie_invariant_diagnostics_sample_size", 10 + ) + ) + max_rows = int( + translation_config.get( + "tie_invariant_diagnostics_max_rows", 100 + ) + ) + invalid_sample = invalid_stats_lazy.reset_index().head( + sample_size, npartitions=-1 + ) + log_dedup.error( + "Invalid tie group summary sample: %s", + invalid_sample.to_dict("records"), + ) + sample_group_ids = invalid_sample["group_id"].tolist() + if sample_group_ids: + member_columns = [ + column + for column in ( + "group_id", + "CRD_ID", + "tie_result", + "z_flag_homogenized", + "object_type_homogenized", + ) + if column in labels_dd.columns + ] + member_sample = labels_dd.loc[ + labels_dd["group_id"].isin(sample_group_ids), + member_columns, + ].head(max_rows, npartitions=-1) + log_dedup.error( + "Invalid tie group member sample: %s", + member_sample.to_dict("records"), + ) if invalid_groups: raise RuntimeError( f"Global tie-result invariants failed for " diff --git a/tests/test_dedup_partition_safety.py b/tests/test_dedup_partition_safety.py index 0a642f5..fbb00d7 100644 --- a/tests/test_dedup_partition_safety.py +++ b/tests/test_dedup_partition_safety.py @@ -14,12 +14,13 @@ _dedup_local_with_margin, _log_representative_radius_diagnostics, _validate_local_tie_invariants, + build_global_tie_invariant_diagnostics, count_global_edge_group_mismatches, count_global_tie_invariant_violations, ) -def _row(crd_id, neighbor, ra, flag): +def _row(crd_id, neighbor, ra, flag, object_type=pd.NA): return { "CRD_ID": crd_id, "compared_to": neighbor, @@ -28,6 +29,7 @@ def _row(crd_id, neighbor, ra, flag): "z": 0.1, "z_flag_homogenized": flag, "tie_result": 1, + "object_type_homogenized": object_type, } @@ -55,7 +57,7 @@ def test_boundary_component_has_same_canonical_group_from_both_pixels(): assert from_b_pixel["tie_result"] == 0 -def test_custom_priority_keeps_star_semantics_without_flag_ranking(): +def test_custom_priority_keeps_excluded_star_outside_graph(): rows = [ { **_row("A", "B, S", 10.0, 4.0), @@ -66,7 +68,7 @@ def test_custom_priority_keeps_star_semantics_without_flag_ranking(): "custom_score": 10.0, }, { - **_row("S", "A, B", 10.0 + 0.2 / 3600.0, 6.0), + **_row("S", "A, B", 10.0 + 0.2 / 3600.0, None, "star"), "custom_score": 100.0, }, ] @@ -89,7 +91,7 @@ def test_local_invariant_accepts_single_winner_and_hard_tie(): frame = pd.DataFrame( { "group_id": [1, 1, 2, 2, 3], - "z_flag_homogenized": [4, 3, 4, 4, 6], + "z_flag_homogenized": [4, 3, 4, 4, pd.NA], "tie_result": [1, 0, 2, 2, 3], } ) @@ -170,7 +172,7 @@ def test_global_tie_validation_detects_invalid_patterns(): pd.DataFrame( { "group_id": [1, 1, 2, 2, 3, 3, 4], - "z_flag_homogenized": [4, 3, 4, 4, 4, 4, 6], + "z_flag_homogenized": [4, 3, 4, 4, 4, 4, pd.NA], "tie_result": [1, 0, 2, 2, 1, 1, 3], } ), @@ -180,6 +182,12 @@ def test_global_tie_validation_detects_invalid_patterns(): assert count_global_tie_invariant_violations(frame).compute() == 1 + diagnostics, missing_group_rows = build_global_tie_invariant_diagnostics(frame) + computed = diagnostics.compute() + assert missing_group_rows.compute() == 0 + assert len(computed) == 1 + assert bool(computed.iloc[0]["multiple_winners"]) + def test_global_tie_validation_accepts_compact_labels_without_z_flag(): labels = dd.from_pandas( diff --git a/tests/test_deduplication_core.py b/tests/test_deduplication_core.py index 8a82bdb..b16d697 100644 --- a/tests/test_deduplication_core.py +++ b/tests/test_deduplication_core.py @@ -6,18 +6,30 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "packages")) -from deduplication import deduplicate_pandas # noqa: E402 +from deduplication import ( # noqa: E402 + deduplicate_pandas, + validate_object_type_inclusion, +) INSTRUMENT_PRIORITY = {"s": 3, "g": 2, "p": 1} -def _row(crd_id, compared_to, *, flag=4.0, instrument="s", z=0.1): +def _row( + crd_id, + compared_to, + *, + flag=4.0, + instrument="s", + z=0.1, + object_type=pd.NA, +): return { "CRD_ID": crd_id, "compared_to": compared_to, "z": z, "z_flag_homogenized": flag, "instrument_type_homogenized": instrument, + "object_type_homogenized": object_type, } @@ -68,11 +80,11 @@ def test_full_dedup_emits_two_and_three_way_hard_ties(): assert three_way["group_id"].nunique() == 1 -def test_stars_are_isolated_from_nonstar_deduplication(): +def test_excluded_stars_are_isolated_from_deduplication(): result = _deduplicate( [ _row("A", "S", flag=4), - _row("S", "A, B", flag=6), + _row("S", "A, B", flag=None, object_type="star"), _row("B", "S", flag=3), ] ) @@ -132,8 +144,8 @@ def test_singleton_and_dangling_neighbor_remain_winners(): def test_catalog_containing_only_stars_keeps_each_star_isolated(): result = _deduplicate( [ - _row("S1", "S2", flag=6), - _row("S2", "S1", flag=6), + _row("S1", "S2", flag=None, object_type="star"), + _row("S2", "S1", flag=None, object_type="star"), ] ) @@ -141,6 +153,86 @@ def test_catalog_containing_only_stars_keeps_each_star_isolated(): assert result["group_id"].nunique() == 2 +def test_included_star_without_quality_uses_internal_half_point_score(): + result = deduplicate_pandas( + pd.DataFrame( + [ + _row("S", "G", flag=None, object_type="star"), + _row("G", "S", flag=0, object_type="galaxy"), + ] + ), + tiebreaking_priority=["z_flag_homogenized"], + object_type_inclusion={"include_star": True}, + delta_z_threshold=0, + ).set_index("CRD_ID") + + assert result["tie_result"].astype(int).to_dict() == {"S": 1, "G": 0} + assert pd.isna(result.loc["S", "z_flag_homogenized"]) + + +def test_internal_half_point_loses_to_real_quality_one(): + result = deduplicate_pandas( + pd.DataFrame( + [ + _row("S", "G", flag=None, object_type="star"), + _row("G", "S", flag=1, object_type="galaxy"), + ] + ), + tiebreaking_priority=["z_flag_homogenized"], + object_type_inclusion={"include_star": True}, + delta_z_threshold=0, + ).set_index("CRD_ID") + + assert result["tie_result"].astype(int).to_dict() == {"S": 0, "G": 1} + + +def test_object_type_inclusion_requires_booleans_and_one_enabled_type(): + with pytest.raises(TypeError, match=r"param\.include_star must be a boolean"): + validate_object_type_inclusion({"include_star": "yes"}) + with pytest.raises(ValueError, match="At least one"): + validate_object_type_inclusion( + { + "include_unclassified": False, + "include_galaxy": False, + "include_star": False, + "include_agn": False, + "include_qso": False, + "include_galactic": False, + } + ) + + +@pytest.mark.parametrize("object_type", ["star", "galactic"]) +def test_default_object_type_policy_excludes_stellar_and_galactic(object_type): + result = _deduplicate( + [ + _row("X", "G", flag=None, object_type=object_type), + _row("G", "X", flag=4, object_type="galaxy"), + ] + ) + + assert result.loc["X", "tie_result"] == 3 + assert result.loc["G", "tie_result"] == 1 + assert result["group_id"].nunique() == 2 + + +def test_unclassified_rows_can_be_excluded_explicitly(): + result = deduplicate_pandas( + pd.DataFrame( + [ + _row("U", "G", flag=4, object_type=pd.NA), + _row("G", "U", flag=4, object_type="galaxy"), + ] + ), + tiebreaking_priority=["z_flag_homogenized"], + object_type_inclusion={"include_unclassified": False}, + group_col="group_id", + ).set_index("CRD_ID") + + assert result.loc["U", "tie_result"] == 3 + assert result.loc["G", "tie_result"] == 1 + + def test_missing_priority_column_fails_clearly(): with pytest.raises(KeyError, match="unknown_priority"): deduplicate_pandas( @@ -149,7 +241,7 @@ def test_missing_priority_column_fails_clearly(): ) -def test_missing_semantic_star_flag_fails_clearly(): +def test_missing_quality_column_fails_clearly(): frame = pd.DataFrame( { "CRD_ID": ["A"], diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index 631ffd9..d41bd94 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -97,6 +97,23 @@ def test_translation_schema_validates_output_domains_and_condition_shape(): validate_translation_config(invalid_condition) +def test_translation_schema_validates_diagnostic_controls(): + with pytest.raises(TypeError, match="tie_invariant_diagnostics_enabled"): + validate_translation_config({"tie_invariant_diagnostics_enabled": "yes"}) + with pytest.raises(ValueError, match="tie_invariant_diagnostics_sample_size"): + validate_translation_config({"tie_invariant_diagnostics_sample_size": 0}) + + validate_translation_config( + { + "tie_invariant_diagnostics_enabled": True, + "tie_invariant_diagnostics_detailed_enabled": False, + "tie_invariant_diagnostics_sample_size": 10, + "tie_invariant_diagnostics_max_rows": 100, + "label_merge_diagnostics_enabled": True, + } + ) + + def test_yaml_flag_translation_applies_direct_default_and_condition(): frame = dd.from_pandas( pd.DataFrame( @@ -303,8 +320,8 @@ def test_cosmos_web_flag_translation_compares_normalized_string_type(): result, *_ = _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) - assert result.compute()["z_flag_homogenized"].tolist() == [ - 6.0, + assert result.compute()["z_flag_homogenized"].fillna(-1).tolist() == [ + -1.0, 0.0, 1.0, 2.0, @@ -525,7 +542,7 @@ def test_2df_6df_and_2mrs_use_only_documented_object_classes(): ] -def test_z_flag_6_requires_an_explicit_stellar_classification(): +def test_stellar_classification_does_not_overwrite_independent_quality(): config_path = Path(__file__).resolve().parents[1] / "flags_translation.yaml" config = yaml.safe_load(config_path.read_text(encoding="utf-8")) config["tiebreaking_priority"] = ["z_flag_homogenized"] @@ -565,13 +582,13 @@ def test_z_flag_6_requires_an_explicit_stellar_classification(): -1, 0, -1, - 6, + 4, -1, - 6, + 4, -1, - 6, 4, - 6, + 4, + -1, ] @@ -842,7 +859,7 @@ def test_user_homogenized_flag_rejects_values_outside_domain(): _homogenize(frame, config, "demo", LOGGER, type_cast_ok=False) -def test_semantic_star_flag_is_preserved_without_being_a_priority(): +def test_quality_flag_rejects_legacy_six_and_accepts_all_null(): frame = dd.from_pandas( pd.DataFrame( { @@ -855,18 +872,29 @@ def test_semantic_star_flag_is_preserved_without_being_a_priority(): ) config = {"tiebreaking_priority": ["custom_score"]} + with pytest.raises(ValueError, match="Invalid values"): + _homogenize( + frame, + config, + "demo", + LOGGER, + type_cast_ok=False, + require_z_flag_homogenized=True, + ) + + all_null = dd.from_pandas( + pd.DataFrame({"z_flag_homogenized": [pd.NA, pd.NA]}), + npartitions=1, + sort=False, + ) result, *_ = _homogenize( - frame, - config, + all_null, + {"tiebreaking_priority": ["z_flag_homogenized"]}, "demo", LOGGER, type_cast_ok=False, - require_z_flag_homogenized=True, ) - - computed = result.compute() - assert computed["z_flag_homogenized"].tolist() == [4.0, 6.0] - assert "instrument_type_homogenized" not in computed.columns + assert result.compute()["z_flag_homogenized"].isna().all() @pytest.mark.parametrize( @@ -875,7 +903,7 @@ def test_semantic_star_flag_is_preserved_without_being_a_priority(): ("concatenate", 0, False), ("concatenate", None, False), ("concatenate", 3, True), - ("concatenate", "6", True), + ("concatenate", "6", False), ("concatenate", 1.5, False), ("concatenate", 7, False), ("concatenate_and_mark_duplicates", 0, True), @@ -938,7 +966,7 @@ def test_combine_configuration_rejects_invalid_mode_and_empty_dedup_priorities() assert priorities == [] -def test_cut_six_warns_early_for_remove_duplicates(): +def test_cut_six_has_no_legacy_star_semantics(): logger = Mock() validate_combine_configuration( @@ -948,5 +976,4 @@ def test_cut_six_warns_early_for_remove_duplicates(): logger, ) - logger.warning.assert_called_once() - assert "final catalog will normally be empty" in logger.warning.call_args.args[0] + logger.warning.assert_not_called() diff --git a/tests/test_tie_treatment.py b/tests/test_tie_treatment.py index 67b1149..f7c2ead 100644 --- a/tests/test_tie_treatment.py +++ b/tests/test_tie_treatment.py @@ -16,7 +16,7 @@ def _results(): return pd.DataFrame( { - "CRD_ID": ["winner", "loser", "tie-a", "tie-b", "star"], + "CRD_ID": ["winner", "loser", "tie-a", "tie-b", "excluded"], "group_id": [1, 1, 2, 2, 3], "tie_result": [1, 0, 2, 2, 3], } @@ -35,7 +35,7 @@ def test_final_tie_filtering_policies(option, expected): result, effective, resolved = filter_pandas_by_tie_treatment(_results(), option) assert set(result["CRD_ID"]) == expected - assert "star" not in set(result["CRD_ID"]) + assert "excluded" not in set(result["CRD_ID"]) assert effective == (option if option != "invalid" else "remove_all") assert resolved == 0 @@ -48,7 +48,7 @@ def test_draw_one_keeps_exactly_one_candidate_per_hard_tie(): assert effective == "draw_one" assert resolved == 1 assert set(result["CRD_ID"]) == {"winner", "tie-a"} - assert "star" not in set(result["CRD_ID"]) + assert "excluded" not in set(result["CRD_ID"]) def test_draw_one_without_group_id_falls_back_to_remove_all(): From b286981dfe5f25bcb942fc0d01132a54e88a81c5 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 00:34:49 -0300 Subject: [PATCH 10/16] feat: bound transitive dedup groups by representative radius --- .../crossmatch-deduplication-design.md | 7 + flags_translation.yaml | 1 + packages/deduplication.py | 192 +++++++++++++++--- packages/specz_homogenization.py | 10 + scripts/crd-run.py | 44 +++- tests/test_dedup_partition_safety.py | 31 +++ tests/test_deduplication_core.py | 66 ++++++ tests/test_homogenization.py | 3 + tests/test_spatial_config.py | 8 +- 9 files changed, 327 insertions(+), 35 deletions(-) diff --git a/docs/architecture/crossmatch-deduplication-design.md b/docs/architecture/crossmatch-deduplication-design.md index 2ba5da0..28f3485 100644 --- a/docs/architecture/crossmatch-deduplication-design.md +++ b/docs/architecture/crossmatch-deduplication-design.md @@ -253,6 +253,13 @@ Crossmatching records graph edges in `compared_to`. Deduplication then solves the connected components and assigns `tie_result` and canonical `group_id` values. +When `max_representative_radius_arcsec` is configured, each connected component +is deterministically split before tie resolution. The highest-ranked remaining +row becomes a reference, rows within the configured angular radius join its +subgroup, and the procedure repeats for rows left outside the radius. Ranking +uses `tiebreaking_priority` with `CRD_ID` as the stable final ordering key. This +limits transitive chains without discarding their more distant members. + This stage returns to LSDB/HATS because partition boundaries and margins are spatially meaningful. diff --git a/flags_translation.yaml b/flags_translation.yaml index a923a78..10a1297 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -19,6 +19,7 @@ delta_z_threshold: 0.001 # Maximum allowed redshift difference (|z1 - z2|) to # Applied only when initial tiebreaking via columns results in a tie (tie_result == 2) crossmatch_radius_arcsec: 0.5 # Angular radius (in arcseconds) used to perform spatial crossmatching. +max_representative_radius_arcsec: 1.0 # Split transitive components so every member stays within this radius of its ranked reference; null disables. margin_threshold_arcsec: 5.0 # HATS margin used by independent partition-local deduplication. margin_warning_fraction: 0.8 # Warn when a boundary component spans this fraction of the margin. validate_global_graph_edges: false # Distributed edge/group consistency check (costs two joins). diff --git a/packages/deduplication.py b/packages/deduplication.py index 8de7197..5f86bae 100644 --- a/packages/deduplication.py +++ b/packages/deduplication.py @@ -170,6 +170,7 @@ def validate_spatial_safety( crossmatch_radius_arcsec: float, margin_threshold_arcsec: float, margin_warning_fraction: float, + max_representative_radius_arcsec: float | None = None, ) -> None: """Validate parameters required by partition-local spatial deduplication.""" if crossmatch_radius_arcsec <= 0.0: @@ -184,6 +185,116 @@ def validate_spatial_safety( "margin_threshold_arcsec for partition-local deduplication " f"(radius={crossmatch_radius_arcsec}, margin={margin_threshold_arcsec})" ) + if max_representative_radius_arcsec is not None: + if max_representative_radius_arcsec < crossmatch_radius_arcsec: + raise ValueError( + "max_representative_radius_arcsec must be greater than or equal " + "to crossmatch_radius_arcsec" + ) + if max_representative_radius_arcsec >= margin_threshold_arcsec: + raise ValueError( + "max_representative_radius_arcsec must be smaller than " + "margin_threshold_arcsec" + ) + + +def _angular_distance_from_reference_arcsec( + ra_deg: np.ndarray, + dec_deg: np.ndarray, + reference_position: int, +) -> np.ndarray: + """Return great-circle distances from one array position in arcseconds.""" + ra = np.radians(ra_deg) + dec = np.radians(dec_deg) + rep_ra = ra[reference_position] + rep_dec = dec[reference_position] + dra = (ra - rep_ra + np.pi) % (2.0 * np.pi) - np.pi + hav = ( + np.sin((dec - rep_dec) / 2.0) ** 2 + + np.cos(dec) * np.cos(rep_dec) * np.sin(dra / 2.0) ** 2 + ) + return np.degrees(2.0 * np.arcsin(np.sqrt(np.clip(hav, 0.0, 1.0)))) * 3600.0 + + +def _split_groups_by_reference_radius( + df: pd.DataFrame, + initial_groups: pd.Series, + *, + max_radius_arcsec: float | None, + tiebreaking_priority: Sequence[str], + instrument_type_priority: Mapping[str, int] | None, + excluded_mask: pd.Series, + crd_col: str, +) -> pd.Series: + """Greedily split components around deterministic best-ranked references.""" + if max_radius_arcsec is None or df.empty: + return initial_groups.astype("int64") + + ranking = pd.DataFrame(index=df.index) + for order, column in enumerate(tiebreaking_priority): + if column == "instrument_type_homogenized": + if instrument_type_priority is None: + raise ValueError( + "instrument_type_priority is required when " + "'instrument_type_homogenized' is used in tiebreaking_priority." + ) + score = _score_instrument_type(df[column], instrument_type_priority) + elif column == "z_flag_homogenized": + score = _effective_z_flag_score(df) + else: + score = _to_numeric(df[column]) + ranking[f"__priority_{order}"] = score.astype("float64").fillna(-np.inf) + ranking["__crd"] = _canon_id_series(df[crd_col]).fillna("") + + sort_columns = [ + column for column in ranking.columns if column.startswith("__priority_") + ] + ["__crd"] + ascending = [False] * (len(sort_columns) - 1) + [True] + ordered_labels = ranking.sort_values( + sort_columns, + ascending=ascending, + kind="stable", + ).index + rank_position = pd.Series( + np.arange(len(ordered_labels), dtype="int64"), index=ordered_labels + ) + + ra = pd.to_numeric(df["ra"], errors="coerce").to_numpy(dtype="float64") + dec = pd.to_numeric(df["dec"], errors="coerce").to_numpy(dtype="float64") + group_values = initial_groups.to_numpy(dtype="int64", copy=True) + excluded = excluded_mask.to_numpy(dtype=bool, na_value=False) + result = np.full(len(df), -1, dtype="int64") + next_group = 0 + + for group_value in pd.unique(group_values): + positions = np.flatnonzero((group_values == group_value) & ~excluded) + remaining = set(int(position) for position in positions) + while remaining: + reference = min( + remaining, + key=lambda position: int(rank_position.loc[df.index[position]]), + ) + candidates = np.fromiter(sorted(remaining), dtype="int64") + if np.isfinite(ra[reference]) and np.isfinite(dec[reference]): + candidate_ra = ra[candidates] + candidate_dec = dec[candidates] + local_reference = int(np.flatnonzero(candidates == reference)[0]) + distances = _angular_distance_from_reference_arcsec( + candidate_ra, + candidate_dec, + local_reference, + ) + selected = candidates[distances <= float(max_radius_arcsec) + 1e-9] + else: + selected = np.asarray([reference], dtype="int64") + result[selected] = next_group + remaining.difference_update(int(position) for position in selected) + next_group += 1 + + for position in np.flatnonzero(excluded): + result[position] = next_group + next_group += 1 + return pd.Series(result, index=df.index, dtype="int64") def _assign_canonical_group_ids( @@ -1108,6 +1219,7 @@ def deduplicate_pandas( logger: logging.LoggerAdapter | None = None, group_col: str | None = None, # new object_type_inclusion: Mapping[str, object] | None = None, + max_representative_radius_arcsec: float | None = None, ) -> pd.DataFrame: """Graph-based deduplication with vectorized per-group resolution and Dz collapse. @@ -1304,7 +1416,15 @@ def deduplicate_pandas( na_mask[pos_na[is_star_na]] = False next_gid += n_star - out["__group__"] = gids + out["__group__"] = _split_groups_by_reference_radius( + out, + pd.Series(gids, index=out.index), + max_radius_arcsec=max_representative_radius_arcsec, + tiebreaking_priority=tiebreaking_priority, + instrument_type_priority=instrument_type_priority, + excluded_mask=excluded_mask, + crd_col=crd_col, + ) gid = out["__group__"] crd_s = crd_norm @@ -1618,6 +1738,7 @@ def _dedup_local_with_margin( margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, object_type_inclusion: Mapping[str, object] | None = None, + max_representative_radius_arcsec: float | None = None, ) -> pd.DataFrame: """Run dedup on (main + margin) and return labels for main rows only. @@ -1719,6 +1840,7 @@ def _dedup_local_with_margin( logger=_phase_logger(), group_col=group_col, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( @@ -1727,34 +1849,43 @@ def _dedup_local_with_margin( group_col=group_col, tie_col=tie_col, crd_col=crd_col, - radius_arcsec=crossmatch_radius_arcsec, + radius_arcsec=( + max_representative_radius_arcsec + if max_representative_radius_arcsec is not None + else crossmatch_radius_arcsec + ), partition_tag=partition_tag, ) ) if group_col and group_col in solved.columns: - # Every participating edge whose endpoints are present in this local view must - # resolve to one canonical component. This catches graph/label drift at - # the partition boundary before labels are merged globally. - semantic_exclusion = pd.Series(np.nan, index=solved.index, dtype="float64") - semantic_exclusion.loc[ - pd.to_numeric(solved[tie_col], errors="coerce").eq(3.0) - ] = 6.0 - edge_nodes, edge_uv, _ = _build_edges_fast( - solved, - crd_col=crd_col, - compared_col=compared_col, - zf_series=semantic_exclusion, - edge_log=False, - ) - if edge_uv.size: - group_by_id = solved.drop_duplicates(crd_col).set_index(crd_col)[group_col] - left = group_by_id.reindex(edge_nodes.take(edge_uv[:, 0])).to_numpy() - right = group_by_id.reindex(edge_nodes.take(edge_uv[:, 1])).to_numpy() - if np.any(left != right): - raise RuntimeError( - f"{partition_tag}: participating edge endpoints received different group_id values" - ) + if max_representative_radius_arcsec is None: + # Without radius truncation, every participating edge must remain + # inside one canonical component. + semantic_exclusion = pd.Series( + np.nan, index=solved.index, dtype="float64" + ) + semantic_exclusion.loc[ + pd.to_numeric(solved[tie_col], errors="coerce").eq(3.0) + ] = 6.0 + edge_nodes, edge_uv, _ = _build_edges_fast( + solved, + crd_col=crd_col, + compared_col=compared_col, + zf_series=semantic_exclusion, + edge_log=False, + ) + if edge_uv.size: + group_by_id = solved.drop_duplicates(crd_col).set_index(crd_col)[ + group_col + ] + left = group_by_id.reindex(edge_nodes.take(edge_uv[:, 0])).to_numpy() + right = group_by_id.reindex(edge_nodes.take(edge_uv[:, 1])).to_numpy() + if np.any(left != right): + raise RuntimeError( + f"{partition_tag}: participating edge endpoints received " + "different group_id values" + ) if representative_radius_diagnostics_enabled: src_counts = solved.groupby(group_col)["_src"].nunique() @@ -1842,6 +1973,7 @@ def _dedup_alignfunc_with_margin( margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, object_type_inclusion: Mapping[str, object] | None = None, + max_representative_radius_arcsec: float | None = None, ) -> pd.DataFrame: """Adapter for LSDB/HATS `align_and_apply`. @@ -1874,6 +2006,7 @@ def _dedup_alignfunc_with_margin( margin_warning_fraction=margin_warning_fraction, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) @@ -1892,6 +2025,7 @@ def _dedup_local_no_margin( crossmatch_radius_arcsec: float = 0.5, representative_radius_diagnostics_enabled: bool = False, object_type_inclusion: Mapping[str, object] | None = None, + max_representative_radius_arcsec: float | None = None, ) -> pd.DataFrame: """Run dedup using only the main partition. @@ -1964,6 +2098,7 @@ def _dedup_local_no_margin( logger=_phase_logger(), group_col=group_col, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( @@ -1972,7 +2107,11 @@ def _dedup_local_no_margin( group_col=group_col, tie_col=tie_col, crd_col=crd_col, - radius_arcsec=crossmatch_radius_arcsec, + radius_arcsec=( + max_representative_radius_arcsec + if max_representative_radius_arcsec is not None + else crossmatch_radius_arcsec + ), partition_tag=partition_tag, ) ) @@ -2037,6 +2176,7 @@ def run_dedup_with_lsdb_map_partitions( margin_warning_fraction: float = 0.8, representative_radius_diagnostics_enabled: bool = False, object_type_inclusion: Mapping[str, object] | None = None, + max_representative_radius_arcsec: float | None = None, ) -> dd.DataFrame: """Compute dedup labels per partition via LSDB; align divisions if margin exists. @@ -2132,6 +2272,7 @@ def run_dedup_with_lsdb_map_partitions( crossmatch_radius_arcsec=crossmatch_radius_arcsec, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) else: # ------------------------------------------------------------------ @@ -2199,6 +2340,7 @@ def run_dedup_with_lsdb_map_partitions( margin_warning_fraction=float(margin_warning_fraction), representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) # Build a Dask DataFrame from the delayed per-pixel label frames. diff --git a/packages/specz_homogenization.py b/packages/specz_homogenization.py index bfe9dd8..94994d9 100644 --- a/packages/specz_homogenization.py +++ b/packages/specz_homogenization.py @@ -82,6 +82,7 @@ } _TOP_LEVEL_KEYS = { "tiebreaking_priority", "delta_z_threshold", "crossmatch_radius_arcsec", + "max_representative_radius_arcsec", "margin_threshold_arcsec", "margin_warning_fraction", "validate_global_graph_edges", "validate_global_tie_invariants", "validate_crd_id_uniqueness", "repartition_prepared_catalogs", @@ -190,6 +191,15 @@ def validate_translation_config(config: dict) -> None: or config[key] < 1 ): raise ValueError(f"{key} must be a positive integer") + max_radius = config.get("max_representative_radius_arcsec") + if max_radius is not None and ( + isinstance(max_radius, bool) + or not isinstance(max_radius, (int, float)) + or float(max_radius) <= 0.0 + ): + raise ValueError( + "max_representative_radius_arcsec must be a positive number or null" + ) rules = config.get("translation_rules", {}) runtime_hints = config.get("runtime_schema_hints", {}) if not isinstance(runtime_hints, dict): diff --git a/scripts/crd-run.py b/scripts/crd-run.py index 8714a59..2c1c497 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -744,15 +744,28 @@ def main( margin_warning_fraction = float( translation_config.get("margin_warning_fraction", 0.8) ) + configured_max_radius = translation_config.get( + "max_representative_radius_arcsec", None + ) + max_representative_radius_arcsec = ( + None if configured_max_radius is None else float(configured_max_radius) + ) validate_spatial_safety( crossmatch_radius_arcsec, margin_threshold_arcsec, margin_warning_fraction, + max_representative_radius_arcsec, ) log_init.info( - 'Spatial safety: crossmatch_radius=%.3f" margin_threshold=%.3f" ratio=%.3f', + 'Spatial safety: crossmatch_radius=%.3f" margin_threshold=%.3f" ' + "max_representative_radius=%s ratio=%.3f", crossmatch_radius_arcsec, margin_threshold_arcsec, + ( + "disabled" + if max_representative_radius_arcsec is None + else f'{max_representative_radius_arcsec:.3f}"' + ), crossmatch_radius_arcsec / margin_threshold_arcsec, ) # Minimal summary of loaded translation (no heavy dumping) @@ -1729,6 +1742,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: margin_warning_fraction=margin_warning_fraction, representative_radius_diagnostics_enabled=representative_radius_diagnostics_enabled, object_type_inclusion=object_type_inclusion, + max_representative_radius_arcsec=max_representative_radius_arcsec, ) log_dedup.info( "Labels graph built (lazy). Persisting compact labels for " @@ -1855,6 +1869,11 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: ) invalid_stats_lazy = None validation_tasks = [] + representative_limit_arcsec = ( + max_representative_radius_arcsec + if max_representative_radius_arcsec is not None + else crossmatch_radius_arcsec + ) if representative_radius_diagnostics_enabled: representative_radius = labels_dd[ REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN @@ -1864,10 +1883,10 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: [ representative_radius_valid.count(), representative_radius_valid.gt( - crossmatch_radius_arcsec + representative_limit_arcsec ).sum(), representative_radius_valid.gt( - 2.0 * crossmatch_radius_arcsec + 2.0 * representative_limit_arcsec ).sum(), representative_radius_valid.gt( margin_threshold_arcsec @@ -1926,7 +1945,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: "fraction_exceeding=%.6f exceeding_twice_radius=%d " "exceeding_margin=%d max_radius=%.4farcsec", representative_components, - crossmatch_radius_arcsec, + representative_limit_arcsec, representative_exceed_radius, representative_fraction, representative_exceed_twice_radius, @@ -1943,11 +1962,18 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: dangling_count, ) if mismatch_count: - raise RuntimeError( - "Partition-local deduplication produced " - f"{mismatch_count} non-star edges whose endpoints " - "have different canonical group_id values. Increase " - "margin_threshold_arcsec or inspect long components." + if max_representative_radius_arcsec is None: + raise RuntimeError( + "Partition-local deduplication produced " + f"{mismatch_count} participating edges whose " + "endpoints have different canonical group_id " + "values. Increase margin_threshold_arcsec or " + "inspect long components." + ) + log_dedup.info( + "Cross-group edges are expected with representative-" + "radius truncation enabled (radius=%.3f arcsec).", + max_representative_radius_arcsec, ) if dangling_count: raise RuntimeError( diff --git a/tests/test_dedup_partition_safety.py b/tests/test_dedup_partition_safety.py index fbb00d7..bcb2392 100644 --- a/tests/test_dedup_partition_safety.py +++ b/tests/test_dedup_partition_safety.py @@ -57,6 +57,37 @@ def test_boundary_component_has_same_canonical_group_from_both_pixels(): assert from_b_pixel["tie_result"] == 0 +def test_reference_radius_split_is_stable_across_main_margin_views(): + rows = [ + _row("A", "B", 10.0, 4.0), + _row("B", "A, C", 10.0 + 0.5 / 3600.0, 3.0), + _row("C", "B, D", 10.0 + 1.0 / 3600.0, 2.0), + _row("D", "C", 10.0 + 1.5 / 3600.0, 1.0), + ] + first = _dedup_local_with_margin( + pd.DataFrame(rows[:2]), + pd.DataFrame(rows[2:]), + pixel=None, + tiebreaking_priority=["z_flag_homogenized"], + instrument_type_priority=None, + group_col="group_id", + max_representative_radius_arcsec=1.0, + ).set_index("CRD_ID") + second = _dedup_local_with_margin( + pd.DataFrame(rows[2:]), + pd.DataFrame(rows[:2]), + pixel=None, + tiebreaking_priority=["z_flag_homogenized"], + instrument_type_priority=None, + group_col="group_id", + max_representative_radius_arcsec=1.0, + ).set_index("CRD_ID") + + assert first.loc["A", "group_id"] == first.loc["B", "group_id"] + assert second.loc["C", "group_id"] == first.loc["A", "group_id"] + assert second.loc["D", "group_id"] != first.loc["A", "group_id"] + + def test_custom_priority_keeps_excluded_star_outside_graph(): rows = [ { diff --git a/tests/test_deduplication_core.py b/tests/test_deduplication_core.py index b16d697..0442d2f 100644 --- a/tests/test_deduplication_core.py +++ b/tests/test_deduplication_core.py @@ -233,6 +233,72 @@ def test_unclassified_rows_can_be_excluded_explicitly(): assert result.loc["G", "tie_result"] == 1 +def _chain_rows(): + return [ + { + **_row("A", "B", flag=4, z=0.1), + "ra": 10.0, + "dec": 0.0, + }, + { + **_row("B", "A, C", flag=3, z=0.1), + "ra": 10.0 + 0.5 / 3600.0, + "dec": 0.0, + }, + { + **_row("C", "B, D", flag=2, z=0.1), + "ra": 10.0 + 1.0 / 3600.0, + "dec": 0.0, + }, + { + **_row("D", "C", flag=1, z=0.1), + "ra": 10.0 + 1.5 / 3600.0, + "dec": 0.0, + }, + ] + + +def test_reference_radius_splits_long_transitive_chain(): + result = deduplicate_pandas( + pd.DataFrame(_chain_rows()), + tiebreaking_priority=["z_flag_homogenized"], + max_representative_radius_arcsec=1.0, + delta_z_threshold=0, + group_col="group_id", + ).set_index("CRD_ID") + + assert result.loc[["A", "B", "C"], "group_id"].nunique() == 1 + assert result.loc["D", "group_id"] != result.loc["A", "group_id"] + assert result["tie_result"].astype(int).to_dict() == { + "A": 1, + "B": 0, + "C": 0, + "D": 1, + } + + +def test_reference_radius_split_is_independent_of_row_order(): + forward = deduplicate_pandas( + pd.DataFrame(_chain_rows()), + tiebreaking_priority=["z_flag_homogenized"], + max_representative_radius_arcsec=1.0, + delta_z_threshold=0, + group_col="group_id", + ).set_index("CRD_ID") + reverse = deduplicate_pandas( + pd.DataFrame(list(reversed(_chain_rows()))), + tiebreaking_priority=["z_flag_homogenized"], + max_representative_radius_arcsec=1.0, + delta_z_threshold=0, + group_col="group_id", + ).set_index("CRD_ID") + + assert forward["group_id"].to_dict() == reverse["group_id"].to_dict() + assert forward["tie_result"].astype(int).to_dict() == reverse[ + "tie_result" + ].astype(int).to_dict() + + def test_missing_priority_column_fails_clearly(): with pytest.raises(KeyError, match="unknown_priority"): deduplicate_pandas( diff --git a/tests/test_homogenization.py b/tests/test_homogenization.py index d41bd94..a70031e 100644 --- a/tests/test_homogenization.py +++ b/tests/test_homogenization.py @@ -102,6 +102,8 @@ def test_translation_schema_validates_diagnostic_controls(): validate_translation_config({"tie_invariant_diagnostics_enabled": "yes"}) with pytest.raises(ValueError, match="tie_invariant_diagnostics_sample_size"): validate_translation_config({"tie_invariant_diagnostics_sample_size": 0}) + with pytest.raises(ValueError, match="max_representative_radius_arcsec"): + validate_translation_config({"max_representative_radius_arcsec": 0}) validate_translation_config( { @@ -110,6 +112,7 @@ def test_translation_schema_validates_diagnostic_controls(): "tie_invariant_diagnostics_sample_size": 10, "tie_invariant_diagnostics_max_rows": 100, "label_merge_diagnostics_enabled": True, + "max_representative_radius_arcsec": 1.0, } ) diff --git a/tests/test_spatial_config.py b/tests/test_spatial_config.py index 71feda6..2b70d83 100644 --- a/tests/test_spatial_config.py +++ b/tests/test_spatial_config.py @@ -9,7 +9,13 @@ def test_spatial_safety_accepts_radius_smaller_than_margin(): - validate_spatial_safety(0.5, 5.0, 0.8) + validate_spatial_safety(0.5, 5.0, 0.8, 1.0) + + +@pytest.mark.parametrize("max_radius", [0.4, 5.0, 6.0]) +def test_spatial_safety_rejects_unsafe_representative_radius(max_radius): + with pytest.raises(ValueError, match="max_representative_radius_arcsec"): + validate_spatial_safety(0.5, 5.0, 0.8, max_radius) @pytest.mark.parametrize( From 63cff44bfadc6a5643abbcbae29a8a13980cd807 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 09:20:26 -0300 Subject: [PATCH 11/16] Correcting distance diagnostics to use the reference objects from the subgroups --- packages/deduplication.py | 57 ++++++++++++++++++++++------ tests/test_dedup_partition_safety.py | 29 ++++++++++++++ 2 files changed, 74 insertions(+), 12 deletions(-) diff --git a/packages/deduplication.py b/packages/deduplication.py index 5f86bae..e32175d 100644 --- a/packages/deduplication.py +++ b/packages/deduplication.py @@ -30,6 +30,7 @@ import pandas as pd REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN = "_diag_representative_max_radius_arcsec" +SPLIT_REFERENCE_COLUMN = "__split_reference_crd" # ----------------------- # Project @@ -225,10 +226,13 @@ def _split_groups_by_reference_radius( instrument_type_priority: Mapping[str, int] | None, excluded_mask: pd.Series, crd_col: str, -) -> pd.Series: +) -> tuple[pd.Series, pd.Series]: """Greedily split components around deterministic best-ranked references.""" if max_radius_arcsec is None or df.empty: - return initial_groups.astype("int64") + return ( + initial_groups.astype("int64"), + pd.Series(pd.NA, index=df.index, dtype="string"), + ) ranking = pd.DataFrame(index=df.index) for order, column in enumerate(tiebreaking_priority): @@ -264,6 +268,7 @@ def _split_groups_by_reference_radius( group_values = initial_groups.to_numpy(dtype="int64", copy=True) excluded = excluded_mask.to_numpy(dtype=bool, na_value=False) result = np.full(len(df), -1, dtype="int64") + references = np.full(len(df), None, dtype=object) next_group = 0 for group_value in pd.unique(group_values): @@ -288,13 +293,18 @@ def _split_groups_by_reference_radius( else: selected = np.asarray([reference], dtype="int64") result[selected] = next_group + references[selected] = str(df.iloc[reference][crd_col]).strip() remaining.difference_update(int(position) for position in selected) next_group += 1 for position in np.flatnonzero(excluded): result[position] = next_group + references[position] = str(df.iloc[position][crd_col]).strip() next_group += 1 - return pd.Series(result, index=df.index, dtype="int64") + return ( + pd.Series(result, index=df.index, dtype="int64"), + pd.Series(references, index=df.index, dtype="string"), + ) def _assign_canonical_group_ids( @@ -355,14 +365,31 @@ def _log_representative_radius_diagnostics( if work.empty: return diagnostic - tie = pd.to_numeric(work[tie_col], errors="coerce") - work["_representative_rank"] = np.select([tie.eq(1), tie.eq(2)], [0, 1], default=2) - representatives = ( - work.sort_values([group_col, "_representative_rank", crd_col], kind="stable") - .drop_duplicates(group_col) - .set_index(group_col)[["ra", "dec"]] - .rename(columns={"ra": "_rep_ra", "dec": "_rep_dec"}) - ) + if SPLIT_REFERENCE_COLUMN in frame.columns: + work[SPLIT_REFERENCE_COLUMN] = frame.loc[ + work.index, SPLIT_REFERENCE_COLUMN + ].astype("string") + reference_rows = work[work[crd_col].astype("string").eq( + work[SPLIT_REFERENCE_COLUMN] + )] + representatives = ( + reference_rows.drop_duplicates(group_col) + .set_index(group_col)[["ra", "dec"]] + .rename(columns={"ra": "_rep_ra", "dec": "_rep_dec"}) + ) + else: + tie = pd.to_numeric(work[tie_col], errors="coerce") + work["_representative_rank"] = np.select( + [tie.eq(1), tie.eq(2)], [0, 1], default=2 + ) + representatives = ( + work.sort_values( + [group_col, "_representative_rank", crd_col], kind="stable" + ) + .drop_duplicates(group_col) + .set_index(group_col)[["ra", "dec"]] + .rename(columns={"ra": "_rep_ra", "dec": "_rep_dec"}) + ) work = work.join(representatives, on=group_col) ra = np.radians(work["ra"].to_numpy(dtype="float64")) @@ -1220,6 +1247,7 @@ def deduplicate_pandas( group_col: str | None = None, # new object_type_inclusion: Mapping[str, object] | None = None, max_representative_radius_arcsec: float | None = None, + preserve_split_reference: bool = False, ) -> pd.DataFrame: """Graph-based deduplication with vectorized per-group resolution and Dz collapse. @@ -1416,7 +1444,7 @@ def deduplicate_pandas( na_mask[pos_na[is_star_na]] = False next_gid += n_star - out["__group__"] = _split_groups_by_reference_radius( + out["__group__"], out[SPLIT_REFERENCE_COLUMN] = _split_groups_by_reference_radius( out, pd.Series(gids, index=out.index), max_radius_arcsec=max_representative_radius_arcsec, @@ -1649,6 +1677,9 @@ def deduplicate_pandas( else: drop_cols.append("__group__") + if not preserve_split_reference: + drop_cols.append(SPLIT_REFERENCE_COLUMN) + out.drop(columns=drop_cols, inplace=True, errors="ignore") out.loc[excluded_mask, tie_col] = np.int8(3) @@ -1841,6 +1872,7 @@ def _dedup_local_with_margin( group_col=group_col, object_type_inclusion=object_type_inclusion, max_representative_radius_arcsec=max_representative_radius_arcsec, + preserve_split_reference=representative_radius_diagnostics_enabled, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( @@ -2099,6 +2131,7 @@ def _dedup_local_no_margin( group_col=group_col, object_type_inclusion=object_type_inclusion, max_representative_radius_arcsec=max_representative_radius_arcsec, + preserve_split_reference=representative_radius_diagnostics_enabled, ) if representative_radius_diagnostics_enabled: solved[REPRESENTATIVE_RADIUS_DIAGNOSTIC_COLUMN] = ( diff --git a/tests/test_dedup_partition_safety.py b/tests/test_dedup_partition_safety.py index bcb2392..9083267 100644 --- a/tests/test_dedup_partition_safety.py +++ b/tests/test_dedup_partition_safety.py @@ -10,6 +10,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "packages")) from deduplication import ( # noqa: E402 + SPLIT_REFERENCE_COLUMN, _collapse_within_dz, _dedup_local_with_margin, _log_representative_radius_diagnostics, @@ -267,3 +268,31 @@ def test_representative_radius_diagnostic_warns_for_transitive_chain(): assert "Representative-radius diagnostics" in logger.warning.call_args.args[0] assert diagnostic.notna().sum() == 1 assert diagnostic.max() == pytest.approx(0.8, abs=1e-6) + + +def test_representative_radius_diagnostic_uses_split_reference_not_winner(): + frame = pd.DataFrame( + { + "group_id": [1, 1, 1], + "tie_result": [0, 1, 0], + "CRD_ID": ["reference", "winner", "edge"], + SPLIT_REFERENCE_COLUMN: ["reference"] * 3, + "ra": [10.0, 10.0 + 0.8 / 3600.0, 10.0 - 0.8 / 3600.0], + "dec": [0.0, 0.0, 0.0], + } + ) + logger = Mock() + + with patch("deduplication._phase_logger", return_value=logger): + diagnostic = _log_representative_radius_diagnostics( + frame, + group_col="group_id", + tie_col="tie_result", + crd_col="CRD_ID", + radius_arcsec=1.0, + partition_tag="test", + ) + + logger.warning.assert_not_called() + assert diagnostic.notna().sum() == 1 + assert diagnostic.max() == pytest.approx(0.8, abs=1e-6) From 8a0cfaacad7389e8ce69c2fc873789acba610340 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 09:28:50 -0300 Subject: [PATCH 12/16] Correcting logs texts and comments for the new object type oriented workflow --- flags_translation.yaml | 2 +- packages/deduplication.py | 30 ++++++++++++++++++++++-------- scripts/crd-run.py | 4 ++-- 3 files changed, 25 insertions(+), 11 deletions(-) diff --git a/flags_translation.yaml b/flags_translation.yaml index 10a1297..6c4802e 100644 --- a/flags_translation.yaml +++ b/flags_translation.yaml @@ -39,7 +39,7 @@ crossmatch_saturation_warn_fraction: 0.01 # Warn when >=1% of all source object crossmatch_saturation_fail_fraction: 0.10 # Fail at >=10%; set null to disable failure. crossmatch_geometry_diagnostics_enabled: false # Separation/component-shape diagnostics over materialized matches. representative_radius_diagnostics_enabled: false # Component radius/extent diagnostics during and after deduplication. -dedup_edge_diagnostics_enabled: false # Additional star-edge diagnostics while building the deduplication graph. +dedup_edge_diagnostics_enabled: false # Additional diagnostics for object types excluded while building the deduplication graph. instrument_type_priority: s: 3 # Spectroscopic (most reliable type) diff --git a/packages/deduplication.py b/packages/deduplication.py index e32175d..b26528e 100644 --- a/packages/deduplication.py +++ b/packages/deduplication.py @@ -624,7 +624,7 @@ def count_global_edge_group_mismatches( tie_col: str = "tie_result", group_col: str = "group_id", ): - """Return lazy counts of cross-group edges and dangling non-star edges.""" + """Return lazy counts of cross-group edges and dangling participating edges.""" meta = pd.DataFrame( { "u": pd.Series(dtype="string[pyarrow]"), @@ -847,7 +847,8 @@ def _build_edges_fast( leaked = [cid for cid in nodes_edge.astype("string") if cid in star_ids_fast] if leaked: _phase_logger().error( - "Star IDs leaked into fast-path edge nodes (logic violation): " + "Excluded object-type IDs leaked into fast-path edge nodes " + "(logic violation): " "count=%d sample=%s", len(leaked), leaked[:5], @@ -1275,6 +1276,13 @@ def deduplicate_pandas( out = df.copy() inclusion = validate_object_type_inclusion(object_type_inclusion) + excluded_object_types = sorted( + object_type + for object_type, option in _OBJECT_TYPE_TO_INCLUDE_KEY.items() + if not inclusion[option] + ) + if not inclusion["include_unclassified"]: + excluded_object_types.append("unclassified") object_types = out.get( "object_type_homogenized", pd.Series(pd.NA, index=out.index, dtype="string"), @@ -1310,23 +1318,29 @@ def deduplicate_pandas( if edge_log: lg = logger or _phase_logger() tag = f"[{partition_tag}]" if partition_tag else "[global]" - starB = diag.get("n_edges_starB_excluded") - if isinstance(starB, int) and starB > 0: + excluded_neighbor_edges = diag.get("n_edges_starB_excluded") + if isinstance(excluded_neighbor_edges, int) and excluded_neighbor_edges > 0: lg.warning( - "%s Star neighbors excluded during edge build: star_rows_excl=%d, edges_raw=%d, starB_excl=%d, edges_kept=%d", + "%s Object types excluded from deduplication graph: " + "types=%s, rows_excluded=%d, edges_raw=%d, " + "excluded_neighbor_edges=%d, edges_kept=%d", tag, + excluded_object_types, diag.get("n_rows_star_excluded", 0), diag.get("n_edges_raw", 0), - starB, + excluded_neighbor_edges, diag.get("n_edges_kept", 0), ) elif partition_tag is None: lg.info( - "%s Edge build summary: star_rows_excl=%d, edges_raw=%d, starB_excl=%s, edges_kept=%d", + "%s Edge build summary: excluded_object_types=%s, " + "rows_excluded=%d, edges_raw=%d, " + "excluded_neighbor_edges=%s, edges_kept=%d", tag, + excluded_object_types, diag.get("n_rows_star_excluded", 0), diag.get("n_edges_raw", 0), - str(starB), + str(excluded_neighbor_edges), diag.get("n_edges_kept", 0), ) diff --git a/scripts/crd-run.py b/scripts/crd-run.py index 2c1c497..17ed891 100644 --- a/scripts/crd-run.py +++ b/scripts/crd-run.py @@ -1684,7 +1684,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: ####################################################################### # Diagnostics / outputs - # - edge_log: enable edge diagnostics (warn on star-neighbor exclusions) + # - edge_log: enable diagnostics for object types excluded from the graph # - group_col: set to None to disable exporting group labels edge_log = bool( translation_config.get("dedup_edge_diagnostics_enabled", False) @@ -1957,7 +1957,7 @@ def _submit_pair(executor: ThreadPoolExecutor) -> bool: dangling_count = int(next(validation_results)) log_dedup.info( "Global graph validation: cross_group_edges=%d " - "dangling_nonstar_edges=%d", + "dangling_participating_edges=%d", mismatch_count, dangling_count, ) From 177d441579672a008e05df36afef28c242698c4d Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 10:55:35 -0300 Subject: [PATCH 13/16] refactor: replace legacy z-flag 6 sentinel with exclusion mask --- packages/deduplication.py | 199 ++++++++++++++++---------------------- 1 file changed, 85 insertions(+), 114 deletions(-) diff --git a/packages/deduplication.py b/packages/deduplication.py index b26528e..b470e1e 100644 --- a/packages/deduplication.py +++ b/packages/deduplication.py @@ -709,19 +709,17 @@ def build_global_tie_invariant_diagnostics( ) -> tuple[dd.DataFrame, object]: """Build lazy per-group invariant diagnostics and missing-group count.""" tie = dd.to_numeric(df[tie_col], errors="coerce") - if z_flag_col is None: - nonstar_mask = ~tie.eq(3) - else: - zf = dd.to_numeric(df[z_flag_col], errors="coerce") - nonstar_mask = ~zf.eq(6.0) - nonstars = df.loc[nonstar_mask, [group_col]].assign( + # tie_result=3 is the public, type-agnostic marker for rows excluded from + # the graph. z_flag_col is retained only for API compatibility. + participating_mask = ~tie.eq(3) + participants = df.loc[participating_mask, [group_col]].assign( n=1, n0=tie.eq(0).astype("int8"), n1=tie.eq(1).astype("int8"), n2=tie.eq(2).astype("int8"), n_invalid=(~tie.isin([0, 1, 2])).astype("int8"), ) - stats = nonstars.groupby(group_col).agg( + stats = participants.groupby(group_col).agg( {"n": "sum", "n0": "sum", "n1": "sum", "n2": "sum", "n_invalid": "sum"} ) valid_single = ( @@ -731,7 +729,7 @@ def build_global_tie_invariant_diagnostics( stats["n1"].eq(0) & stats["n2"].ge(2) & stats["n0"].eq(stats["n"] - stats["n2"]) ) invalid = stats["n_invalid"].gt(0) | ~(valid_single | valid_hard) - missing_group_rows = nonstars[group_col].isna().sum() + missing_group_rows = participants[group_col].isna().sum() diagnostics = stats.assign( multiple_winners=stats["n1"].gt(1), no_survivor=stats["n1"].eq(0) & stats["n2"].eq(0), @@ -747,10 +745,10 @@ def _build_edges_fast( *, crd_col: str, compared_col: str, - zf_series: pd.Series | None = None, + excluded_mask: pd.Series | None = None, edge_log: bool = False, ): - """Build undirected edges among NON-STAR rows (vectorized path). + """Build undirected edges among rows participating in the graph. Returns: (nodes_index, edges_uv, diag) where: @@ -759,17 +757,16 @@ def _build_edges_fast( - diag: dict with basic diagnostics (counts). If `edge_log` is False, only cheap counts are filled; expensive ones are set to None. """ - # --- filter A-side (rows) to non-stars - non_star_mask = pd.Series(True, index=df.index) - if zf_series is not None: - non_star_mask &= ~pd.to_numeric(zf_series, errors="coerce").eq(6) + participating_mask = pd.Series(True, index=df.index, dtype="boolean") + if excluded_mask is not None: + participating_mask &= ~excluded_mask.reindex(df.index).fillna(False) - A_df = df.loc[non_star_mask, [crd_col, compared_col]].copy().reset_index(drop=True) + A_df = df.loc[participating_mask, [crd_col, compared_col]].copy().reset_index(drop=True) if A_df.empty: diag = { "edge_log_enabled": bool(edge_log), - "n_rows_nonstar": int(non_star_mask.sum()), - "n_rows_star_excluded": int((~non_star_mask).sum()), + "n_rows_nonstar": int(participating_mask.sum()), + "n_rows_star_excluded": int((~participating_mask).sum()), "n_edges_raw": 0, "n_edges_kept": 0, "n_edges_starB_excluded": None if not edge_log else 0, @@ -778,15 +775,13 @@ def _build_edges_fast( # Non-star IDs present on A-side present_nonstar = set( - _canon_id_series(df.loc[non_star_mask, crd_col]).dropna().unique() + _canon_id_series(df.loc[participating_mask, crd_col]).dropna().unique() ) # If diagnostics are enabled, precompute star IDs (for B-side exclusion count) - if edge_log and (zf_series is not None): + if edge_log and (excluded_mask is not None): star_ids = set( - _canon_id_series( - df.loc[pd.to_numeric(zf_series, errors="coerce").eq(6), crd_col] - ) + _canon_id_series(df.loc[~participating_mask, crd_col]) .dropna() .unique() ) @@ -822,8 +817,8 @@ def _build_edges_fast( if edges_raw.empty: diag = { "edge_log_enabled": bool(edge_log), - "n_rows_nonstar": int(non_star_mask.sum()), - "n_rows_star_excluded": int((~non_star_mask).sum()), + "n_rows_nonstar": int(participating_mask.sum()), + "n_rows_star_excluded": int((~participating_mask).sum()), "n_edges_raw": n_edges_raw, "n_edges_kept": 0, "n_edges_starB_excluded": n_edges_starB_excluded, @@ -836,11 +831,9 @@ def _build_edges_fast( ) # --- EXTRA LOG (edge_log): sanity-check that no star IDs leaked into edge nodes - if edge_log and (zf_series is not None) and len(nodes_edge): + if edge_log and (excluded_mask is not None) and len(nodes_edge): star_ids_fast = set( - _canon_id_series( - df.loc[pd.to_numeric(zf_series, errors="coerce").eq(6), crd_col] - ) + _canon_id_series(df.loc[~participating_mask, crd_col]) .dropna() .unique() ) @@ -865,8 +858,8 @@ def _build_edges_fast( if lo.size == 0: diag = { "edge_log_enabled": bool(edge_log), - "n_rows_nonstar": int(non_star_mask.sum()), - "n_rows_star_excluded": int((~non_star_mask).sum()), + "n_rows_nonstar": int(participating_mask.sum()), + "n_rows_star_excluded": int((~participating_mask).sum()), "n_edges_raw": n_edges_raw, "n_edges_kept": 0, "n_edges_starB_excluded": n_edges_starB_excluded, @@ -879,8 +872,8 @@ def _build_edges_fast( diag = { "edge_log_enabled": bool(edge_log), - "n_rows_nonstar": int(non_star_mask.sum()), - "n_rows_star_excluded": int((~non_star_mask).sum()), + "n_rows_nonstar": int(participating_mask.sum()), + "n_rows_star_excluded": int((~participating_mask).sum()), "n_edges_raw": n_edges_raw, "n_edges_kept": int(uv.shape[0]), # None when edge_log=False to indicate we skipped the costly check @@ -1095,14 +1088,16 @@ def _to_numeric(series_like) -> pd.Series: # ----------------------- # Guard restore # ----------------------- -def _only_star_neighbors_series(col: pd.Series, star_ids: set[str]) -> pd.Series: - """True when compared_to is non-empty AND all neighbors are star IDs.""" +def _only_excluded_neighbors_series( + col: pd.Series, excluded_ids: set[str] +) -> pd.Series: + """True when compared_to is non-empty and all neighbors are excluded.""" s = col.astype("string").fillna("") lst = s.str.split(",") out = [] for tokens in lst: toks = [t.strip() for t in tokens if t and t.strip()] - out.append(bool(toks) and all((tok in star_ids) for tok in toks)) + out.append(bool(toks) and all((tok in excluded_ids) for tok in toks)) return pd.Series(out, index=col.index, dtype="boolean") @@ -1111,17 +1106,17 @@ def _apply_guard_restore_local( *, crd_col: str, compared_col: str, - zf_series: pd.Series | None, + excluded_mask: pd.Series, tie_col: str, tie_col_orig: str, ) -> pd.DataFrame: - """Restore original tie_result for non-stars with empty/only-star neighbors. + """Restore original labels for participants with no participating neighbors. Args: df: Partition dataframe. crd_col: Name of the CRD_ID column. compared_col: Name of the compared_to column. - zf_series: Optional z_flag series. + excluded_mask: Boolean mask for rows excluded from the graph. tie_col: Name of the tie_result column. tie_col_orig: Name of the original tie_result column. @@ -1131,25 +1126,21 @@ def _apply_guard_restore_local( if tie_col_orig not in df.columns: return df # nothing to restore - # Stars (fixed as 3) and empty compared_to mask. - is_star = pd.Series(False, index=df.index) - if zf_series is not None: - is_star = pd.to_numeric(zf_series, errors="coerce").eq(6.0) + excluded = excluded_mask.reindex(df.index).fillna(False).astype(bool) cmp_str = df[compared_col].astype("string") cmp_empty = cmp_str.isna() | cmp_str.str.strip().eq("") - # Local set of star IDs for this partition/view. - star_ids = set(df.loc[is_star, crd_col].astype("string")) + excluded_ids = set(df.loc[excluded, crd_col].astype("string")) - only_star_neighbors = (~cmp_empty) & _only_star_neighbors_series( - df[compared_col], star_ids + only_excluded_neighbors = (~cmp_empty) & _only_excluded_neighbors_series( + df[compared_col], excluded_ids ) # Rule: - # - if star -> keep tie=3 (do not restore) - # - if non-star and (empty compared_to OR only star neighbors) -> restore - restore_mask = (~is_star) & (cmp_empty | only_star_neighbors) + # Excluded rows remain tie_result=3. Participating rows with no usable + # neighbors recover their original label. + restore_mask = (~excluded) & (cmp_empty | only_excluded_neighbors) # Apply restoration. df.loc[restore_mask, tie_col] = df.loc[restore_mask, tie_col_orig] @@ -1167,19 +1158,21 @@ def _resolve_group( tiebreaking_priority: Sequence[str], instrument_type_priority: Mapping[str, int] | None, delta_z_threshold: float, + excluded_mask: pd.Series | None = None, ) -> pd.DataFrame: """Resolve ties within a single connected component.""" crd = crd_col out = g[[crd]].copy() out["tie_result_new"] = 0 - star_mask = pd.Series(False, index=g.index) - if "z_flag_homogenized" in g.columns: - zf_series = _to_numeric(g["z_flag_homogenized"]) - star_mask = zf_series.eq(6) - out.loc[star_mask.index[star_mask], "tie_result_new"] = 3 + excluded = ( + pd.Series(False, index=g.index) + if excluded_mask is None + else excluded_mask.reindex(g.index).fillna(False).astype(bool) + ) + out.loc[excluded.index[excluded], "tie_result_new"] = 3 - cand = g[~star_mask].copy() + cand = g[~excluded].copy() if cand.empty: return out[[crd, "tie_result_new"]] @@ -1299,18 +1292,12 @@ def deduplicate_pandas( crd_norm = out[crd_col].astype("string").str.strip() priority_set = set(tiebreaking_priority) - # Reuse the mature isolated-node graph path internally. Value 6 is never - # persisted; it is only an implementation marker for configuration-excluded - # rows while the legacy graph code is being generalized. - zf_series = _to_numeric(out["z_flag_homogenized"]).copy() - zf_series.loc[excluded_mask] = 6.0 - # Pass edge_log down so diagnostics are computed only when requested. nodes_edge, edges_uv, diag = _build_edges_fast( out, crd_col=crd_col, compared_col=compared_col, - zf_series=zf_series, + excluded_mask=excluded_mask, edge_log=edge_log, ) @@ -1366,13 +1353,11 @@ def deduplicate_pandas( # Try to bridge NA rows via neighbor groups from the fast path label map if na_mask.any() and labels_edge.size: pos_na = np.flatnonzero(na_mask) - # Stars never participate in non-star components, even when they point - # to an already-labelled neighbor. Leave them for the singleton path. - if zf_series is not None: - bridge_is_star = np.asarray( - zf_series.iloc[pos_na].eq(6).fillna(False), dtype=bool - ) - pos_na = pos_na[~bridge_is_star] + # Excluded rows never bridge into participating components. + bridge_is_excluded = excluded_mask.iloc[pos_na].fillna(False).to_numpy( + dtype=bool + ) + pos_na = pos_na[~bridge_is_excluded] cmp_str_all = out[compared_col].astype("string").str.strip() cmp_lists = cmp_str_all.iloc[pos_na].str.split(",") sub = pd.DataFrame({"pos": pos_na, "nbr": cmp_lists}).explode( @@ -1396,26 +1381,20 @@ def deduplicate_pandas( if na_mask.any(): pos_na = np.flatnonzero(na_mask) - # Stars within NA. - if zf_series is not None: - is_star_na = np.asarray( - zf_series.iloc[pos_na].eq(6).fillna(False), dtype=bool - ) - else: - is_star_na = np.zeros(pos_na.size, dtype=bool) + is_excluded_na = excluded_mask.iloc[pos_na].fillna(False).to_numpy(dtype=bool) - pos_na_nonstar = pos_na[~is_star_na] + pos_na_participating = pos_na[~is_excluded_na] # Next free ID, compatible with fast-path labels. next_gid = int(labels_edge.max()) + 1 if labels_edge.size else 0 - # NA non-star rows. - if pos_na_nonstar.size: + # Participating rows without a fast-path group. + if pos_na_participating.size: crd_arr_all = crd_norm.to_numpy() cmp_arr_all = out[compared_col].astype("string").str.strip().to_numpy() - crd_arr = crd_arr_all[pos_na_nonstar] - cmp_arr = cmp_arr_all[pos_na_nonstar] + crd_arr = crd_arr_all[pos_na_participating] + cmp_arr = cmp_arr_all[pos_na_participating] sub_df = pd.DataFrame({crd_col: crd_arr, compared_col: cmp_arr}) edges_na = _build_edges_pdf( @@ -1426,14 +1405,14 @@ def deduplicate_pandas( if nodes_na: if edges_na.empty: # No edges: each row becomes a singleton. - gids[pos_na_nonstar] = np.arange( - next_gid, next_gid + len(pos_na_nonstar), dtype=np.int64 + gids[pos_na_participating] = np.arange( + next_gid, next_gid + len(pos_na_participating), dtype=np.int64 ) - next_gid += len(pos_na_nonstar) + next_gid += len(pos_na_participating) else: # With edges: real components. comp_map = _connected_components(nodes_na, edges_na) - gids[pos_na_nonstar] = next_gid + np.fromiter( + gids[pos_na_participating] = next_gid + np.fromiter( (comp_map.get(str(cid), -1) for cid in crd_arr), dtype=np.int64, count=len(crd_arr), @@ -1441,22 +1420,22 @@ def deduplicate_pandas( next_gid += max(comp_map.values()) + 1 if comp_map else 0 else: # No nodes: singleton per row. - gids[pos_na_nonstar] = np.arange( - next_gid, next_gid + len(pos_na_nonstar), dtype=np.int64 + gids[pos_na_participating] = np.arange( + next_gid, next_gid + len(pos_na_participating), dtype=np.int64 ) - next_gid += len(pos_na_nonstar) + next_gid += len(pos_na_participating) # Mark as mapped to avoid later collisions. - na_mask[pos_na_nonstar] = False + na_mask[pos_na_participating] = False - # Stars (each is a singleton). - if is_star_na.any(): - n_star = int(is_star_na.sum()) - gids[pos_na[is_star_na]] = np.arange( - next_gid, next_gid + n_star, dtype=np.int64 + # Excluded objects are isolated singletons. + if is_excluded_na.any(): + n_excluded = int(is_excluded_na.sum()) + gids[pos_na[is_excluded_na]] = np.arange( + next_gid, next_gid + n_excluded, dtype=np.int64 ) - na_mask[pos_na[is_star_na]] = False - next_gid += n_star + na_mask[pos_na[is_excluded_na]] = False + next_gid += n_excluded out["__group__"], out[SPLIT_REFERENCE_COLUMN] = _split_groups_by_reference_radius( out, @@ -1473,21 +1452,16 @@ def deduplicate_pandas( group_sizes = pd.Series(1, index=out.index).groupby(gid).transform("sum") is_singleton = group_sizes.eq(1) - if zf_series is None: - is_star = pd.Series(False, index=out.index) - else: - is_star = zf_series.eq(6) - is_singleton_np = is_singleton.to_numpy(dtype=bool, na_value=False) - is_star_np = is_star.to_numpy(dtype=bool, na_value=False) + is_excluded_np = excluded_mask.to_numpy(dtype=bool, na_value=False) tr = np.zeros(len(out), dtype=np.int8) - tr[is_star_np] = 3 - tr[is_singleton_np & ~is_star_np] = 1 + tr[is_excluded_np] = 3 + tr[is_singleton_np & ~is_excluded_np] = 1 is_multi = ~is_singleton - non_star = ~is_star - survivors = (is_multi & non_star).copy() + participating = ~excluded_mask + survivors = (is_multi & participating).copy() zf_num = _effective_z_flag_score(out) @@ -1677,7 +1651,7 @@ def deduplicate_pandas( out, crd_col=crd_col, compared_col=compared_col, - zf_series=zf_series, + excluded_mask=excluded_mask, tie_col=tie_col, tie_col_orig=tie_col_orig, ) @@ -1908,17 +1882,14 @@ def _dedup_local_with_margin( if max_representative_radius_arcsec is None: # Without radius truncation, every participating edge must remain # inside one canonical component. - semantic_exclusion = pd.Series( - np.nan, index=solved.index, dtype="float64" - ) - semantic_exclusion.loc[ - pd.to_numeric(solved[tie_col], errors="coerce").eq(3.0) - ] = 6.0 + semantic_exclusion = pd.to_numeric( + solved[tie_col], errors="coerce" + ).eq(3.0) edge_nodes, edge_uv, _ = _build_edges_fast( solved, crd_col=crd_col, compared_col=compared_col, - zf_series=semantic_exclusion, + excluded_mask=semantic_exclusion, edge_log=False, ) if edge_uv.size: From 5c8a1d6b2952803c9a946abd5ec3b68af944db50 Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 11:33:16 -0300 Subject: [PATCH 14/16] docs: add CRD_ID duplication incident report and impact analysis --- .../2026-07-01-crd-id-duplication.md | 251 ++++++++++++++++++ .../instrument_diff.png | Bin 0 -> 23122 bytes .../sky_diff.png | Bin 0 -> 34568 bytes .../z_hist_diff.png | Bin 0 -> 23970 bytes .../zflag_diff.png | Bin 0 -> 26139 bytes 5 files changed, 251 insertions(+) create mode 100644 docs/incidents/2026-07-01-crd-id-duplication.md create mode 100644 docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png create mode 100644 docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png create mode 100644 docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png create mode 100644 docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png diff --git a/docs/incidents/2026-07-01-crd-id-duplication.md b/docs/incidents/2026-07-01-crd-id-duplication.md new file mode 100644 index 0000000..09ca784 --- /dev/null +++ b/docs/incidents/2026-07-01-crd-id-duplication.md @@ -0,0 +1,251 @@ +# Incident report: duplicated `CRD_ID` generation and leakage into the clean product + +## Incident timeline + +- issue identified: 2026-07-01 +- correction finalized: 2026-07-06 + +## Scope + +This report summarizes: + +- the original failure mode observed in the pipeline; +- the most plausible code path that generated duplicated `CRD_ID` values; +- why stars could leak into the clean product; +- the correction that was applied; +- the observed impact when comparing a pre-fix clean product (`318`) against a post-fix clean product (`342`). + +The goal is to document both the software issue and the likely scientific impact. + +## Executive summary + +The original issue was not a problem in the star-selection rule itself. The core problem was unstable distributed generation of `CRD_ID` values before the relevant dataframe state had been frozen. In large distributed runs, this could allow the same `CRD_ID` range to be reassigned to different rows. + +Once that happened, identity semantics were broken: + +- different objects could share the same `CRD_ID`; +- the deduplication graph could merge or label them inconsistently; +- a row with stellar semantics (`z_flag_homogenized = 6`) could coexist with a non-stellar row under the same logical identity; +- as a consequence, stars could appear to "leak" into a clean product where they should have been excluded. + +The fix stabilized the dataframe before partition-size measurement, generated `CRD_ID` values from that stabilized state, and added explicit uniqueness validation. + +When comparing the old and new clean products, the impact appears small in global percentage terms and concentrated at low redshift. The comparison did not indicate losses above `z > 1`, which is the most sensitive regime scientifically. + +## Original symptom + +The pipeline was run in a mode where: + +- stars should receive `tie_result = 3`; +- stars should not survive into the final clean product in `concatenate_and_remove_duplicates`; +- clean survivors should instead be the non-stellar winners or hard ties, depending on the configured policy. + +However, the output contained rows with: + +- `z_flag_homogenized = 6`; +- `tie_result = 1`; +- presence in the clean output. + +That combination is inconsistent with the intended semantics. + +Further inspection showed cases where the same `CRD_ID` appeared more than once with conflicting attributes, including different: + +- coordinates; +- redshifts; +- `z_flag_homogenized` values. + +This indicates an identity-level problem rather than a simple mistake in tie labeling. + +## Why stars could leak into the clean product + +Under normal conditions, a given `CRD_ID` should correspond to one object. If that invariant holds, then a stellar object (`z_flag_homogenized = 6`) should remain semantically stellar throughout the deduplication logic and should not survive into the clean product. + +The leakage became possible because duplicated `CRD_ID` values broke that invariant. Once different rows shared the same `CRD_ID`, the pipeline could no longer rely on `CRD_ID` as a clean identity key. That opened the door to inconsistent grouping, labeling, and consolidation. + +In practical terms, a stellar row and a non-stellar row could be treated as if they belonged to the same logical identity, or could survive different labeling paths and then be merged or consolidated incorrectly. + +## Most plausible software cause + +The most plausible source of the duplicated `CRD_ID` values was the distributed ID generator in [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1893). + +The generator works by: + +1. measuring the number of rows in each Dask partition; +2. computing cumulative offsets; +3. assigning IDs of the form `CRD_`. + +The key risk is that this procedure is only safe if the partition contents are stable between: + +1. the moment partition sizes are measured; and +2. the moment IDs are actually assigned. + +If the upstream Dask graph is re-executed non-deterministically between those two steps, the same ID interval can be reused for different rows. + +That is the exact failure mode documented in the code comment now present in `_generate_crd_ids`: + +> "Otherwise a second execution of a non-deterministic upstream graph can reuse an ID range for different rows." + +This explains why the problem was much more likely to appear in large cluster runs: + +- more partitions; +- more workers; +- more opportunities for recomputation; +- more scheduler and worker pressure; +- more reshuffling or re-materialization of upstream state. + +In small runs, the bug could remain latent because the graph happened to re-execute in an effectively stable way. + +## Corrective action + +The fix was applied directly in the `CRD_ID` generation path: + +- the dataframe is now persisted before partition sizes are measured; +- when a distributed client is available, the code waits for that persisted state; +- offsets are computed only from the stabilized dataframe; +- `CRD_ID` values are generated from that stabilized partition layout; +- uniqueness is explicitly validated afterward by `_validate_unique_crd_ids`. + +Relevant code: + +- [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1893) +- [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1950) + +In addition, later work in the deduplication path strengthened: + +- canonical `group_id` assignment; +- boundary-component consistency between main and margin; +- tie-result invariant checks; +- global diagnostics for invalid states. + +Those changes reduce the chance that a future identity or grouping inconsistency could survive silently. + +## Why the issue appeared mainly at large scale + +The bug was almost certainly latent before it was observed at cluster scale. + +Large runs make it easier for this kind of defect to manifest because they increase: + +- the number of partitions; +- the depth and width of the distributed graph; +- memory pressure; +- task recomputation after worker loss or scheduling decisions; +- the chance that two evaluations of the same upstream graph are not operationally identical. + +Therefore, the most likely interpretation is: + +- the defect already existed; +- small or local runs often got lucky; +- large cluster runs exposed it. + +## Product comparison: pre-fix `318` vs post-fix `342` + +Two clean products were compared through spatial crossmatching: + +- product `318`: clean output generated before the fix; +- product `342`: clean output generated after the fix. + +The comparison extracted the objects that were not matched spatially between the two products, i.e.: + +- objects present in `318` but not in `342`; +- objects present in `342` but not in `318`. + +### Total sizes + +- `318`: `6,616,173` rows +- `342`: `6,668,608` rows + +### Unmatched counts + +- `318`-only: `4,156` +- `342`-only: `56,938` + +### Relative fractions + +- `318`-only fraction: `4,156 / 6,616,173 ~= 0.0628%` +- `342`-only fraction: `56,938 / 6,668,608 ~= 0.8538%` + +These are small fractions in global terms. The asymmetry also shows that the fix did not merely remove problematic rows; it also recovered a larger population of valid rows in the post-fix product. + +## Observed properties of the differences + +### Redshift distribution + +The unmatched populations are concentrated at low redshift. The comparison did not indicate losses above `z > 1`. + +This matters because the scientifically more sensitive regime is `z > 1`, where the reference spectroscopic surveys are sparser and each loss would have a larger relative impact. + +In that sense, the comparison is reassuring: the disagreement between the old and new clean products appears concentrated in a regime that is less sensitive for the intended use. + +### Flag distribution + +The unmatched populations are dominated by high-quality homogenized flags, especially: + +- `z_flag_homogenized = 4`; +- to a lesser extent, `z_flag_homogenized = 3`. + +This is consistent with the fix affecting real clean-output membership rather than simply introducing random noise. + +### Instrument type + +The unmatched populations are dominated by: + +- `instrument_type_homogenized = s` + +This indicates that the effect is largely in the spectroscopic population. + +### Sky distribution + +The unmatched objects are not uniformly distributed across the sky. They are concentrated in specific observed regions and footprints, which is more consistent with a localized distributed-processing effect than with a uniform global scientific shift. + +## Figures + +### Redshift histogram of unmatched objects + +![Redshift histogram](./2026-07-01-crd-id-duplication/z_hist_diff.png) + +### `z_flag_homogenized` distribution of unmatched objects + +![z_flag_homogenized distribution](./2026-07-01-crd-id-duplication/zflag_diff.png) + +### `instrument_type_homogenized` distribution of unmatched objects + +![instrument_type_homogenized distribution](./2026-07-01-crd-id-duplication/instrument_diff.png) + +### Sky distribution of unmatched objects + +![Sky distribution](./2026-07-01-crd-id-duplication/sky_diff.png) + +## Scientific interpretation + +The bug is real and it affects object identity, so it cannot be treated as purely operational. In principle, any identity collision can affect: + +- completeness; +- purity; +- local group labeling; +- the presence or absence of rows in the clean product. + +However, the empirical comparison between products `318` and `342` suggests that the overall impact was small in percentage terms: + +- less than `1%` disagreement in either direction; +- much smaller on the `318`-only side; +- concentrated at low redshift; +- no apparent losses above `z > 1`. + +Therefore, the most defensible conclusion is: + +- the old product was not formally clean with respect to this bug; +- the new product is more correct and should be preferred; +- the observed differences do not suggest a large global scientific degradation in the old product; +- the old product likely remains usable for many purposes, with the caveat that the new product has better identity consistency and more reliable clean-selection semantics. + +## Recommended wording + +If a short technical statement is needed: + +> The original issue was caused by unstable distributed generation of `CRD_ID` values before the partitioned dataframe had been frozen. In large runs, that could reuse the same ID range for different rows, breaking object identity and allowing inconsistent clean-product behavior, including stellar leakage. The fix stabilized the dataframe before offset computation and added explicit uniqueness checks. A direct comparison between the old and new clean products shows small global differences, concentrated at low redshift and with no apparent impact above `z > 1`, so the old product does not appear catastrophically compromised, although the new product is more reliable. + +## Status of this report + +This report is stored under: + +- [docs/incidents/2026-07-01-crd-id-duplication.md](/home/luigi/linea/pzserver_combine_redshift_dedup/docs/incidents/2026-07-01-crd-id-duplication.md) diff --git a/docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png new file mode 100644 index 0000000000000000000000000000000000000000..d0787910769ec81f6f8845f8b59f055fdfdc9321 GIT binary patch literal 23122 zcmeIa2UOJO+bufAmT1%@iUmZXVgm&gMw)=pD2__+J&LqJdhaAriAvE?q)8ExUPM5e zfQq1k^nsBsA|PF)_q*RA<$u@ze)pcczO&9b_kPFaNEYQ7B)X{(1%e#q8jw)%f2s+v7Bqukpw2>x+Nj`z>}SHSClvjqDuH z+Za-eEiBCq57-*m7#dpGnpoNmEiIJ7hjx<>ov<-HZ)a+0@uP~Vxgo{Q(B{X3B0tjY zFa3D<;Nc@b9ug5hL=``D@cAEOS`^BU6xz=}sW{#1Z*f&owVz)YX0`MExHadx&)%5R zdIv2ZUMy5p{%NG1HRN;T$VEeX!{wl?G$Ww!@9@>p#xAS0Zi55$HlE|ILvvEbCdtmjQ~mh|j)a!)#sgC* zpVM=gc1zh`GVL#$vA;OCLx=q@zx>e7$^M7iUt@gmC+O#FGDMVTyjdX@kpleu{Iw0~ z#+`kA{5Oh~Y6r-VDED3(rN>7`9y@>jyp8ofX?fY~BklbS>8sYQOYJ$r{zTo!(Vkj* zbGCJk^^44J3JMAsDegHQ9v;-@HRR{dG{20DaBSOV@$p&24xLad_kGzB5j=}e;<1D; zZ&%RDkR{}U|NeL{C0TN6YHBeI1|!5`=|LwaCzZ)w@==O3PkZCFGiT1&eR%mzKtMp9 z9lzJ};^LF#A=KO6-ZHXTU*R*&N4{psT}W@Z6CAuxU0wZ(x3`e;A6oa5lFk`dhN~JG z#e@i(D)ro6OK#rG`6BbSPQHt3pn$HDhDOlS{Cu-q?q}a^-I|ncT-n=OmuTbYnA+oj zk9swIRU0p-q@~42+n;|j*>=wH%k8p zs&L^#R2uj9-`~HodJ|`prlw|ZZLGhWn;SFor{B-K;!{#mirRQU%jA_Gcf|}v_0#9} zB8+Or?icNdP%qU$e&>}W|%(Mr04Y0 z-1JN+tP;P3CBmjBE1uMCZh_zoP9RZGmQNxS}ZZoG-!m=T|A-=}7wZ*85H zYu6KX!h5aiO6&weBbhk#v1BKF}14hwYi48DhPE z{=5C?C&dRbj~*$sG4a-ySY6rnC|6wAFY-Vcbj~4_1i5`=ZmiP;KMP~tJki5 z;yUf(;*u+F-y8k<_30~@FRO^F-4bbaIw5~__~$D#yXC`^9zA*#+S_APu#Uz_rQbO$ zB;*@`@hY=ez*KYa@T9)l=6cS1Eq9F{w}i0Uq*Y!B`G*^u1iQ{(#N!@2Q1q7C|Ls=M zDvO!UK>hj;#Ww;%LK2RcH=fSjcyHI1^d^pmOtU~TH?D&@G7R#jwACB;SN*zlCG-9X z?^XW$_U~sZ%S%3_n$^E%vDDY^k=w7n9!ulp%b$1c-p#jT$MG#&wlJbQ%L2__{_%Zd zuDw}K+QYMfcO@!u3A@=z>#?D>!sdcCTu1iq+I2EfJ+=Po8b5W;=C>2uX|1iTtJbVZ zwrZ;OCnzYGx264hiAV3pH@7?Qx;I@qB02r(fyCh>hu1ihG z5Fu~T^?iz>oQlewnwlEB_7}^O_42iLp7N=BV%}u2@0W!hc~2z^i|Duar`@v$MnVkJ<&?x%004=E`Nu?r3k+RlR=wx=N~^WL_Fs_89gMaeI1z zz9lnGv7`_)pNM^7KiI6fa_v?H#2ZyLwcRwA`Kj))wuQMiMzmwAvy3m7Nc~D4x5z^V zC7WNlIy?8&JUpw~x)aap72$8#zV}s8(TM|E84BGMVOYfbTkKi70RpKhr?K%6#iBiD zJ9w~{0ta*Z1Y|>x$c9Nc*6S}!vsU@1HC$4yo0t7&)hhB%E$_O@L$P`IRZN!&1Uh%;x#>I)1-{PY!)~~p+zwfl9VR5cnyEbJp zjs%^^zIx(FsBCd@v52mxyINgw@jR=nuC6ZV{#-mB>ou+wA0J<$fjbw>PxnPsrx|dI z+f@bXdzfzO)%TpU$9JbqO-;|7JzG0H*0)Pn`@{T~GC}p8F$B2;e)W{i_j8qF<-UxL zGBKToOr0E?lY8XDRcd2o2mB;$KAwKK?g~k^3PKti*GiDsimSwa{%l2}E%p{RimHvx zW5wIIZ$H3wdJxYT7ni-Ls#3Ue^=fa0#E{C3jR%B{-fk`r6_Nk-*I%bU7p==guy&sv ze#N1e7ruZ^dx|nYFZPm zBhO$ybsphrof-Hvb6Q@Wb)AEM^x4vNd$ES}-G;C1J{J_ZmRo%D(7W-Gk=onxQt_s> zG5d}l)xm_-HRsswq7}{!So)fU9d%Y}bDs!vnHbbP*|v-C+g=gL(#IRVUWPBUMZ{waxFQA&aswARNf6h+_%5c@AXz z-94ZkvvQkLWQ|!RLRI1XSSr<|>c_Fc7VU`^#|TXG$2*ll5=XEI-}=H{I~vcadX#tVG}nxxp$uOz#Wy4}BFO`goFi}jiqXi`UVv{w=lsYgW3ywvdc zR0_cv?@O|~|08(!0++*|-QTCMk=DU${~2Ub*WckUt%l^N0&G^BOfFow_p;eLO`ws+ zOtU@G($WYx`*!a>#dYLTT!zQnphJ%}_B;G4vS*J><u%dHuFn@8jp?z2M3eghbXDrqj9p;W5%3n-l&4)FI2K)$Bv6>8vP#gldq)j zN;n`S(OD^B2s5$RKy&TuJhRWY_4M>ir!GtjE?>E_tg=!bOAo=iyPv*d4JX+*aeDbK zbY|SG4;>x$T8l0a~8&19|2Qs%2~fLEI2+2ULTxHVkD6(Z&C zk~2&o6msR^Cw;6`%Pi0N*(sN?I(4e!ppJb{&B;JP{rapn5A7a%fJE}}d4suULq#m_ z^3F|Fde%Stbtws~n*z-|J9-<^V*$IwsC4FOvXyts-HU6R?GkCsv5n;=_#WGb^QhCK zR<{vZUEpBEGi4o}dlmP@b~-vb(rOI+C1Xe`DO{KvvYebE%SeU$M!8X5=IHVhe{YzN z-QC7nL;q}<7?-i$xi&woZ7KXeV2@xP=pSF-DtYJF<{m4cn`?qm>}@M7{9*n2fxDIq zOp zJJ-%M$BbLl>KJa4Wn*hQFrPIuJjKrl4NcUa>s8mvvIxUdsrGcwrQNx+0o%56Pz#A? z)z@EF#mI&-^Bjj(`OkcM;6M8Q_jL)`*&2g`gN<315rYLY7GdJ{XWHA_8PP@F>k<*W z#64!SLqseEXr-@TE8t0LYHRIZaXa3TuzW~1eWXSF;DY`)qS_o6HyP(HZhx-52M(wL zNRbI!zuv$nFd$%`l$74>-LiZlp}N6+gdR_tmF1q2Yx?vy5txqmh^UQxGWX4)emOio zH9bx5t52TpmR=y>ur5_!pG(AoN#1g<`h$Ambbws=QOzF5l^BFb`ag*74r+tgaF;U# z(P2k#r!^LO7V5bUP!H(ls%*T`@$fVPCfNvBoXJSRs#<~J;VBsEye(e<6i=Q!8N6_e z1I1c`6>SuWr!L>s5fE1n<;>;z@vK5J9n(Ofyt<*8g*Khn^?+l5fa1+qDK5w?W91@k zCS`XIaOm-iifZtS}8WPfwC>f26K>6_OH_ciCF zj}Nvavh-YKE%V3jO3m~O85$cW;x?z;O-Zx3D?5MBkLkoS$nZfU&6Pe1(xV4(7_`Dllik!n`ZN zk3aps0>zIiFnQ@1Fjq9NMS^Og8u$uzIz8dSg$pFA_W;l%RRs=zmod@mDnCCzKg}wf ze+ZoLT2x+6PA*(W8U1A;L7wzOdh$%1ZT1 zmy!q$sg~gpbF)$dg!?!+n1T^A*e@b7aW%E(@B;#Y03^AuPNNn-!}vZSweU$_Vrs!* zx{Zy^=-fnGQ_F--aAf3KF>c z0IU(9H_?8w+>X`#(#7dOB0M6RJc1P8-n|NlGfym95NMhq>^XMBU|uEOY375GE&OF%-ikDup6)^H6s&T!3H3&t82J=r@A9Vj7$=> zD#ImJu~|?CCID$Nq7ex9QK_nuuH$#==ykwGRF7Gwj=5R}6ZnMWMMU}9CdWjJyy9wW2$yutFs|Izf)M^?VuFPnmq@4$ z05_4lr4v21GKiNower%*#^KJV5ZefA(A?bIpAxoMNaXDr$Cyy=-LP6Cf%-%ld;v;) zM2)%>T?t^+L_`-6cfYxr3A2g87RJT>f`x-GudToS#~*Q@vyr8GQP6V=o7{ABcc-3z zv8=PRbFRPAW+(7C=i!Tgkie*!ZbV}om6Xi1={!>&bm#=O8zD$gsXjzhYU0jwo3ioc zk=Da40x@s7jzcM!hCIU|MBAaKBYxZ5M-^!-qg*8YgnRe4p8WaePZ(bo*14|>LQAq{ z`YDov&!4{!ej>WSeFmZYjZ^7lm}6@^DqGVMpDn1-Z~a*yGBK$q%bfo;s!0;A~kkHPUV0 z9p*@HD{$`|>@M(Ylci8B)<5m$%;IXbUBi^EuC8|J3{X#)4N!ga<_)K$i(TeQv!2FM zFOfF4^qOA?4t*v@P;SQeJ5Q-hiAA`5dBZ2&U7p(=?rMi>i1}^|8_P?d&*kMSfi-vgdt-=^MwGX|$d^Eg&e7hXQmF2l7b-|DDYX|*fuSi^5tm^IOXPtBM z^2&;@6dFW!ff4LM&5dEW8_KmXMd~rtD+`VTycm36PHN4y^?UBzx}^w)bH%1jn@}nJ zyz$1v7;UoUEx0b5?fLTgv%!-#=Hn%YlFgaqgv3s92+<8#_}BdRDD9m ze_N&ms;;zf=dLYaZQdZ4{=Q;OO|X!0KOMLWESg|q9baNy!sMirZno9)lrCnxGN-6j zB>2D$h|HHSF9T;GI#k3SG7IfGDccs1Kp-ADrM=hj92qB7Qi!x?0ihG}T_^a2g;m_A zM-_ki=~u~YWH$f6fUhK9CW8nXw1uzHgKbEHx_6(#3s z2B}X;1k9gktp!j42#MWw_U{J zbI9F$_wPRkgw|X0@P<*vy?BYi>=S$B?kT9L-9@GAs|7a1eyCMf!f}v^hs*F>@X*b% zDNfmq-$Hz@1DC?FX;Uns^c#96=W{JauEPMC6%aCfpnvlm29!Vu`TlZ%h7y2JK%gEx zUL8p9Rjz^p0&1Aww6KT>K^h?dL0Jd5%V4;?b_(1L8KlH%WlAcG^`>eS*&{z?AmpJO zRYJh6Ju8?W4eY3nMFMW=E^j>OS%+7`;49d)W>V6AXD6rEZEb}*Bu40xw$V!bE9zua#{PJZSP$Z%B$m`M`YxFNHGfrUtdVud+3m&kZ}b`79cjLwq56(U0uy4Kesb@v5;=k_3@4?R^8b+-|Non0^}k<)XpHQ|cP=5#N)mKD5;t@tP6MPeC=lvI(tP@q`9NkJ z2x8`7_JOOXL@uAF7OH{>Q!kAjM|E60WuN*6k&@eSQ zNr>ZbzWD}e*SdA(;y!&s=hYc%$y2v1oO=QgZ!9uj%AaZiqDm{nQf`2j z;77J6O4sFY*56xbCuoh5ML4K*)$ZkjdQan^74#xqGD-4*hG^FQ;>zQOOJG`*ao?D8 z!8>{y_uE6Eq#9Rj_1z+L3e+Q_Nml)DyaK1MX&iHJj`6%}`b4*${jmlq%+qD|KQ?(D3aX0jT} zh?dbqccsKm{d5l(Zck57E55L>M>Mot&7wcn zLrYn4t}=cq9_zCgdLZ@52-|3o^8WoVcZmoSsg>jA$+Z4n!P`moAWbW@dGD(;b(>Qr zhT2k*uR-g)+01|7Kpngp{G;O{ib@O%rl3s9}IL{~@WadQVO3C5RJ>K<$DP>R%qFqH+(Ag8(l_qc_z}F| z8g6kF5MD$&yepu~2X5+t5j3AOr%xXY=aYgg1kLgFyP@9R1h9%fgXUAsvS|6(+4&GJ zVAlH7xo+b8;{AN-o|R(*mnTR?7FqH++^pH6chWV*Y=XN3lR}{|m;4nn|JO<%j3O&t zz?QRt0$Z8@mbJ32?gQD}YuwC!YWpiiR+3Jie>tFEpu-?aF{v;}0kLA;wj`8668S%| zKS!7FZl8t#FzvZ8v_NC1C2Kug+^8qY0tH!psfB~1E7;={H?Z|;A8z&rE(FFM1|td; z&uds|P%o&SI(4;{i^zSH$S1a&Q?xTo?!!!?5Gx;153UA+13!(#yNHO0fBl%gQ~PqF z90k> zeXn>tb)=-EdhyFTI`gSdAxwsV;#Ll2S4fcpR-01-QJi4~+H_#mlOfn zw~8;sKZ#*qCGPB2sj5pOUFBMtbk&^UbyAbZz{uL3QD*;?xod6F7^GkjmMRuUd3k?E zHn#hhpV?z3hyrQx6n7-Dw{i&KBq)&r)|*ohWJkdr1#(Hlc>s=>e0P~yj0WD`u&0sduUChTGGAiEAwAL?9&`1In0)MJ^X-uUAWLYm@beYadwvg-VcYl2I#( z|1}q1dH3ZJb&a@EutEchFB~d?q@E1_L+lZfv?#}P-&=J}BfG5n=9_&YdKwxgo1U;| zXY!%V=9D3*IPQ?&O2)^I&HY)HSM;&VATRM3?T!2){pmnPK>C=AzMRhYzEiUe4#0g??`TYKQ zX=$kvyehmsh$CuNR*$5nKkY#okYkP>NCrr%t6kI06`D8aeYD#1a zFvlr=+ZL}{x3xlt2bK$vGPhCoAL<$!N`)CilZbc!>~c|CZty0y-4y{`l@%3)U%&U) zBB!y3vTZV0%r@qSAAZQ7*YS~ClF|{p{~ho|Vzyl>j)jXaEY2aY+7e>#{*5=%%+ni$ z?LH|OyfsRvF;=YI8VA%#H(5_U=+&6`!Ybde>~8NMdtG~p(p ztF<0qyawR88$koAO5gA+p1?Y~y@TiC{an^YY%H*BlY;uI1OJx-WHlmGlt)PGgUn_T z55xmPs8A?$s<#lUAvt$qX*C%Ewjn zK-ySXScDvPek|@d*w5@@|1@s(R)N(MFmwPHmHH9{1q2{xO6;)|uiTgCv$q|&6hzWI zHdZeg*)&43oPcqqEU(cTAM2ul1b; zi(W|_&R*SKO}`eoE^@@reE-e1z;U$?W6mccqRtq_PDPqITi5t!)Wd?9 zewSuIG-O`F%%jRks{bnp^SNm`VX2M1Gw|;zm>IkcRugZiQs^L6DHOh7#JEKo&UYLxinB~|Hj4o{{5!Z7l!SMii(6ygzRCM-oQ4Zzy`2? z!uI{MU$t}JK}R5x1Ku4fkvup+a+S0;qjXbP&X788L(g;>{d6B&qA~0Nm>yz~fE(^( zN5=ywSv>$}4pi_P;7PEo0_y0hkTL{xvVT#xg(|sbtEd99BhnkOz2Gq{dPdlf|6|L` z+81S|rG!@7waE-FQlrZ7WY|VbLC6r-9~BT{=dr^7kYKg;14vBP%~Jzi)L7u2CxG&5 z$NhGP6lgi3h1;jFA9MI~8m?E_O^ zn>!19PrIjI9kFPR!}P>sM0@L#3B+(1Y>tQP&jdCV=*yLrbRc>aBDF3G{SRR9+>4qC zDEX8x=r^JV0s7P7jUifaicZc2_4OM!#=r|i2Y*UPWXRJOGqngWOA;NFHA3Vf3gWwX zq@q)B3uR}Eg4#7hg$#R#@fAsk8Hw51@7`5ZRtDOcI}|1*^&7jd!&^M7Y(M^_#60GS zF`vww*taZV&Lb*adAMX=^W>d<$`)ceJpY##%0+(&VQ_L1?H5P{)7iP7BIYqu3US7 zqMGodL|%+`_I;kV6rJyZRk3BKiTTh5=A!v*ED)!%H9&Ox;6dIbpi@QI`U@|=CXDAh z%d~e3-&7c-2XM{AN7SpREZEXL=*YP);|<2X1E{sHczZ+X4l-=V9plsl+IZ}BmNB>M zsg5QI8po@rwyv3w_?89H5pYNuoMwxYmR4h>OI0wQ9IVB5TJJyNb$NEBINldE&*9$m z^mJd%F7O$UE%u6tuq?*KB_y;VAg3zY4mRi5up~X_pSC!Iay8GAMr=|~(b*^RV9o#q z8LSKYMFu~7*s*DocjZ-*QxfC`3M4tQ4cQEO<@pKFyEUJx9}sTB9q&Ol9oe&RUz(uI zis8y=xCC1nrbqY(O(}aC0XkT%KhB z3O|IhNmyT(zT_-k8lqr52q9=ByJ*)T>%(h7yx0(A`QRPH7Q!bVXtc7$w`{|;MVZAH zs-c*&_2v~8`uc4V_GthKM5#x${8%fQxWga>`Fcmm!kB^O#3|vB1a7$Q;n__Oqod#H zTK@OpiYL;gHepUF?*j*7T5Wx=4~G*rj=33MH_38)R@x5~7_h4vFLAGbtas!$DCcs| zXBWV;LHwJkh4Z<&(`;ZAS&?%-+W~G#uDS4;=(J07IDl&rX3ovPG z+iZ2wJFqYnu~1DcOz zDl%-ofWp)ind3_V3NZue0(FSN+i>YfER`*C9@G7BP-g*lR8pElSc!*YlX&$Q(FCEa z-+j*NUx7TPr4tqiSX|#UH8pMK0p!3$g#A<7xu(ERo?+c_3Vx(_7Do>sRzk5WXSN8% zb6>Q-wTJfI;xlI7=DCm$jpYAS)sSui#1xSy2R7aS7!?NNfbyv}zMMUF3Y$cAHcF9y zgG`BRuS}y|BXR+j7gu7<0Cd8a5Pnd`Z>zP-?ZtUFLJd+|TT2WI zJ2lQS;MxgyAJ=>s8!PcowbY)AGz%tEBWD<2kbLzw5R3+RA+J3=1Vw~Ygs=R0)KR?b zqQM|NjS)^jPEnT+T)f$ZjiKd6Gd+y)*QDR2)@PqZCV3Z~9voktbb0Eo5PN z&|);`xl1GI;w&%a@v5bFy{bOJhw&X=)ANvS@<&!6F1WDQcl z8h}O|c#^7<69qdGpO}~?95psti<86TKqa1S-NCnYfh+?o8dc{L>IMQ9&2XfwyZo7D;<+ukV zRFkX$cn-nXlD$ftLA9KV0V@U6PONOG8n-+9^9sgu{eRU1)PM>d2&V5^%yhp;*#${J z%~^Tzs=ETN_Mya7bEyJa8Y|(ld~K92_HZBjhrJ9CIk32u`4SEv4G{^%2ItM=q2W1I zrvEqX^jx;Z)*bdFLURr@X4ZmshOZz7#b7+lYZZe8Pf@(IhOV}YLu7?bP!Z**9eR{V z0<9+YRBZL$-akKw>~4R(qrTMh8qWh!kW*^eJT4 ztWRQ3cbx-j24t|(va;I|5&E)O7!@yl#y$~`^mPi|Z{u6my>$Q>4a@QPcuNL*XK^Z; zf7B7w&CTF!V!v?YU~YUYyBmSxRr+r=)c;iWGhVA`3@>U208S?(@n6x8EZG2>P+G$JX~exp*XEfpEzlZrCx9I#DOGc)KR&_}&X z=pp5DVwy-)7ewBT!%BfMDgyy5_3az>VouQ-<4j?pB@bP3N^ppiz85&FAE0~!Q}`kw zSjs*3@#DwB1pCd6v4~=Yq-59s?oi@xG*KbbsY8z-%q3DYsci5^tsLxbCL{vlYQ^rW z!K%9x9&Ry^nvkFbi=I6?BZ-oQP)d48s1E(_rkxi9mHHa*LnYb0#p}4mqi^2)j(E)o z-Qsfy{x=1f2uK>C6(%cr!)gGkj_Bb?vK!E|0oVcmvMd8!g!De7Z1lsZK$;;9p1^5{ z=cLnv5sfeQ!7@OsB``+9-$=AG^mOe-TN~p}SXlD&-&aBe{2A#Yu)px>QymyEF{lLZ3F&%l;-V=Kun>|Rx#uj{NZx%AB^W9S3Xxa~aPetV5mbq@j9t|M zZxRDAY|UYG^a)@WYMZ}%^b;z)#%${caQ7#pltPle1Dff`rFW6<-f5UOXDgzj$owsn z7icgpD(X170$dx~58AF@Uj_GoMiWFkMs#m4T~%G3Rh>ton@$K<1E+fM)$n+Mx`?tD znTJlHDy>H8vOwxGMl>FdrUZ+nvN`_aF*_`iSb|n5h$)bcw(0BWB!PbBOx^Os4(MuRB1{nWa?!VyGz^<776KpYBJB=1 zTk832Q8Z(wRJF8vGh7JhlnWErq)IQ$+WN92K|j|Xg{Ue=+*bA=I5_zmjh$UwN^5J= zOR{0qYi^!YBTJC^;Od+c-{(B)=sElN`QA)d10TXik*1*qSW)5eH;bKvo1c{EKzeI{}t$7-pZ)5iZb@(xsB_KI4E&S(A#)a2Lfk zXu;n3k8EhDp6sHV;{gJg&;KL_D)xcj_jnbI# zQsC*dGKpy(RVpzRqxXs+CXACZ8! z5EVWFaG=NV9zK~}%!Q$#e9mK~mzuWz&t~(`Kby_p{nc##U+NN9uP!}<(*{Dwxd7~3@XP-m z7s`KTjgU+VwE`m5K4g=%-Y0lGXVYpu417pW0jrc=IJrsFX3Gs1pRfuWChfWeR zF+V$fj#QRKL4{VG&Y>Ye7!y!1^?`ViqoP+Htzh|tdWGCa2kU~!^7QZ?_72ESs58r! zLL(rNP@pIh|yq`%S2qHwFMefxq^vL()ktV$WNZOiI zWI=)tiA+HfH)=m+@S&N72?=K)%rIes%kY?UL8n9?Tmejko7CpNFvMB|AL}=(CAP8~ zmd6M9d$Lsf@WSxR68A&EXcM135)KG}7)arX2#R31h&lnTmY0U2@;1DC#IBB1w-@3Q z>^(^Cx6!WrBaM84CM+Ol0T}a0k&GHp-0Hm-5-BeYuR%f*=qB`cIcjz*=;;Yy;NY^> z9cSBF!kp^cd)Ip9QputPiev|jV@6yN6V0~%NV~Y#W5XMA5mEyx zq}?4up+!X(1xd;rNxR}vq=Hv;OfZFE0pIU~odJ;?zf#5_DNn-Xv%1)8iCm(fIW-t) zy6zR;29E}oKy?EACkm)Qd_&S-fd}P>|D6vjN6mR7PYKvt%I z`S^xfLZB144c!bP-b4yaJ(*2U)TZ<@(-=RfXIUwBdu-G&=RB9a77YJrQ-Y0?Q#Msk zN$EoJVeczf;vYX&0&6ap$U;gbES%~HoW`u61sE3PMo4TG@fXn51M z%vWWoXlllTXE4DeGl=P$bW=i7AlZW#JLEJmP(udVMT5$O++GXPgY*ELVJzlTqvfBW z1>}#`oCP8D__tZmKWL7RzkK=f>hVwLl9!jSJ1Ho*ke_{2PvG*toT@&ntF`sv&`AD9G$4NEr=TPcJS1}~NuF@j(@ zsG{)CS%LAZRqnL4s+5VNJ>O zBd#U1AyMKD5Hs;yQ$hLp(CY zD}^9t1b8e~Y3kbp-7ClHyoQLf?p5|#l9kVZ30oHDp_a;%(&p5$hV(7tXdPvUjG#=qd+)NBs_$DWH`{%K zY)6{b$$1yV$rBILKM}POwb@q@-odiVu0Wzhcq6*&J^=waNOo$dNYI<#*IU1MvDZ|t z6dXX5X;`)qrJaQDo;`cau&)Jr*}8SrW~#OQ(iGSl3geI}@87>K*Ma{)5lI;f`m0n! zBN!tZByR$OdMOoqtv~T7J1=z(_O9^CCFjVH;7lqyUW0SVGpP+$?stpYts0M%FgHt> zArwPNxA7eYt;b(O)`Zpe%!L~qCNGUvX6OE1W`lYhWj||kp;3`N%sXHP^$Y@X;ruM2 zk82X?9_<%N1E>|b#cX~?f8wTZKT>+-oSk!!cr^VL}L z6OO+0lm@E;uzhr&*fvSD1Vh63`@N|cvI&gPJs`nZ0eIj6R`WeG*Wlk`d=KcYtnnXa^M zfFimQL4flLenSg=GXx+x4(S2L@sl3Xpv6P5?8I4XXY{#)iv(%mdbvm z%$!{6`w+}g)!w79DdX^o=)tEWa!6rj5Y;##&tRA-k$PC8rYLJe;O_wSNc3iPJ?E1+ zyvoYn>>1Txe)Y<#_kb-uceO35VOY~`A=^`dKYY<7_U!N4&RScPy)KTV*|522um86C zPq2eO)Nin~@)n53)}-pk`Y zdW*C|*S*nMtO=~c-gchxrZr0GWyC@?+lLonnE^YrA~Kiu%4jV&NuI z1dH`DvmC<>;xWZyDMjIx23|^dZ+A-eDH$2F(mVTzDunu*Ko`iBt&i!5S47@~+?=hT+B9^z_PBMPztxYlq<$y*d z(m;!2n~3fYXJsb5Js4!tQAeCV=>U`OzyzF${vt0HBO)9?j&$oPN! z@ff6VB~{fr!%BV{R0Bd?!Ayi^duarCaz2n8)gdKTWkl{W9O-@HVKcVlk_UiWLAt>2 zvIAYl#1l&nxlooDES!Ce^8#?B9XXK-5nj2l4=ww?Fa%2cF{q0+~ z&Lat&BR_x#%o+aAbFddF4}je@C+A(n>cNWi7GBD7{C+v&v=0JW15OM%R1yR=J_zj) zLKk0uH-hfoET!f~l{*Fo%v)Xh4kX5) z7T+x(P?4mYSAPkf4kXba*k^)3^kKZwd?}1&NhN2#z}<=BCFv0I>25KxhHp0?eAfRi zt;lVjXD+EJ^Z$xZ+<-;vezijz(%@HW%Z_x-G1aU7E z!9KV2{}CD%0Rk+h>m>F!RL*wV5x6iBrWk~{n)!g09@%$hH!e85DKGM z=fM4%7to|AC_01|dEcV;gE+~eBq-#3zsYhCpe70;tB=q=vI<4pm)rWaw6!lE%wO2bhs zRncfrFz|s2?0@SbId0Ev0o&;U`332PDS3mpmhD=M1{{r|DufgA$kZZH8u@}=2m%3t zSwTNf3^@{M4z|=Hx%TNZ`ju$>tc^OkIvqzvVV=nuRKp`9LTG}wNyx*YH{~eju!TVC z*1MFW5K<`224h_n+2S@*UhgTfbuF~{cnAkImRZu4O;2QDry^1doM!Htwx{405*2;t zAPr#1h=yZV%$FS4!Y1%BgP)2w>8rpmn;d~dmXO$b$T4j8dg$oHu^w98n(7!`^u5X- zz!5}@j0?r9$S=dE9jUB~>Xhv(>T|=4@6p%S?{gz1EW!zAR+0W|oIYiYL@F_6Kdyro z6arCTVt(F8&al9CNL)!nCwjfFNUMoCBo#j70C+zgIcFqC260Fb5j~YtyDjzK^aTSb zASTHHT-M5pv|P+@#atsse;f?~9;%-~Bd&fQbt#1@C{^_>h9zv_yaBQujyGdOH(BJ# z;zOlx-c;j^9!p|!!&0roJX%$Yxx2a1eNgki+1w%QFlhTMbF9NkR6x|>$ty%AVvRxH zQR*ReNW}tuNBh|R1!!DYfoHK5Q4NP601G-d=X8?{gu_zm<8qp%HFWxQDhpQ_Z!t*P zCLrwH;#}$J+vu04#KO7{bNO`NdlX5VAA7cE3`09c&Pf#Qnl*mD1W1qMY&zs0;=sb? zw>iCELVB2CZ&1WR71$ZQqra+cS`WzyJ)UNHkcjYZPiFE4o-~nT#&9B9*SboaaGn-?`QkJI*QK!Sw%&44Gk_oVNGOZX7NDe=8bPYk8Phml*-2jd+% z%%tA?OBnj3Q|irDP~!BPH?F0aWFX8xM1v8s#aZ;=R?iF-rb0+DJ1K~nCrci;fq>#t zokLF1grr7HgT(KJ4fxTn?+ecu@h}h*0iiUZSP{04EC~c)0arhG4v@K%HPVg~od^p= z(D|I5T?Qd*&?X0;W3d+19>yW$e4+qhsAB4yyy2p8Co7vg5RwnO3t+355@QtEB#{`M zYh(S19RAyHUlBb}UUKXX;DS>_x7~I~A>^l!EC@B}4$0O9nlm{{2yU6~>lG3RaKw*~ z?r@UhF>JXBCrEz zb8OjSn*3DnHM~dmcuFFaLLV!{ag=?d=Z1KQL7#(P4bDAv$7XA9xMJcEj$jJY9p6e^ zI|Okz+g8Hj!(5fSNeF}!HZ{MLlpZ;r1odpD<)^CqpoEz)U(h8TzyqTBr5>IJ(syHq z7Gq-49c@alK;2S@43vuUgCE82iL2idA*b)sE?0;E9+6 zl2dP>doy8}&Y$WEB}PO-804U!I=1tfc{_z7d>Fod#q;OemruyZ$RyI%vzqTg1}s=T xXjYdHfOXcJhPu6X-CxoZxjXsuA8Sx^(l1Tl4fnbvDUi?6PRjk9a{QOy{tuRD9KHYm literal 0 HcmV?d00001 diff --git a/docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png new file mode 100644 index 0000000000000000000000000000000000000000..6400317a72e80e1e8ab9fe78fd76b53d98a8d835 GIT binary patch literal 34568 zcmagG1yogA)HZz7tKJJL*8mg|R6uD&KpHH>LwARCN_Srr0|ewy(%mH`9g1{HBdK%< zN;m&pAosrC_}_1QXAJcS=j^@LT64{Ko@cJt5@Ld<@ksD67|dzx$IAQ(FQ-4XU%~$`SqUguNt^0g*=ktmVDwB(pXxAJYFX&$m{{tY zS}h!_;DZm{Kp%Qwp`&4CU}|zz#^9+A#!AQH>Mi!GhStVcZ{NCo=PENh7xNvi+jq`d zrr*b4u41qc@5$JO&kfqE$gJ$v{#rY7q3y2uF=heFEf1xlx1l%QO*eO{G!O;Pq=p#K zeDEI}z-u=VQVq@&)-5pFkmxWXkj}m@+;LyD#Lz(A97~ZW(R4b5vx5KC>$7j~O;(C5 zIq;E9o?YVA<6GinnF;4}Dj(K*muP^74zHhPF5?jgZ!sfyc!~#af8t$$=|Ha$LTc?J z`|m30ByWHg=74wRj9k89S7*7i zW7*n}qleF6p?Tbk)0Y#wGnDCW-I92rF*Wk>nLtf*U0vI#FMsB2W{KFxj~_ifJ=@!2 z9$2sc{6^QZ5@9<;vm9z}Ztm&jWyv)UUzqt4_cB#B?MBtD3*_WX*v*MB(+ig`r_su$ z*4;Wqb@QhB>dNZEa8_mI-KRsPMZH=lPLohp-RjKM6Ay0DSeqXt;k1bVT40*h5J=Z} z>r{q%MR}dpY)=+b)h)cI6p!9eFRpuEW}j){UXW2xc<27?Sy!RCmS`kT-}F1N2rkYc z0t_ZyDsH^2ohTMT8`eBwrDDMMpo5joc3vglWKbz4CMH#_tXNY^3nrO1X>|2R zf08F5ogq93)x=+zNb2a)^O&m{#V9r-xtQ#1t=gHitn&7L(~68&um0@HR8_{YYS;Z8!D;!3-09~(pFVw( zsc_k8tU>qb$~ThFHytjo+ewqlXom;BKteKOQLbEI(gpuPPhdUWEV_vD)+sUePrlFxx$~O!=K|q%N?%l=li3!7B->YYaD=Kb? zg(nBAIn+Ph2<};_4rMcvNf08e)#9)CRI$ZxGxyGp36^b4g!yO*FJ1B>=U4N; zDc=3+EX6X+*Ua|D(q=9HZcHSvqhVi;&P-2M6lDt5{{{(%S#*uhm9*2B@2K37$NYSJ z1LNEE;i+=2$zZXV)}65CUrR<=Wx7JQJ32b>d2I)atrICNhf3|mM!kp_Zc~MjyR3gB z9M605(OpwLH~Nt`NyuP{EfXu*-+w>hTkX|iEbE3lieM*Tl%4~wi(}YaFzksugN~-; zWa?Z&UoysOhIKMV#hlLp+KGvYME2=v=B_Z1T#NAr#wg{biRNZOGW*X7T1PNuNwJws ziwQ|dVLd9A-W%Uul854&B;y`&{@{1nE;K0U$LGz03+g5a1vV{BwyH~tp`GBeZ6}Sh z-(LT9PHTC7us>Zo$!B9WtD<;0^my6c?iP#&rbBGB>LmgD&h~cGLd9PB=ZgZ)FRW)e znYnG|rdVy}2RL(u0%<2yZ3o9$%dKbSq;W+C9f?wt1ekH}6c4wPX_9d-jvfuaeA8<) z zIu~Dn+3gvu%yjvD*xsf&R#a3J%Io++YuiG%H4+&)EO>t1jyUg1fMHK&#+Nw3U^~X+ zMVn+nHKq1TRufH^c-ICj8L#l${d~fpl=nDEJj%wA5ra_?lk6ehctJp(nW{=sv zbqf0Poi-NFX1JR;OhOJ(_yE(hkt!x^|<%*#;1vgHhSuc1VuBcRElhNqmCu}{{b}vJ@p!@k>=L3R*5{F&3MEeU&d+?b`qvbM`rdMG%&n>OZ z^~Eu&7Egg)J4{B{F2i-F;bFftGF3SBO@iYQ3S&2wd-(7<@^#H&>_%W6=qa_XcskA( z)iS$q!Y-ErKj}%&K6G2P9Ui z`o7f4FAvvV-dP)-O$;-Ux_j3RTx!BuibvGk+$9oCg@qj8TM~OyVlbFOue2UwxNAp` zx@*QI7TthNHYsUo*<4-Gu2k9CoLc+M6%WdM{V(@-cX#m!2zs6o@G0m&{lZkfem~KG z_V#Tl0-m34nMD&M$B!SUVPwqw+!GWZ9!|QkvBBvw{f-IZ6mI!P!7!174_(x^cg$LO zy3HibQT*zPii+UPqL#sJ&UB_o3+%QtF*1s!$!7%_gi=kM!bCo2gQ>r5G4^<8DQYi( zQZ`*d(zJXdkqkT=ohRYne@AzA%A6+UkYFEn%GKh5?=}W8&{_SmLN1`5Rn&R4PtRv1 z+F^MrW}-PvebpZPJ(ya=(PPJC^9^Oe4Yy=7@r&8n6?WK3-ezI39;rNXnw+n+tsJeS zFVEtuNS956LAz1!U~kKObeTI(zb0!`y=Z7?Xva|rFE4k98`b(;+3cy<9p{*z`VzrDV)ca>Kk=!v(mRGYEU{L-b1BP0@yJ;jE z!6ggs`0IzKVK9FbGz(?8#qPeDfuh_X2IWK+y%zoNpI;aM_|U)HC9gjH;<5?#otINn z`hCWC_O~v#jnx~|fkH{#GMM58v^W}0PNfj8*$g|W!E=h>WHOZ4V6FvcAeo4Hm!Xv3 zKG&C7*HN_Mx`{YY|M?9yP}d znC}VdA&ALp-Xv`8V9QL7dUgLCI|-==mW0o#kceI(I#D#t5IIAEA+MPWmg6ic@1Oir z^X5_L(XfeEAxZO5Po8I7lSJ}s+(L1e8_l(uHn1M$Gkmm_ts%$CI#|B zTs-$esWdJCeC|s!hsSJ2-So2Q8%vW$a04M>VQDZhN^HT9LmHROTr%W~w49uZV2Mnt z8x`A&Ue~W*$MYs;)AfKMVQEW)deFXC%+;N)BH-_Yt61-D+hbkx2BIMGfo#%e)rsyD z3F~7P-KI7eD5*y}%+l;jau;eVzg|eXs%vYFAaHvmTAnz5oU`RT29p+G<(zFbErnwE zWp)#okEHO(Fn|9zh~23W0zHb0U{j>I>};kW3xF`8`u(d6Zc)HmlEldN)RoqN4?omau)2Z829Kjx;g>x^4qv)=`by!cb(9cg&zciVdt}|Kk0S=FWq(rZkEF>|d+8klStVJ~cZ8<B)6RT~01&l;;SBnl4yPeeP?%FhoP- ziaKo5D4Qd=8i-RehvkJ{=+ z1h;K>Qj|+-lE-Br$wE*7dv$EIFaK^~Tej3t=L^VW=(xDPHpsw~jJ3~tfi)G@Ey!VK zI+A=L_C$1B2w}03QGCvw*;;}b3fZwTs%?=D&XVEa{-)r=W}9TD{U0(i%T|>V;^R36 zx5s<4wZ0Zv>b;(|B7{uY8McH7A+1asBt@BZb&ppThATc#s%~DI$TE_9Zz5g&YZ+fM-V20;39thV>v2?@BhLtye^wE2Gq4V4D6@lo3&WQMOWM!3i|G@*V=6bMP%ITWRrk&CEo>}dz^l%e7x`JI@ID0lmFUq;w zJUl8YDp5Qt({Z6pzRYp0Tg#VsKX5IM)u+O$pF4Lh{5#2unBHE6ByK|A%j{y12}&SO zu>Mn{Y-hE96RLnbsNIgAJjv~2xx$hzpA`woXA-C71U2LxwnO$wj4FlE9{9wD9f_i$ zJoXkbDN)KFjK7Go4~|WhAH!s69Hq83t=RcsKbuhy3cml=v7<+uAjb{O&d#2w*j>rG zUke$0D9&>LRyD$B2NLuqsQkj9@MJ%B>{v6no)E~$qKX{XOoA7`<{9|+WvX(s+`85F z=GCiPQT7v;EL*}k6De7Ec?Zo^jE#+BdVD-RZ#yneNm4_a4&H>ZY!^b7sH>~%5*RTR znexCavKd`7v-cr^f#Vm2#l@LcdEa79QF}SAA&dzhA4l^s&kTsg#G3=~M2SY{oVWFjMAZ6BknB8-YPA>hH zmZoMCSmI&*AUj1eyFA^INV&ATwO~u)3gu%HnifJjIaZ_2WDRfxgA?Gzm8aN*mM%qd z+qO7?4K!UgDg1cqufLenWYcs%KrZIAP_`ygGN?wK1$9YCR>f|y@WLgG-MK1OSM|!{ z7@j5wO^l z5Mk3%g*e#~VfZE1BOyzc1pH4AWTIAMb!WM9Fm_kE>yD0OySuxynty)?PT+@UBI(>v znKBd<`PY(xV>q2k-2TN2we1niR}}1>ybm zm!HPP@rW}Rj7|%yXat|LlAoX7;?k(Cirlx*W0;Xyr;X52Fu8{M`fN&@YEMGK3m3Gp z#^Pg39_d=OJ}FQnk4FHAH$GqIM^U``)u@r%o1Y<*6N9N6HR>UqY>krNIngsUc`A(R z7sd7K0g;jOEY!gj zf90p;gyAdBdrw?(D*uh0J;_(;U<{kyeOF#nASZupe=gj5Vkr*rv`S5z2Nr7<3pV88 z30Og*!#TV18=xb~t>3vKGe7@x@r2dC_*|Rc0V?+-TZP5c(pE(M!Tzr{L)`QE=%wTF8QEG7}veEv+$#iaz^oj$FDh8-fz{+EAh8WSlVgu2CrzylxFq347;U+?5$s^h+ETAix?ZIH zJ{UG%vFC!70*uOPs_hyT71eaiOEM{Ng7Kv->0k4WxFdr@LKKsQ)1m>oJ$~YZQWy`Q z)As1m+}w>5ua$%?+}3CLVd6b`8lG$xin+xlOR?7BVRakFp_dS&j|n_{s1InE&kn$H zU*BsyGbtJmy4_5;Klhk!^e9k$Ob5!VNtQXRbfzn^I4?D`@2B3UE?uWF6+#u{x%1}{ z;LBz*pn!56nH1cKQx_##`VO||_BHt<4c3&d;+u1rM|=1#J|8Mn3igh?1-Yx9b>b{baIPhf-ot)m3JXgjnh2d&yyVFh}8k z8GGL;epS|i0Y9C8pV5x;Klfy`N!PP%YbjFjVhQhXJ}T)BKl+4uUTKQ#fBAW~N{2+I zK_Tz|aJ$)Fi}G#*CB0h$1&@}_9F7CV{Gb2t5Wmw=w)4=&zk29kLNUG%|D7(z2Ju~! z`J2VmhY$BZe`$=f7%Y7hBzmWyX6b*656ofP+Lo65ekI_B4k;g^68+7_)@EsZrqErq z$(*vR4vZ9&_1|5MZ)^L1TYXpl0&GPsp@La#$J=~A++r}%r2kGswpU`*aU<;wHCNn1 z1oMIYl%9joRoZZFDR5D2e>6frVz6#e>Vr{p!6h2lXk!9|2&|hhhNLmujv% z*0LE&UFJ156>HX`{aQ@#8?5ZW!UCH)(r@s+`67y!5 z6}*PPD8^#@#+88_A03|6Hw2^{(S5mkY38Ff2)0Ey06@*NgSNwglseklk8wa9MMM&l zkCSa4;!N%~+)m#wie* zWDrgW75@*ba1D;ulWd~7Z3_1w4Fu{ZK#SXA>=o4pIKJT;NPjSHTlRA#icq!7L4nlK zyV9ec1%MzH2Y?LaxV0{@KK|t+ZM|k} zSR-;=16O?J3YR|A`eIrDiAc-H1W=~jdv*errw!(bbj#AhB2T00#GpkZgH9;j!6c_o zW?xZ+6t9b+U?BGM=ZAw(cWnY<%%;~u+8n|nC^Z!DX?@*}>c)*!m@mVD!fbHnFe6l_ zFW!m{&pj~a^=Hk?{!w8)d5hI-IXoZU5`sS&Zw#ipcI_V2kW$Z1oDK*NPY3ji%f}uN zSZiP!yh%Bw0PQt|l3hwl>cY8mQa&LE;h++aOt}sh59xU-B-_>V?57@_zai2FhWJ9c-=op^4zK|L|j8m7`0Vd zcLqsXFaM9!`CkX!r~TMOEqt)%(6zhi{g*Zpi|}uhfb$=o6<;i4L3B2(o&0~lFG{@D zVYNh}XyP+@`hRn^xr@2lbi#r@q>_L<0Vr=>b(&ots&1Xl&2!+{qh&TpcJc@4w_MZF zHnuAuExG1e0A0Mr*Q8Ns`p{vWBz1@nYuT~rC zj)?fdN_O#La%E-Z{aV|FAyve$-5CM~jn!&OVjp})vjAJgaCNRvNJt1rsR~;iC}P?B z_uU~YA+n9|Y&@L%pPE40$et}ZdG}BY5J)Ras_gP5h1CLf!PEXb@cU+iQea3o0d}Us zCK}8@9pC-5U@#GC`4XzN_Wu@lQuKj^rJ$TkDdZ#6tgM~PDl+VHSg~-9YhlfaC;|Kh zf9-n?U<2Y28U)C?S$ftA6o1bC1Bg748_9j0PlE(4`MRlYUf!Bnvw z3mglOH!e~@9IgDSC8q<3AVN`{@bEHxSAma7wBMNNfg+O4Vyy0VL-O)rwoqxf0V#xa zywVQp^M~S&^n-G&AggG6(20ZLXY&kJXYkg*$2#?b*%s2Lv#b+Am}CcYyEyV09}cQmH-+C4e*qAnm zHN9r$fRFSZ;`OpELbHrr7Fe;1^B+lVmA326fAS@Q^vHEZ9t*NT2 zy0Ybo3kZk@0Sd^+qXtjt@#8iC#7X-BP#Fkff%=RO)p>UnvBJU~5Nuct zJChg1sx6L;N8C04@U%a%sYwJBAzu-~y}U6QWdzI&l4~3mRzCGjP4t}!vJ#**fua6a z^0^5sWG50DY1{iCju%NJ5CYl@IA9!_XkZ76=Zj|%-U%*?1S02(0B$)t*OUKMC3C^wE?Z~kP3Z4Cx z#VFK48I@`Ge~q<6DF+Lw@@b7Ld7wEBPZ7jAwhqP@U)&IR7yZ}y8v(*db_FkLZZ+B8 zyXU8nT)KhITG<&UU)8xQzkmJeFnod)TWN2MsNzq<$H~9+Ol6fn+FKOckLj5g4-po* zIEqGwLnr-S$>mz(U|2k;Mg#-}jllbiJK?(HAXj&~s~%So6eirjy$?MQJq89#Utixo zN%F?Or4dLHxa=3>5$&m?tJ`2w!2QGV=f{((p2#~mVFne@6E<`p_7UcN2CQTs2HTc= z&T^`1{5}!TJ^N!&Xz^UhE={7CO-YJue|znL)#kDR68J!N;`0M|6l+dEp$x5U?L3(DF{Xdo~!c$nJBfub3 zep%(v%jczuUz%5~S zwV=WP3JAopB5@nG9X`QkGpB&4D&#|i|C{cXb!Wig7H&DN8Y89*X$`dIGE3&$PqPn1 z-;A{T1gm>kEnSjY=df@8&b zu#hU(bN#q*UDPW%w zgQmqjh)I1uNa4RwB|ju{*Vbh9Ht(~v_5xFNAlAO8j=n#95c;lUNea;EDXP0{Bq$Y_ zWPrYp`v*Dffd8=^%>i3UM;*RsEhng|(-aD(#YS&%eXbNWCOz~OKxUcY_&H$087^XBrGp)$t^A7)W$C?q7b zwc|n2KuAoS1Q}+q^lv1FS_ra8kU*sXivwm_$LzB8qW~dwjI~ee}eGJ!27B;kjwTv{WIlC0m)8eIpAgG%5AjcNYYn>VLW%4;qH z@d|jW72tD;E>C{)z;9@2vH1#@8`wt|M?UB2FP($Pipb|tbC+KzGeEf^JnILFw5uJg zDUOei3EPblnFObmI9GfMo7pL%@tf6#qV(rvSGiK&*#PU z#~T7Sx0hP2fobBZ0hZ#hz-Es8KOxd!5Q4ZBPQa}SWgzRxGyC=_>Ef2R)1(U1wu) zT&cpP4B+#Tyo&$oYoxNoN&%H|$GA6|MmjM-@HdVjkgF+5$D~9dj5EFy2Kz5_p?pgW zbh=Pw%Ok7)B`hxKnSsj#hlTj?zk+)0oZ z(bCZgg8|CI4Bo$gzkx$}-^$z?>(qN12ARt~zt^Zj2=CLuVp-tHaY)F@yZy5Y=~?cW z4$VoC`Z8#T=+^I;ztGSJbSdgLCAV$?xa@TE1{@wLB^d9C2 z5pDqf`24Wou`xWjFlLw3VReoMTMTMxos0Gn+8d2RQ2I3f>b-TU&-jDp=zktab$%f_ zr-U=o1M~)i=MLjDh91(`*&vQXSC+1;^q!-Hm|=CEk$x57@$mW=alNbfGq&WE!@p(3 ze?o7CW>(6Qu!7;9y8olVQ7i>GYZ`EOpW2sau1Hx}91 z`x#Jj*s~I3Gr*JZdtYR}e^WeC2H-qdAY>yr%~V+686}##24cAwZM^+AN7w#x#TxLIT4` ze(PI|n9ptXB_e z2&jllO7jB8?q5p-iR$e)pG@CqTtrw4?0eQHl^6h2JI9IlGdIjn8aa{U%kC*2P^brL zTm*q2WQ8<&!?#W$&pK{LMU;bl1B8`i2pXvLhnqphu`j0rOhc*EE;~^qBvzb%M+OAB z)PsEoGd3bl2LfxrIzIH9bRabm$oXo^ksgkgRtP9z##*}Uw}eod965FZ1uUQHLZAha zCJlnAGi28!%$m;;DF&3m50l?Y(kKEG^f*{~~f&>5`X)$-F-bGFODhHku2C zD{2o{6qEhr!`4T2xW zTAJ*)NVeng@}&`|DmS4Rq{6~ZBH^}4M{|LdDli$es=Z=cxE*_N5x#1{p1KBkz^k1h zi%5lTh>mq|v9e_j*1@9z3U!NXYdwYLqpT(a96EY>!Z?66B79Onm(j@d8a%97DrL3H2#B|z6%kD?Ij%7DdsjPtp|o!HhUp=#O8Cja)JcylB8 z^~)z65u9jDurn_F{Z<}fO%AYTmeY87k{YS{ZeCXOYIbz6w*36Q%;2qg=mwxnMml)nOQz4P}Rj97|z z-C;5Z4HH^EsBoonb(=|TX-VNTYR)UN`_lk}$>}rps6kSC&>93>|6KsD$AaMY$3T%Z z)V1mEpt!Ix-WLn1yQF)IEu;|j-9rUH{91T+VvdQ)KDJup|GK4(KfCkPKgVVTfKa@TJk6IS`Y z&Z)lJJ6@yuW%y!J0>0rGFNf*W3nq~t+iSgPbr*wObQ5+&_MiG;)dY-{jFiTf^3xzOeE+aaY4|{W01P} z@e@x%^Fce{D$adcLiE~5vNzLTkOksV8BlUz+rL%4+;8>)t>0(L^bdBwG+_buXAku) zZM3?k0#k|1lwz2lj*D+sWsy7Eb7sAM&ntA;xoaPGay`B?)F*>`wP!3BH*+&hE=tBCZ9~H~y46y(Pt&JFQH(MXolyhAzIO%tYa};3VO_NJ15}J*f>< zooZfUGzh1VX5Eg(@x~Qk2mz2)b=_H+87wI{dUSC&s{pz>5ay$!1#TVXs^vSY%FqBJ zHtf1r2F+{$WSfA8X>55)PlvKIP>HY^_uj&yM6VkJ&yZu#(a@xX=l(Xrh?iLgItl;J zR30K@eChE5RD%N6s%l?DR99dB1yIXnC~&yFZD+ehp<>tZYeg+*Aa@+GQ!zX43}dlH zaxcsX0XSx9aHmMu3B52_94IN4LHub1{efu53Y&Yv(m;lYQhN}=so*U_4rwq_`47Y+ zb7to?4v@Y=HG`N^gw`S{^q3{o-Jq8Hob$NVB3MkZLH%LL$nuRuHhW(jUs`P9KoQsR zEQvXq`6xMsvDqo#X`8cqemR=`wA#;>E_~ijdAYHts#z{IcA#h0eYH+%okaui=GhmReVO}J4Yb11diGu_!-aETrL;%QZ?3=^e0G@igwW3eAR;b zAH*T!y?`N05z#o*~~Lw_xUj2N2h65g56 zXa^?UjB=6!!KoZHBOr_qkrgiq$WiZJ12@}&zlaJ94V_tim)U$}`3Vxyx@?RSL+*=3 z8Gb0cX(kB96(l7k$0I_O16c)VT**5Gc1fF7{S#JQdFG9lB>`$MINFqX7@A zqU4q(6?X5b+`W*V$ExMUQ@!LLt_joCbhcNIwGY=Nf~JG}MQ10$(x|G74>JYl!pRbC zADbQf@#c4%FJx}Y7mwWBS#>0SPCQPqZLt1CP!8k_{aOyr*MxD8d`MHEG!L>RIhY{o z8tBPnbKP}<=wl0o0YFJ4x3ynEIQedIs;VjK?Nl7nSVE^I60QN3HXh;Q-)L?8{?&4n zvb}TparJ@<=1hr6854gwk4Vc*o8!m#tR!z{?rGf{%?&*LBV{_tt6q!L)yFa{uf6Ci zp;f6*#c1K1RF~rUWYLo|>uDY7BG~r!`uq!K*G*mLd`gE4$qX+>?{(ejoDB3gnW|+{2Av;Jw#6R94HbI=(^f>gxYt`nVLHN07zZQIkBI|l`z=-!n7Loc1Kb|}F^QwXgy);EX z+Jo-sMPBU-*bK+vNyUo^$SJzeVG}#E6=>HOO-xB}{Wk*p68wj9mT6P0Q>@ZG0!FeM zY;Iz0u=0VAu|pp&PziX@}$*K)i_)T~!YcaPhmzAvm6E$t*&~!{xkB&I#i{x3mmx3#qPqyQecJOV2B! zrU}$-2U+%s049|}3Dh?wmwvU3;li&ICxZ^P04-v_lr&m86e#2!cb~qeoE~}h;4zbN zO^cYcQR=M9lD(5x(B$$v^0#;1Sv17$ZA~64{(t+z9Qb4P1HW&_*GH-NpR{F-1mfn z6G|}~=7G5I_SC)y@!GR_Y(@2+bbhH>%lakhZ+B-JMFL%Pr0>y%*{IH+ZYFORv%s=v z&~}WU&p58+zaS#tPhnD4VEDqqp($9KK|*~>^yG5cyg?Wqm35Zw5!+!c`5wDLr;51h z*N-B6k6^CGs^kl%31`UUh6gOm(6oP}RNZ(jSfcCvG%z6?$73#~8~!3e zVdbf@zexgRO;P-PQ|B&oDf65FDF#)gCxQ0HB)t>w`Sd%)QkQ*PZ+}l%d`PnsFEcHA zPlU!llRcpmkEX*hvN_&-xW=^N+i(q;H2Y3L#(a@r{xrQ;a`Nxrlz$!VyD1xX1A z)1=hRL{V~zEBjjaxAN$DNzvuE6EacBIP?c;c7@1lCSZl62nu8fv&QmFW#ztnqP65q zFRzuKPShF7Ng=jc!Hl?bMu?4%k54-eZ>in2cX25PvDW${iVsT>?*TeSl!V*{z|y@V z@0s@ftlK-?7gMj!p-wZ@#*X?dp?j6fY5lz#BvvRQ?QLf5eI0RVx$Gw^B?aZ?bEWZL zzkbm&GMcEQ9Td%)eg@&M+fM05NNPfpJ@{(=ooR7d)u&JA&ytg8?XCOnwYeY1CsAp& zw`CxO+7Xl~%g}WAM?fpETF~^`4K+Pu^oAcrK*^1<<#$6@KbvAmfKN~Z}^N$`IS?R{k3W1i3EA=D&{x3O!nx<;d_P0$l0cNXN1T7NunMp zTG|S@kh|fZPvicRjZIDO;q_7fOHzuS)^R&Mat)D#47ol2*PMq!g^5XLp7i857U(zU zUEAF%iM(TNHlf#UkQ6W_8fV!bKXz`#CP05J8{17Hz5f%>JCZGqb*So{Q(J&^?i1`4j@v z$K3?M=2q7}t`(OS7}gwDV;g}(3v6Rtdp_CY$A7-MprjyqtvT|NY?gtdVo3S= zcYNsVYh)}D0b0h}VcVV}hsb1Kz6U5?+uvlz`3Z0J zdeyT31ELC0xo^3XyAHjg$0FS!nd2NYLMRY70-ACpxI-S82~JMmsbSH9f$ePWyQZU!L}8u2ycpz<2MGA^s_iZ>=rJ0|#(< znR0*qFw6`mZsw@jeLIfQYbczd^Tz?YWG@iQJJL4XdNU%b?M=q3RI&5JbOki_&Fc1Q zYLOteN&)N5_ftK%#$9?4I=n=l4vT~07a2B?-KsJXlM!`*=J>g6kCLZBl}gLR)PwqB z-QABOmGROM=W7(0s|rm;9ooj5-kp0a*!E1}uR@U=D<<|T3&Y|k?xri{ zCq|8N#OD~kq%^27@b|HLt3=pH?k*8Y-zRcOoQG=ZUOWDghaA4{t zsfYD>?Yl6(_SDnP@ue5iMi#G6=gPo>rNs5qavKJJf5=MYHuYrR_{ty2iGta8G_wz< zLm#l=&ykNn<)U*~2<25`K!rUk$iFH1Y=(34VNmVp7j!NJKq0E?&MSeZsw5DO-2b$| z0F=?UPf<`u9E6d(OKg@CA^<)(E4_bmFJn=CXGZ>XWL-{|?Ms8uVkpwt4B9DCvoX|b z-RL&ZD5r!Zvrz9cW&Ylyul@o)umgHii~{%iD|W(G;f#=EZn<5UX*!92$ABUf&^33Dy!)&-C$f_2?UtBlA8n1|mh!B4L?o91BEk%`rlhjHrc#{MfldQy*WIP4 zkl)fG4sz*_>K(M6-F48And1V#LH#tS(G0|?NMV6m;@5^oz#DbJ36nGoxPlA9w6H$kBLx5w=I6naiImq5F zf6<1oN4rdT6JT+WS@q291FsNGeBp-BtD8z-VWIZ!m4JIT#;iQXAj)7$%xA zKF@=^2mS2N>nsliFPIgnz~UuC+q&5oYv$h_Yi@5}19Au2HtmZw0zK9LX3^DV4}-Kb za(CgJoL{jpp@$;sKd{s{B4hQhr=_L}p^i~RWbf{5(SZ{CZ7j#=1@#{YfedFZ9sxYvl}U@BXd ziX$(8Q#BAD1>y{?yBi>j$V82%A6IVq9j^$S@Qe;Wh>M{~z?P%}U+C?k0_vu_ltHb( zMr+5-I7n2)ABt%^1NjunVIGShIl$79s|jV%^GQujJ=)UP*eDIU03dduF)|Zm8Qok5 zyYlMX^{dy>=`PTBDGB`+r>fcjjx~(Pb=t51fJGE=8c=6+12xgHUf%ufCI7a@UCLmM zi;H$VND%^u8-P83cW)293V>GY*e|tfD{)vo`VW&psyk+6Av%M-p2H1A`P$V#P&404fHrU^KYbK$oQtKT;}p zDni}0(5Nj3nx{^X)d2l5>18+rh`Tj}d35Rn47nZ7gGol*BjgL6aIO`2n(G$VXHnNT zOut3qqtZS3;olB$rt zD`x~FrXA4TVeijo=z)kH&HD|}P)e)=9EJm$4=&!NCKjL`cLnmZp@jfo>)=S}y6*d7 z1M@KhLd!1!G|~V-ZJp>j?{{9&nueAZ?MsyqGiK&l1c(zY6b1`hD^J8Px}8fR$xgX3HzOjV&HxnhbK zeD8?UMbxjCh>MCMe;d;MuDZh+1V4-g{i;TpR1_2xPa<`o-UeNeuZ)sXDijTdz|sR5 zb@y3h-23+uuCRT8P3Hp{$pWx5n|-Y=M1L1r@rf4qZovcVT&&^rg_`{&4?z0mUtocf zp>^3+Kg?I-Vt!;5Ycp4)Gn`_O5F2}a&kvp;yamEi=fMp8`Au^HmT08#?Q8m|+)0_1 zNIhsXj{uL!9T*sh*wfRIpHT-q+7=)UR|2pZIbU>&4l-~60u9Z!LRqiR0%F*8#dg5F z#S&P^nY|74C7fR^v zv?(c*3+jin(^@!N+a;tEMZSo;ES}#ZQN-o@4a4bCaUjgmhqInerzT-<=D>j&fsv6z z(U!iT8;0#o%gwEfP8c)(P&twf${qta0F29FIoVb6J#d$e-^`##lzZy8Jr+J;yPoW> zr1zGRk`jKdeEE(sinu%D=nZ%)rs7?69#U4>YN#t<6~LqGHoZeChm?;i564Iphn<$Z zGb?r-pc_UJWJc`}-zO`WSXiWS`MnD8h|n#`0X(<_+C%7|Ie`6gwGM(zr6PbURzFAX z?fn30q!hR9ybRbcVj@vT7V4OV&9&2KZf|d24Cm%BZ5D082?(IgKuYZ+<473x4FdE|@pBEs>5;6X4BIiGt)6r^$E}P>HhC!VNN_SToiQcW*X}{C1VI z^SYR`&1;UI7VtkAr2ElfOHYDfnB6lbt3S1v>G2}T{PHyu&~`s~d1hre64Gu;12~-w zr5jerDMKo<`NPyTEl$uENe1>0Z3Dryio_OVn&W%+z|)~F(lcZ{$wrw)k8sQ2_l@8P zGn4~8n`IRlZjyKd#kL$MC*g3zPv`2Z`J4V((z=fC7tJZI?XgvIOYSGvF^r~ax(o5< z<`vhqqai+@qr0EV$knZ=vrw$E<@XswZa5j{0Dqell-#p^r2RWWVn}o=-HrgY915*# zfQg@*pICQ>$Z#*q|8@E;BOlkklkUF<{(6lZOpozVxO+u+$+dba)LeJ=lAm=9&Fjfe zhnKsLy5`X#T_9Cqw&(dL%@Lwc{yl4mFql`5(_cg0>NfG&gm(R!qGaC820GRVoTf6g zjKL8V-lwX54(nY!abEGjOTRjW+RM-(W*}PtZ2gD(tj1yN{d{ge_)cyoJ@m}Uxs3>2 zVo12K#B2H({hAt*qHiYmMJO`RsR8Jwps3QBjG9)|$tLFg8A zfM(vV5?fQ?;7$%yXxHNq8W+?Pk;30aVj0MPG%oWoX82PTU=p&kZhJh#PlvUxKmF%> zs3etNC*#$2wuv>Wc9(^-8%|Np<};au6QxRPU2D8!mgQ^8naurrmI7-;4YN_n9|Q+* z@dYPpjGE3X+qJ&H&cy1)U+d17NU^1sJ9jfdUV)!`Kq^+4E9iQHeWMt6y3_Mx_3bo9 z8lC-Qd~&m8=NT_aN495&OQlfXYoN`>tFUMwWv?nWoij+A%O$$K5J6V{;^^H|myE^? zyL?Q#y+<=BO9Z(h?QSs9@vF_q8afl}IPfP`JiV46Bb+pvK-49t=%M;Gka~roVCF4l zNx>Og?i9JHp(kyATY&6N@aX6($+J!={_$U1c>q*I2Y_1 zHlN7Rl=?7TitPn;zBVxYIt>>6#gRIz@{6;5*U}sdLNr_BB+`AD7;u0PeeYYFDv^| zrrgdC!;qqiQ_7dWus;3=|FoJkt4t=dGF_8Bm-FCVtW={YxqiCe>|VHW%ggAt z(wnNzNnG`>^PY8@r=yEg77m5WI;BJ&x)-Z3 z!ginifOLoTouXD1b_wyZ{32oKXOmWKr#$JKJ0stxkmB;xs@S;)mp(=kxReD467cnJ zq-)h+uvU_qOkL|?y0=hVUd<+vKcQ2JSx|+bf!7?$#u6T?Wf|UfT~*V?@tEJ?rj66H zV#W=mVNZ0Nzfl~lRM1*G9-^eD<5=e2DE;z8vh=D}W{^ZniyB+QR91?pI{rNFKp_F+ zuSRfjuhhll7J3mKf|OIXaK169B3cJ~K4&$2k(x#@wjOlu*7y{}%HCnfCEnbESH~4O z>gDS@a*Iw6508ivE*jdzNN=LJU+pr(NicR+rsc(8zclQ%=_fd~TyqN!UP%6ji+eCt z9)=`-nl13GL!t+=6_W&IuwvMUZi)DZRZ5M~J|(9EmSZEch_)R9iAilNF(0C{Pt--f z5LaSvje&I&thKVL=>dAuf+m*rHvqdT4}XjS&QU;p)C>Gduj#SUq; zU4?hT7xYUL^BsEGu-E9$N{fH4oTHzao{St`^g8$UgU2B^qgu2NcdrI2hwoB2T5@}&*WKbGDlzdaA|^jvUhLr5Y?VT8FJ7@ z>I*nyROl4~C93vZ@sUtWu13PSHzR`9%6z!d?(^IMwLWD6dnzxR2Y;8)+M-^dh>{tTjal=~`h@DatmfH$r5;-%B-#SI-*qqUJ8IqL9rR>_wewFCOp-R+ z<{R*NE=`uB6K%HIcqM(U!dp92sZE?3sS5VSTKZIk3nz~Y350@JB4orOADHOerESuj zjTau-U3Mm@C@EeD_`Txb3yt`#P3rcPT?-8dY+4+hzUR`hOKIZ`FnkufDV|{tH2Q7# z(|uc0TugsSYkF}QHrdEZXuq4IpU7Y$j}cw~6RN5-%hwyJc7ykQdjos0`zoKNRikQg zN((d|mU{LzcP(;`e^f*Pd$~uNc|zJ4BKTv)oXh3KQGv?A3;A^AO{{PFdW}atJLD1` zij39KoH%-en-+t;QAnUW>aT0X_)3)f=p|g0o(@~jytPfA0Ty?acWOL-B5pUH3md8T z^3olw4JRQJ!}(n++2%)eR$6+DW}L&&tyiLZE=Y1=-JG3`Ux-LWzhs&o>|6ru)yZJFxKognz-(;vZbty=XyoR0#QU~s{ReDzlf?)ul zb$n2Ff;CY=uElV32QXuC zmnZWQ({4~wY|*Z*o6nFYEq2a|8%~8L52KsGmYrw@u z+5P%xp*N`Mlu*7uk!LdVHjXMQk5%)wdpY+bD|yO}u9h(zmsS1gQzAnXew2KL+=Bac zjhDpEu%@Tvjo|N=cB?jO$jID7MJBItk%9E;@4CYp`^a>@W6=Otrn%W+k%77$Yr4IK zB+t~pcY9h<+ahq$@2l;j(G_YkIMN!*r<0+AtLiAXfhR8N7>bE3vVFapCS~epuqrz# zG+?(cn?|0Tcdo^G2_C1NFc`+Ed0SFHd1}A;lS9hf97=CYSJO3JmuX_uyX%Q| zg~qK+*hqX*l>Uo`1X0dzUUs)Hvlq`K@vpu;Xqeq|sCizk%Vx?mm7R&>a_7Cxw<~Eg zi{4NkF~`Nl%-Xl_tQ1Vz+H|A)S^S9$N?aja>X^sQl=B~~ZDgFkWHwA=xK-bkBrFSH za!wXDu|+y5`5$19JE(+n3e5~TP?angFl9!R&@KqNWX{V?p)-vZ?!iekHks^W{{$DgL_cWy9^s_+f`eUa`&i5til$_kB}SiI zR0{5{8)%|R5V#<)C-}F8)NWm6x=egf(Jy}?mAIgRki93)hq}{0wA1CJJ+8Z}Fx0Is zqf&{RCqw`6o%6V4y~OH@h zdjI`8rHyHGDgSOwZIRIcLlv{gn4w?8x4Pr;8SmaBbwJ%ei*eW zC`sdkQ}=4#{<@cFrQDdrk$yWaXtc{@*)k+Z$NWZzt(!A9$L*?uzSd8kl9Uu#f^Ht7 zJQEghP^1?1cg%m`V|Tfx?2*CG$wx4I+Z5a2!w`b$dBid-$}8y6-^!@Q zxnDO8QKM$Na{vs8GFrdykD;Bjb~4_j&Ug`f-7oCDRp6CY`b|))r)UDHILZo1UIU00 zh|GM;!(J=b9uPAkS2uX&1_v9dtalB)sfa44ntQ(q4{}bm4-CKL5m4os$u}RRc%~0x$*!6^+$hIU(o^u$R;?op{qqF`eUTU_K32NR_3s&Vy476RWThDcSRx3@@PheiHQu_6# z;OmQuB@a%fT3x>U3{9-j0zo7nS-TJgpda{JZ4U(S#D*F=y+CrOh4Vk2I=-4)D~+Voc6|xcW7r!VMtRmO zfn@Kx9+9KZsPFR(NVF;M%wRqoH|_|MP~xFYpJVsinC_)GJ3nwJV|K*-(ZEz$ibff> z92cEUs}>_-{$ApOwG}V+H5X~>`&T7y$qC>!5xO3>(4L5)!}er_TsWa)o?OlTxLS#A z=2k~^kN`o4yYfa8hduVhQMJd7YSD!=BBDVpVSWfEsAj7KHI{ten||wD||-1lSH~*&Y{jeP#Uef(2!-UD_`4dp)ty# zSyLNnkJm$=e->sMGSJt*YRS9yDd88Ju%#mqHDVnjnR~d#&?jX4eF!)(!fK5_acTo0ThtnTAonN|kQ>gl$f*Hiw2_b@9 z%U>OH8$v@21TQpW{HhCN=d3D~#-xnSYw$mSPc_91fFb%WfD_%#SZh!VeFwzf&Z+(G zl)9=BdM=;gw1{$oLmH!K-~tU|S;=1hx%A1q73J5^JnZfzq%r^U!nF?(-!QDV)8>HQw7nC&L>^)YFgMl&vIIL8 zh}qizdn3ENrgVL(_U+S9$aNW`-pZl9ww~XPvC_NS?SO$QP{X)D)&ja-EvSF*2evNI zBBye0Twf&An^2E68&veGl1!B=;dk3Wi{+~2$ z7p=E-70|oy|NKPBb&HAC%=A*DvwvRCGQ9H}ixqUKN4{sofdqStS=UZ_*7UdM*IRvk zw$M|CFYjZ}kAB`r*zJ5ed<(gQ%wpkkKZ?q7i*6Y(@6RF`{k4^L?nJt-ALvPx4RhW?P*c2X(AT1^dbWwND{?`k!w){P||d?f}jQhPzH4+OHdkS=+@Mch&yi zyXtGrSwX2Q(D^C(B;1GfW)8K=yzdzC z6^h;YiSngB64RI#7zbVj@h&II_$86PzwDEAS~PEkMEtvb-#! zs2X>_Q{1;&_kT}zPg-P0u0IlQ+Wjx|=D#U26JSaLWIm|z5)nH9Da0X*hSitA+`b6n zqEi=5unoW!(9qRI#JMJYXPC-@gVkoZpu1O0*TTLFCij8I7LkwKTyg_)h~J380>8Q$ zsWc;V02tgO@f`>7=fSE$4GAESO-!MnP77>7z;rd~&jF1OF%YCdtnXl+KfG~RDVp^j zn-6zjwE-yvf!+Nt%Ym*pIH{xV17ztIpoZZ=WmA{{=3G2s=@*&3W(69zc0$kt=$M}57jfz zV78euGcz;&3dGfTAAflRIwUp~fh;&oXmt$2=o#iYJj1NxwrU$e0C4f;&G`OT_!OoeL|EkK@% z;!#r`8vql~_h3bpn~*-8wPrD)`?e;eL?H*q%lfzAy^^l{b*sp~GLVgxo8jLOMgECn z1Kt%>+sH4brX~Nx4l&mB`)l_Z-u7;wDZKdnh646qQGP(8o2|{UjP8GXearvjeK#(O zLT)Shm1~zxzVWZICEX?J@5v$gulrc>OpjqV`62ujgl{vgJt^f91vBK?0Hv@NMBSGc z;t(JPVB+GvFAEAJ1_uT>fNu&s_;8Sgfv?X^ChK?YAGMMTW`HwSYHn_B2hTo)5CI6v z2(n^=NQ9{za{^`j%diWY;XI!KQ9>fPZ8QQqn0Y8hNa~Cmj0Lz-*i_%Hee@EU15PG@WRMDX%osj`&usKx$KV z-4eie*MLC-FgYklH$|6-mg`~i^>6PAp>*(Aj0RwvA2{4ac}zV4Y~O2R@Bl)3l}JVj z+W_#IW^iDH*m(hTP|}bTJAyhR2&J^o1{_b|y@D1$oP}tyc6&^#1_uQpUnXSFhj5i3 zPe(!yE0{_sX+h{iNkydzP)&wb9 z1gcY_AW04&f2HJsr)igjlF4Bgz!T(+`hP~tYf3Ub=7HpBaE$d&ac&YjR}9x=^Cujg zQfK0B8i=Ol@M}y<($;*heWJeo!G-I4GK%0=$+?;RjQ&9UsZZ&+ay$+oC!=IG7za4XAnO z4f8-(8zvI!AYlO;a26ys=ShnI&T0bW`RW+pyG9^WQf+WzCZ9yIeDgMD3D0a0-Vb>N zEi99~jE9#I`4#z80LhpL{hF42cP2KYtO{U3I80;GQA!I2g?H^Y(@gI8@$|@Zptzt6 zfYN|4|62N6* zaj!h4u!#CxCo%1d1m8dnYKz>Jqf#C^AV;=aLHiKT(1~=Unul3Y9oS`{^T!Jng#nau=xbz0yc>knOb{@I4SD*t2VyJd zIVNHV;Q5057m;}f@O+m0nOExpD1ZC%m|ap>RK-+sQdDRty?XI^=rq9Zehw~2gAQlD zbiF(--(#9eN^7^{bXsIeTn^z&A_039#)aVLMs>S-EX;fO31P=2#k@%%Y28u>MBG)Jz-H0`g;In*R@2|5>BJ9`3)Kr?-W zzIgs2!WBYa4cSBizuC6R)*x_t))N%5vbwi}r>E*iv2Sg2$eaxdcyvAjLNyMAQxGJU za#*mO*d2IBq}67n=5*apK;8y565tQ=OPT=$?L5iOzqMo7!_zLMwLr{j+;lIeQ`u2p zF$VjRs<3lDUAMNI?kzjwef^a9v)K*UE>iuHTA4Rp?%gh34|^B5nDpbZTh}NFkjE?p zfDqmy$sG{&nV`M13?`kekP)Ii$h`a$rqBxx4Ei2dflbpKcaa%=J9T>H(F~G%M}C@! zC4zME041JBh>aL<%8$SwrXTR|42?c)S8%G=a{|I1TkneH=pxAE2?7EN`H`ZN6`zArHbNREl@)kxK@!2`whwmcTSqBj0JP30z%J-4DNEx~J7(!AL^`gGDM*(QUco z*XBCna*gc)F^-gjK;IaI?k?3B*d;jic!n}k@M!J$=tMD4o2 zIDKL(Efn8p>;;bFeNThlc#?8_xnf3IwiYIrk_Y zu%tMX{=C*^y98I`X;|4d^{-{{@$r|ClpnCspeAaR=C}UZnva&!Vz^?y4x{~fU-9U+ z`EjvOop~Og=2MRky7$8-@)e`bHMz}NUC!@&w09B8i&;N)kyts6G$lK1b}M{N(b^%#f;|` z7P?-hx`G!Qa;ia;eFPXrf<$mo{3hA0T_+gEX6PDE53$k}*)cl5{Pt4%=~s3-si!86 zH7upwA0C{7JujCMroyXo@nUF~ZK@EP$7oxc-f3bj7Nq>#=Ejo&)!bS9^b$|OHn_PU zV*zIuIC1Nc#A@YXk0HG^Pf*;$y)7mc&zGK@DPCPHb_0|yAc7UWT{HF~>Cb|g?4{c|iI_)u_zk>jmR|Ee8nMHXMQD(YYC?~r-M z2UBrosMAb_`)F|K+54(CKaco-;&V{#_a0~Mc>A+Xp7>dT<*?v`*LiIkW}|hr7y*Th zG`7RbFH{29U+H|SELFm%$#Nbh&UDK=wDrxX2S}^eiPdfQu=_gKKRoR@eecWDoSx}r zkRk;6q%U8~iA4e0bI(hvUr9fCK>#r*|Bhv*>i+g9%&R$x7L4^!;DOW5m>wUn=>jJ?}RU1pM@|J)=0=48nT#PT+W8ctbMM5u&AhLW2mrswq1{{ zp`jt&jxD3GFtm(}Y@x69<`C#0nu0T2&aYoiL=q_nYQTt&v1|}3V=S8b>F4VkRyTY) zs_q&tDIuZKLn>++0M!=4X?kGSX9?||UtkHSV`rC|Y?$8yB9*rLm?Xjy(vO=ZwJRzp zT!%&C1kx$_w+DS(3yrvWcrw97u+(F0c6K)8)hl#0!h(Va&%oBEKSUF-C}q?&SuhNd zCLK0+B?MFx5ERP+f5Lh=N0Tx#8f$*o7c(DD{N>U;?ec8Li!VI4Zf}3_pyn9|`l~(q z>%H&1R}br2Sb%g{`tU3FuOKXx3>w9wLg|xG=QecO=0K)bT~(D1wRgw1ZQEprnWa1| z_w3n|4J}AzEH(`&+ocPOi`OQ6PSg=+!=t^wK-m9YVtw@J(X4$^^BsQOhdWSRQzIDHXa)91Lz6D7BR-|=Q5qQ-n;SRw z_&q82WKJE2&(Pl9UhnJ#`wH@3zJnaKe|@?Gr2qZX)n2s_o}F`DFZd|bxmlvfX-eBU zvV+d4dt@PAlTBojR>>B!{J-~3K{6YQ#&V-;bnJNj#>nBU%Ijxaue5`KUq# z-4XDh2YwkkupAv6EMXJx??-dJ!Qo*9mKL*6~1O0%la~LV7upj<2byS$iW1 zMz8~+=7rD7DzWzttWOviUxt3~8F6v8G9eL>L=cI9PKvdy?W-BLuC6YWi9m&r2o%fe z;c0o_$;D;FdPQ_(WR-MHnLZq=1+8)(0|O#a_lZc#H>yV`F0E;qV6r#Jy-m={=TKR^On5 z^j41ub*ctGgNCy5Q&e|B8)_NECh@7M`~1YfZU)AiO>J!q*((`!Qi)KFRE;?$CMT;R z&M6p1oWNS&xB(e%OY{iLI2#{88{M1E*^JWrl7T@|QIXU#v>V!?Muqebsd3c*zQP}$ zoh=MaqQ_E+DJfd|`Vsd(UD2>XW*pU_CtIP7b1n)@m>+@J8R1zgQ4h>D8rokXeY$h`rf7A3o?Xd(J!q zx+*tz0U{Py#0$aIK3X<}4}9&~H4VG%0vWf;B|PRYq3)6tNOXJK!~6oYnQ$0rkH-(g zB<%EK@~2|ZK6mZ`(oBTOgtFVr$7b+_Q}p!oys8KpSJS>LhS}F&z&Mfq7uOgx_aFIj za&Xjt{CF2RNAnsLh>il!5;0iNvM{ojl#n<=6#-3s;Xvq3_rHPtW&q0x=#A>8rthFf z665c`*HwZEmi*(RKvafws}TakafvzTp(jE?1Pi#5G)IJgHNwdEOIZOV#;{_Fk{PDWPk2m}Ik(ef%4S~5Oxa~Ym~nU@i{ zYX!*lcR)u%!RDf8a=tSOweA;yp$qGUe!=7B%}bERP5t)JOEW1|FNTgG|MFVo`M|d8 zW2W0u_!tbDE)oi)qAo%*n8jyYl+@jA1dj%bgYH&xeHe5&;W|_ac@z{^oFLYm_4ok> z1dT&bjvZoR3Yqy?E6o>bQ#JJKw+FJ=n}K8uc%W-Q0(JH0;HMi?9|rl@L#ByvsNMz! y1~~PBz*7Sp3kq-Jf5Yj#70Y^O>i-vaSGMYNNk89K&h3qU9xH!AE=Bgr{r>{&c!eea literal 0 HcmV?d00001 diff --git a/docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png new file mode 100644 index 0000000000000000000000000000000000000000..1429e4ab7a292153979ace11664055fce6f54bc6 GIT binary patch literal 23970 zcmd_S2UJw)wl#`vRuFB)fC}1T0whQj5L8Sh5(J7QK}3*@5+!2*17avdlqgE9B8Z42 z2}%nsNDwH3fT&0+k`g8V^TY0QPoH=0c=x_{-x&WJ{~2erg1dI@{e`vWnscu2x|+%n zjwRen*x1-OC`S*fv$4&kva$WLXwd@v#_z%JC-KKThXWMNMfjh`qBGa<^E$^vI*uB4 zW{xh#_NHuSZS8DKcR85Yo0{4>nAKDH%~Q8By`pTegvGY@6692ls1U2FRimIF%)f7OCyq=R>2> zhTd+EtpEMx=tp-}T}tEuMMcw8oqG=->VB4P3s63DCK|VUyx#8RiT=C^+cH}A@3%I} zirarZIye=!$Kf8P<=wmEo}oP8V`fbAoB@q=*&Q?S` z(-D+RaqrK~IM9?oJuJL++qROrI_;>aC~mn6PqTK(%Nt5HI|jDDd#GU}@8tWCYzgTA ztvT}JpBql5xaBi+xAxVk+9keE)bU@vOWdT0J>8=EFzq>pfP-87sFCMz5T<++MaF$3 z(Kc_)tQn8QEqeN3Off}8Ma#=8PyWZJxxRjWmTzuwWp_llKX>R**~-VK?_}8a`t@t; zk7rFy6O@95uo#k@6L$}`muXm9TE;YPQ;kTBkKf@rHC(6t?rlVcBJJ?awNYmyJ!?Js zySuH@JehPV_a6J`s;Vl-Mw^VcBJ8~lX~o;xUoGUV*>YR%@lK7{Kbo7TZe%gH%`UKl zWiHK5!DSO2k=}+&R|{6YuxWf3%fDxjF3s`Bx6k~y)111scIiCbN5+D$OVxXvjL4Jb z92e~d&G5bMTDRrL<3GCh?;V?%u=?`*b*|Gwxl<{orT1m*zh2|(DwArdd1_qv+@d;m ziHXvE!Gv>R_-Oa1SC_K~%VhRjp2p;g;zPHid(;YvHmhX+#P4+HFhXe%C6QXquKdQljX{c8M4x~6A}}TcXf4< z0mmXrwrn|Rn{`M)JGmx7BkoMPUQk3tGL81@a8E;8iRU)ukm!K*QkKIkre2P7lBCUt zXoDPQ>biWB;$_4AEk=rd>?fR^Ul@9gwH+(wAMA>Yq}w#Uz!H7BPT~BQD`dvZvzqx{vJGxs%dS91ku;!@oYK>2u*#gdS5dv$cLa7e39|JMhOA)`#?NuCo!zMOc|}T1k34ckfnrAMIn_ z+o>@$TxVp}ob5!5=JooqpLQ=Kgx-*LmVtS7>xv1kd2)o4Y%t5(#D}hlMpG>9^pMcp z+0e))`{vD?onm4dajKEEbSgvKx?aTQG$W#m!A0ZSgj8Y4OaE zpXd>LQ(UZlt)iqv-NGWF?50kdSs8!RwARTR{{C&PttMiUb|2-qR;@BGxwkXjD^Drk zikWJ1#do*UG+ayf!^3&j1lAC^-16*bC#r1y~izA3uIXdJtLD-H9vxIM!=1I`VOk zwyv&({`?t38kv5|GYkFP+}z=AqZz#;w|UYVMzJipR`1Z6!g+jvr5C4)!-@ zsCHCDN`)qnNQyTgaoRSkZ8fE8Y_P-W$2a3QzyDs}ljG8FI5j<*pThL$*{|WWOF()@ zo59Y`CCY1%)^^Ego>R&9ax)1SN!Me~TE`TQGxWL@a)Ff)=Qz=uAs{28=j-cxOjWfm zRyp+E-Mhv`VKPp~@b&~aY^5w*w&4!np~HvyvF_@OymLQ1w|H{m$>FXyYm79H9=-O_ zd&V>AWQGc_=g^6m4x10p6Z|-NlPQ7QcJlEZT(M$Bh>Vk^Z&BilCb#Y+vgb#78{?1( zj?o(48H5D#kqz>p-`SvR@Q;#`s)4prr{aw+JH*8`Tf8QcwjF=)=*pFOgX?tX%$dV2 zVRphto;W0pIcB(UwacI9u-aSSsm~~6GFMgnjJVJTco+lUQE-YzjFiFE&7UMlL(c(7N7VOmij=WHJ zxom%pScTjc7Z;bo+F}w*k+qK@1D!vg`s>P-=~%+^7cLAfMs&(xX4thXNz)#Fb-HF$ zIV9ld{{0t!nL9TqGSbMH^49MZLTG3xF;;ZqbRJcIbl{5Qwu zFSL4@bBNZ#>|0fNn0v}IprWDDZKNjwV`0_Tltqiazw0DF@YBwnJ898jVH>t?-)@D; zBOtD;I!<6C<2Yg;@R z<)%yXm*T~?&!0DM$EHpDC=|*#vMK#0FP5|5YX63zqg6v5-OqOH zdR$+x<9qe0_8zT=u7mAepRe*Ji%j>VRq)HoHhLgec|U8AcOS7<4ws>-%8x~p@Ip2h zAgNJM#~uWn@sVBu|2L00)=PH!3+Jg}{xHBo0PAAOfS2RN3e!<@7cMggAS=NT^?T2_ zJGdJgM-c?mot&cr6cZyqGwFc&i#N*mA~nal(NC8-EkC|=WqRC@IP>V(*w~mK-xdXh z{3s1k_CgWd=4MZi^^?TJMCWDF27aQtYi3@C*u+bu8)`_;XR+Mk6S|{fIpQ)i>wks?XdYEpb_UH&*TXHroLPJmbwp>hc+=WQE8yF;DxVui0B|sM0T0!4soqN<~rl%y%{IT5h?AdNCrew#&-H~472_IhAFi@N> zX*4SjOSp4qxl>=0ZiT|kOO$_QHr=&JfvAC16{ez{dp{%`@?TAszSrb;;6=fmJMX?Z zeHRF9PkU?Y7AYwy&*`y_pgspy&CXp$dHTpF!EqgL?5YS_LQX;jL9h=1PrCX0-<&!t z_hT~J^YEV~oSbz)_gbbZ5WOJu{BKofs#xor*zYuS>Qo8<{$4j5ATK-8p1 z=A&@_;i>Ppy=$7X93F@YljclWA%pClV2-`h@U_*yT!AY?u|9j zE(t0CDV~Cm7>Fkck+h*pSk-nS5hPb8PP4T0s{mjblRcJ~W+(nBGDR37N4ia;X<)LP_6x&jp zs4Ir?-{ZAEXorx{{oA)yeZQY?ElL1DVbvrA@_79)o|@|St|J3iDx7~bTUyCwdeSv_ zphSSwrP)s1u}DcLTfAp-C2gB@y0TEW4I$rl7IG?RVs&9G<0mIQnlH2l*t@u7Tn`tw zpyiFT1d!bNwyzmF;ZBlCp2trlaq7$WW@4N3jsD%V!~|9 zjx1Tal$r1CEn)LPlcAj=X=lPX?%Jwm1RRShJwl+!sk>I0=J@T?e%~us;t>pKXT%)V ztX>^kS*f;huiNvI)}h`;ZCzd6@v)A`w$GoV9zEIw+y;2B#Wk>mgM*4EXC@oS0&iI1 zug8uZt3o74CN{Qd&SvThd7{K!vTRu`vYBc#D(?34WU)MbDl(z4T()N7IMq~1R8MgA z-jR9}n$UD*bZji5`7ux&EbMDvE-G#~o5rwFux{QLT%7CFQ!g6&;N?p>>~;YW5w(jK zF9Opf+i&zjK7O=~y-*G1KHpA2vKEiQ0Fq*~H8t;%q7SR;!NjUntK#ii@&*P5suuEk zsC6Y)#sGkAQB_q14k+7z$j5M|I3D5S|AJ*3bWE*XZQCL}Pn@{h)YN2cwy3nlDEr1I@KpDLf?}>* zxgyZ+Ai?sCC9@|v>fP6zqwP69^uF1pWfvtx!s@U8>P?f z14N=l-@CWU^T+3_2zuJsKLp3lnY+LoLAPD}Gq@u%Fs5hD9Hk+%CxE6Q*A9EU!Kj>@ zuex{mo^Ae=Gp30Fk`ARZUOI4W!VEV$aNvOT$oHWkP0%Z@3`3HpMtT{KAOE)NcCpB?-#9jtca{_V;EgAsc z5=$a`xb`I34jW~i&6_gMb4#Clv`4J&h0RGqmEc0ARZ4EGj}M#j!`+8SDPWsBT+3gP zapC)EL~0d8s1MmrjG|i`8Q7mYfoA-S%`tWe-@(%@GZWrqaUmi-LdC6zZ=4JkNff43e%a1S`4#y4>ogha)vtC{VdreGt_JS-0bK=k(QyNCVqMuwn;pD-Tn|^ zClL*N6ca4PRthK>>cN9ov=ViqP@C*jP%uKQt~zO#8}l*OU5N%%7u(rso@A77I5j?8 zJv}uMboXu)CT^#w=n2~MbDs}v-MY1ltXSunotT9Uj(Sx$8Qyp99a3jXRZ{J5aZ7_v zKz1c`VnM+^g09xBTgSrVHy79A{qiOUlqkFHwg|d(R&GwWZg^HTWq9H9{O~;vhp`av z<>%*P4X6olvL_$^dvQSHv$L~H($7*yB+|fcK$aFi{re*22)PEc7O55wjn<-o>Ts8j zJCU4*JKrnfTO~83unl(^zSQBA`{onvFLqKR_V{~%NW%XS9E6XEwR(DZ5CQ^)3=32W z&`Q?$Xn)GkkfQ^$pgSes5R5?->b=M5Cu6st=y{1R|FJq@h0})#m`f-qwoWzY!pH7Lj z`>nUF?Dra#MIcnRd->h3=r*s>XXktbrm}}(M>yqwoQfH{ckkWg-k*{BoxiQU{X;HCdfW@EB^~<=ev;$QKZhxJ8-hGpBcxB* z?|0`_XKHEBHppg1`D6af#C1#IpSyh zz-Z&MvxnZWHlT=0c{y<7@slSExl!B?l-n@Url$EGxj@NSyEp184}I9ga9p3kII&&~)GtMQQYZ-I8N%_ZUw>6M zj-_YMKg8YB(fP;SEqeK0xrD)V5CjiV_2yx$QorZBvNDOY1P@U`%OlN%Hqe`z^qu-Y zHs9QH{>wpMhX^^>XRih8V*tWzBHH!K%gV|urY~Ei>ah4CS+_a_>-y*{daSul4fCgd zvHtMb4ePd)5B6OO5q0h7-#=yLB)2I92Tf%}eP{{pmPw@xV%vTG{F(5gWvPF%ae$a& z#Tfo{*Fm#1V^&p+zuexvHm6anrGu9d+XlK2CBlb1&zyCGqy|z@Xt_xWSf3A-o5ZM? zTK_e|wcuZ^V?B4R_E%R|zjyzBEXp2mXZ*W%DWfLjmT`>d+ldSch+TJO@k)%&V^rW4 zpmG5crL^Z-*CmIO{k>w#;ZBg6L5}As!^k*K2MCkqqsQE&%tS%sgnHmue{~o}1sIk< zMC>;b0>RLw>pGnQnL(iGoz04xDDhDMS^%F(c8?1K3!4<(JkgTpxs`%=Rx>#^*ggG) zq^m<5nQRhjF(K3Xl1DWNGr_;|g^u7cWo0UO@Sur_A3w}dD1}Jb83u`#g-BFCGcDaP ztL~)7d%U`c%r!PXE_qJU2O`aH6zzPG1xq7v+eaTWb8-ZB?D)guj6ZmdB*A^B*7^zQ zXKq4fSWQ2-`KK16sTqywM%rs=nsb)(rx3#!;EJ>w)P3M*{D)zSGB6A-%$P|R?Ls8j{_ei&YH?Z)9-<2y*kLN8-sK|D6b2A)fr|y}(o%kXbmr?GS zU)}dGnEb#UV?TaWYG&OjQrGS%1Fyu!#yw~Dx_=Vk`Zcq%XNd-Z7Uyg)4KmOAE0lv6 zgp!kR{1rd?bU#kW*3#jdPfgwRNZHd2nhsm$st~%>p*zvr1N_R#&*%0CYLbt#Eqg>X zA2zmym9szl%>BvV7R>oCav%SDeO!)PW{z}LHO)+izh(6-0h1d$7jGxMK7?Ph;?6y7 zX3^#02So7v6lP9W%kq(=KeLcY@-JIW#sJquWjBtS>ChkMl0iw{_1_$%eV4_l3R6RY z)1aVLH*elNFgU1zXpEpQ$maJOfPhfdw+F8}!#*B*gwZH{#s1*qn4xnQGso7Z-i~ncNR(%a z-i1ggrJb#)C94lY2Epj4_8?7)n_%8fBtDIRw6(d|`c;ulGZ`5=iZg>(_7G zPzOLPdjK@ZFCtQD9nY6=GQ$?+thfmo$$}HZ@*2ZH8|JSr&F5C|dKuxy+G*30XMkWt zs`kG88LzglUyss$`|UUD5&vt~DxtLUQTFWFgPKA8+_`hU4+f?j?Cn?a@ECrcMA$Za z*M=%ZQ(gUZo-m*?A6OvWq*`vn^SzLMceA(B?X#X+B_~yZT2yp-j`RmT6>R6A6u|IiT-zIcB zK`<}7^?0Pb0gxU;#0y{}T-GJ|gG1dyRNN>)*ZFOhlA74IZcC|^U68$lgVomC+0yOl z^lcKpgHQe>Z3%Z|WgnYun4MSH#>!0~unugbomegEI^dsUv{mcY?dkpr|=IJ_qcARIe6T znnbeGYX3uk_9HWDSyQ7GC1!n=(L%Re1L!A|KM%z-z-0a8lUX5Q3CUm_skV&KSQeIk zEogD;qP&SdSx6K|X?p3FF@OAVlt=|wAT%hS>QJ0?A&uSRbLy%hDmAcuxF7qv-zX*J z<+Z@=C;|m3*by}B({#@X=Vr%hWm+^S6Lq9hvsR})SX8d=?%HczFk0V!yPu{bC{Z3K zl>ACpKDg-J-H_L=e>WK@88BqMbEODA*1#YHA5Bm;;DE zH7J|l+OR8n2c*oy!$VjrY&<4ITX>i5vx8tJ%-%sRkpNTd5QNHQq2=`C*sx0~`8xfG zHzMp$q5{upAvqGM130tVBaUB6N*kMCMAf_e=}}kOrLB8)m12^y)Sp;tYDn+>m!)py zlsiY^Jh|Qft<&UxuO##T>kRoX5ATD@S6xE`kj2^%q;9)&p$JiVVJ!e|qJtIssa5}1 zsh3)a7~O5$v!Fzl`%EkyNd=ot(#UGQ~`hP4hInCxJPaY0?brkar3HIOuAh19U zq(d4Lr08WkCLn4*KmBG2igbqhaGHKa*3eSQ5Y13|J(gKL3u5qDECItU8_z_k_@7URPy-br}Po!hq` zLCGg!0(qASWt=!;o*WJ!Or#^|!q5l!6lj?f$WSNpy{4YSP%(eOf)I`%K0>>Jx%zO1(zs;@i5FP*9B-c2XvU8Ny_Qike@6QYP$9Rk*kqi?wD?ju{>C9Mm%M3m=8( zpP){dd7kc}^SAVfqI@NyGZ6#tLC%HqNChj75a^I${zpngzJLvT)S>8|YRfMYJX#g# z*IUo-zcEb*54W#5m65qhC3)MQ`d}gq{qFM7^$3VaYy+KezJ=SV%{HE5f*$!xadq9QoDljtF|$rG z*uLMvyJ3CPVwc=9nXia{7FWk?VUwL;XeIIAEdKrX&d&Ew(j!(S?@v61dei4{N%GYh zKTraYcoIaWVXPA1+7CNC3O7B8YLyy)7E1o+kYGabcH1_H!YJ z*;hCmR}xEy_Za4e_7+BKM@Mm?5BUfEK>EumVQFa{(A>0WP|q;ZXpa%QLymLrb0`&V zN*l4~|C2$$m(-qRX6>&V{nS;=^ozG=~j@cVO_NZF9#FE8Lwb|RK~I{eTiuYXJ23)CcPJm(&Ysl z>MAO?zI^$@M``gK4JYCxOrSgDK_R%Yh;R&k_^UHX}7 zJUnY$vd(?JLKp5N^Daw$Iw>1AGmXnb^!vpm1A}t@^!~myAses2Z43YI&!>UCl0EK^{HqzCp%J z?vF42SO7#mcKZB!K7|H^zypt5*#prhDl31^teLh3c%qnKBQISj1llmXAY2pUU`K~` z-Pr7^)<~P!L#5#taX)_i$S#?E6z+{Vc^XvhSk*`&pe#s@4(~+RA@ux*92Nc)sf2*R45>oMCyZ% z({Qumf37NM9FB?zzb{^>KHP1W$`QU9U#PDMP7cwV7;_-{RSy^MTd@ZCTvp@ z7rvb$sGR6~&}pIXZKl92GAs61DG4XI^nXOctOTn}WFDgL(VlzHctKn3Ljex)3E0->amSXR+`J57KTzOII|k@%t$N#R4O#XLlK zO8^nkc#~Jg@hE5|;)L4M-rGwgNOJ^Z8nGjHfya{+#fMn+sZZ7sCpH;bQPKA|yWp^Z zHLq-K<~1Kl^`~?%U2YL6^3p5m`thd`A&BSV(&fug{KbjYH~rZMBR98f=l+jL*m9cn z_X7jBP;Oki_H^CtpWN2FweZA-u1g;HCtm9Z;kAyN{$F`5Eq9?56);@u{7!6MlTmt_ zTV2^9;k$&K`qaB8je?^v25P%Exv5i?-)H`vC($rqltv*EU=G~Ay;|s0x+?>ED_PLU z#1=8B1oRkOpG5p2#uEf+0bbrn{#L=&ja~B#Mfh(T^bI^H`I6!ohHBFapWrG|jgU*g z-Kg%<;-4Xt@$rl$v7BbDa3U|Kn5t@V>FoJeuEZ4m^Ap`@AAP7W!3*Nmsk?` z;9LAlYqHJm+D55DY)JsQnD(816ND*rFoBynSy>VnR+Q=r-nSe)e_t*pq(fc1?$hze zvY2E|4RMvfSBC0!ta7`$LT{^Xp2INzU@yKmOGGe%&0Vxuvi69(up%l}BE#94q5sk1zQ798S?J$#!diMUI)ZLs6x zlVc$T9Dn%mAy<)@m{_>|LrXFy@xso{(^9Yokg`o-^4n$LBoh8%Z_x*dS_4&(C3A#a zc}@>_kJz<9AEpEwx>11rz$qL$v{OQY3TT+{PqR{3Kv8bH@Le#j4yv0z#UfVSa~QvE+1W z%@O%?Tya~=;=i2z>6~{{CYQwv;Dub^3$=ph%_)4T{&j@q8wkq_juUAu-oSZRf+2KW z>eU;Zs2Y#*ie;PqDe5Aw5Z3r&1$#>t~`?8L}N#S&V7F0^hB2|;R zH%T|`FbuMY95VM2^VHZ5ngp%DA z8%pd-q%#NuP^)d^8ACRf(=GDFpPu8HXwe|+Ne%cHx-RaEs@Zf!x@_W(G^;{<6NlKSg8m=r*z5`v3=e__oDsR#YCY+An*We59xl zTSY)X3Afz6o}U8uULBmpHuv=w(j~?x{K|0+Tf7rlWvu~HB#v+uje{ zI+}YlhEYgA@a9EiAn^smL5BoIi$)#XmEt``{75H8b!XvmARWlRDJu;O40~(7z@p<3 za#;tOh@Goq?`Vy(Gr8rbuXJlV0*7X#EBSHX6B71UuGIMJ$1{Mv-s4@{J-@$M12HCu zX_?+2iSN);{i!F+&{=T!pqd=lGIhKXAR!cxF=@Kg| zOQzP7Rr@LAl=fA45Tj9^2K&wKTp<(fU{V?8!mWTEMjl99hJ+E|J&v^^`>;+?Wd;M8 zmW{x}S4XiYmwf6$L_}lSbF(sT#88U5@6YZ3NzYVULjqoF(s|?d>Ec3`^hR>Mxjs0K zAY>Qx&tYEwf~X;Ml*Gg({mni`+#gmYOl_G#}klRQ@V4wKdQ|6b(A2 z&Pj;eyRs9z8dj&wYF3NMAnY?ajb9%-b^~*1 zl}GXe{7s{uRMe|8RzVe0@CS;w?`PjwrmBcCv2jIn3_v&O?m zx@l$>pwkHdMIsI1`H0C7$>=LucDqgoj-t5W-@DfUa&?G^@g-JONI3{2R0Qa>O>Qv6I*Y|*1*q4h-c!ECI((CbhID- zlbepTZK1B#M8?odJ?+c)mzz#WI5!$#SO8WrK;LSN({ryWHwp;{#7#%yH25#z(5lzI zONZYd;U$K;Tl%}n#MM8wily_D8WJ!?t97ar;EZ^1QOuDKL@8b+4Xb&c6Ct9AfGDr8 zk^~GZp9R7`@!TsPhcXZ8JswU+;=?CSL(=u~;DLFO<39*`f7H?ZWpUBb2cg@yPry`6 zp)oD7$&$W1)vg9^13dlu#aWkqy0k$dlx}&WC?FfXFn?O|lixmM=@~3T*o8tVxs|Kf_r~n^5+@A31oCSe0fwN5JUpl$Opsa2(bBx|KwLe)$uq zqijn?$xU6W0D~K1k&DwB-GvD45IKQ6nwq8?g$~}Yt0QJG$>a7aCI8uj>gedSNg`Ub zO!c`)_BX>a1uHNng*6TafOQ2Ufo9wU)Rk7G*A&DtA**fkJ!1A3iqH)_pFaDE(Gf$; z;p>|nEGRAlKifmJUnXaprRyQgpxsdb?p5xJd@$l|LO#Dbf(xSbl!cXz@T}wttRKHXy@bS)SNPTd|Gf# zS;n=RhgCNM>Z5kdy<~Ap?My63tPOg1Uv!#`lhKY$urDafY>j~&1r^?Y7DzmpAB$sT`C4+ZejwM}= z&fK@rV)OF)#tqAk{NeUnO!?7O3H@blZZ|p`dAgA=H?YXL00)hFuSR^}+?&qpW~q9& zs!pwfcZI@vn2g77LND@84fd>=59U zG)+xoHY~de4<8kB1++4h!134zpiir3Idh?4B@k`Z=qEr6M)dE$A0(Dxbo$hx%HB#L zE>jS(F=#5Z!ry7-b_w!;indY-WcT`fT(pi3b|HJOwuZ(NnBPJ9>c3wZA$|S;_#b*o zN^ogg6#%HFt}byMQ)u^vjgo6?wFns#5D-9UZx9c^M4s46L8c7D(iBG;Nq_Oe61}28 z6`UT2gZOgM&4e&WITc)hG6$@WleVS+`F8zVg_*B4hmeew1hf(>(L~9B8f26|Gu03E z`@uB5LLI(7`z-wdx7-zJCwwuPsP?`|gflj(KZ#yJ(^$I5-h^hd`=je`>7mI^~P zUY!Y3l^X6hqVxPp7VJ�c;3S@XjMXZG`NE7caE1sZ(tp&8Gc`OzU{u+N~$GSlqcs zXCtTp@3B&&A@m;d1E6mho=rRlm%6Gl5!{B*h*;X|+XMPYf@0YCae^!+|NNhg$R!M@eIHPOO)HG%z6q(s-G zijM(ht-vlh?pl(kPw{S6 zMpm<)<=x_3KpZ>AKefh^=Pj6bV7B;qurGd=k6XEjir>y&PX0W;>8IUn@s(NRFLF_B z_ARz8r;~sEKjXujd!o=eqe)uozqGcJ#yBMFF2wi!4xs5!enMm-(@Cse0D=1*Jme!+ z3mYaP6I-IaF=UTJY~J*E_oF@70>qeK+D0pFQ%ARENJz-|lmr>}wbHM7=yV2yv=fm| zI0Olteh@}d1Zui@?fAyS@MePV>?X#dtY%w4a^+pSc3s?qmR;BYDuMI~85}O}sV8Rk zPR-rj{S;iB4w@%VGD=F0fsCeOF=LY&d*+gv>)9pLq^hH6G`pY#7EGyQkV+E$6?CwI{1lRY0TBsR-AQ^^L5fnb`ALG5 zl|4xV!Z(94izLHrZ0o{_*JRDr)7nNgU2_d7>d+GT5|WcuI5;>$`+LUt1C0?kJZR6}2CXXLy7d=sb}Qb@@nD z`AFe&a2yL#1}3KP|6JGWo+bUGr6M;~(tp?Ay>Rc!#r-E;_8xM4k}{&G`e&GrJIb<( z=N9Ck-j*A(qZ$jnX=>E!E!veCEgs0I(nfcyL;#3^`0b+{0vij52+goe{A`ze&2Vt zE&wC>o>&%!nc54Bnr$s~V8NAGh?XA|39F=$@3dfUIvl`_R+o7zQ(oBpnDW}QVHw;Q{6DVpx4+IeNGgw7 zzv5e-)4vIu%hv6VZgeo_q{7yLPJXfy0cCJ@1COKpKk{AnXTSA)l%F||)GNCvSYlOZ z7to|qV^D7>(OBKxPUvSM#W6XtWDAxVI0M!Zh4H(5q`mllT{p+P6s0}kDtp4qRazk{ z-Z^xlKR4N{6suf#vH4=OsK^yqWoL+w3hm zE2HeCH-KN!1ULStl@g~-pgHt7gpI}T!lmpY?h6^vQ8f@&35*VGhaRV-3ZN5pp=GOz zcs9{yN2wyT4dlkBgR>56`IRGTq)D5iZfIy&`2&XfM8fCMo_mbhlRWqNwXdbMwI&P# z&OP-}pz;eQ3Dk!U6SCL+h2F`N5^&Zd)C)*TN{V#gIFxGIR;6EZ=3y^ufc)rgSwl>* zr}m|O9mp>|k2eq!0s(eD!vTJ|$0Q*`F*}{%Kty)9H9p=W76Z1uujj{+KS>TCfBqjC z)UTywvlZAY)3ju><%31E^e%`ewqYzbWh0d-s++iMzwMj49KSfY2(+M(5dN!$8^z;NQ%@kw0|nv#$Tn&k8XEbt zCtb14RjOBH0#b=X@dIg`Tsmk~oA@5~LSkcq%UKoZlyn~{DJh*o6IT?X>2s9nh-fFt z@gNYAYd(JDfoG$91GBc4{_L4L{K)jABg^vol*kOotTxwpUCXZ>>V7!7oVS(d&Kv!=#zJsr8*L*D39W>%Ii z?d#}7+vvo^c$=XpLUo~t4D(lqw;LX;NgjQk6Y*R2d+mv+xIa^Y2H&uRCdUdcIyW~r z9|i3D{#N{F91Xm`I1uv;`Y{|ss=xp6^IHJBw#)c5a?iy*1~`ZbMHxdTfb;{w#h3eR zO*QAFSN==G6hMpP?|h9Wnm0}jEAiYe4*6oeO~2pb-z1>!HyK+D_#YpjF8r6Y6Feag zpiGYUntdOPEjc>_=43ZLul(;5Q;2}1-}oCR^_{PG&XT}RUyJ&gu-L5 zFX&>%tpS6$)>@%Db8>RpB!abMQTb^9eutmg;#ENys&c>x$mdcb9e-wtcLZSgwp_)IjFwgb#rciI3z3&{V{c2(j?qg9q{O z36Wm6i_L7s0qf&(`m&V=qT^Nhtgm-m6I%=X2RXE49-4uTLtyEK z(>nb>4f`e$?tOc}su9x1(#D3I;sfC^xxT)>Ags>NT^aJ$;+|ink&J8=*3Z&7Zl%U( zxp!J|myWfGD$X!P>a-;N+XXqa(k{!3bx2pVyeEF3s`YuA>Xoj&y)WXfSq?FO7C2;` zuA+Ey1abUk`DDqJr}B!TWyR%o!=7Cg8i8E%$&lSyM1B8ctbQzLCD~OgeAT#YPV;4D z?+F%>3=1N^S{}te{=3q2Tkm}%4U$DZ$$5#*vi$$eia|ckk}YSrfqUcV_NM0Snsm5ATPwuB2AIeqy|fyvW_FYV)t0K5E(1_cc{yHSXhc^ice>i#)yGt_lQ- zXf^xAZVTrm=M4HRqP{5TP#(Ul7CfBfd_{-!pcL#=jDNP$`3EW2Uf)tP-O}@o-1Caj z&7Wo8KJhL8j|wT?Y5zN{kP6mB{?yY_hXkQGXW=pqheNpazB8LOH;dX zTPfFz&NIXNo}9~+SG1I0?WXd&zLHmm=nfjVJ26_36ISZ-r_=e!^WspfJW1>NN9fe> zXDF)brrK8rZmMI_)eGsGZ(eNdx6ey6pl#+DR65%_BP?RuIk|w37VSW@Gp*?E5z3rz zAy?Z=UzfdlSM4i`1Pvs+M>3&Ly<3DND`>-JzgC-R2?}jE;5aD zPAXg`lNrv_Iym-q(BKEJ4-f_hWC$lz?Eq~q=;YGTG_yYHZU>)g*s6>#p>teSsn2uh z)>ppv=J&TOU+>Xl+n?X@x}df(GF^#x%`w&djs&G?ap`BtCPorkQ! ziIp`~RgY_GYE(L`*VD;q1IRQg40WE{an&s9LXxA!ei()@lC$Y8mOV+R1WUc+D>FJ1 z_Wlj!L=O#DanYB($nC-L6R%LksUt;KUZ7IZsFbyn@J#z!p7tIK;=|!}3GmnPP(0nP+Wgjl+UwQJ%EQn5MXQw7$!+*>9(oS{28XO_m~ps2n)cC zY|udjm@-OPtE2djy~K|wMQcie1cr&hkOT-2yX2fCJp*pHE_D8cz{(C~_N>#+&be6Q zq(=h4;R&7258*(Cw3P$cCkk=o5<K0%?O?&%iV74S&2hK1` z_w!QF(L`)Jh%mYaOO8u>jz713`TZ~nljOjQ-h@=PY{DJ$y`NMoIdyk-K}&3%eFV zpzYYcy_jWxs)`&96Bs#p5amXC_Nt+~AWUH4pyFta@i)R=QBcB~ShQ2GW&2+~fwiUU ztir@YGcS676~`P>HR~IQF>`HIs)p*sZGfPPw(Q89dbsE8ypa%aU6Bb4p^7SK`6|a zs9z7K0WmmU!)`O-pVM`pv5ksAUs`Bq)4#lW# zBb5><(KIvyKVE3%AnG2L&pgHsEEA(HXmd;u^2C2n=H_ih1?S`_IPf57aJ-YB7NXU~ zAMr*okv$iM5X4VD;N_zb#SINeuszn=quHTP|EHMvg}{jw!%a ztM~FkDmmDZ9BYz|@RTI9a2p4YoDMlcB$^UP4$+0u!h$n3BU^BY5Po03Um$*;%uH!u z=_0_B9ZK zjy5`trzblUjyRq^Vm>k2U$@bvX@zuOW}iFm2*oBNo_|QIxOgKPljaBHgG(kx!9t_u z$7OL&3OWxojf{kYOJfji;=vbyT}VUxUo!@2(g`l-E@)1qBqbF%Xz!~ncOTc2n0w>K zk|dqf1LPbx6oSM_4WXxnRh9jw<4+Xb^z&@%|2ZAP;0Ge_maB#Dy99)VD_B*ifwp0D zv9Vn{h}4Ra;M^buh5<%v;;Zqc&SCrQZlMT!A?r6>at5V4={+JODmi0HR8+rn4qK*A z5JVFp6V#NJ78WN^%U)cJHFL*CY`1Rnt$jIKnwr}2-0R=WT5?JjzVWO%= zVu^z&t7EW?ZPS4j&|5lRgTMZHj`eS_5bY3=hbb29B;qdHrYfSybRKYZ&GH&%7y(w` w^cG?}__k&q+cllP)SF$zV)Oq}wz)FH88~vFlC^<4Om0Rwq;fF%z^P0B3*9*LM*si- literal 0 HcmV?d00001 diff --git a/docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png new file mode 100644 index 0000000000000000000000000000000000000000..73d8880a8edd6cd6698fac54bc699cbb3fd47c2e GIT binary patch literal 26139 zcmeIb2{_ejyFb3FUG3)GE)|MhQ6VHG3bh-Tip(-rGA(1qkYVq3*%hfJ2^q>fEy+wM z6=jI9$ec_eWS;qbo~3u6_dVx*uXE0Ie%F7vuH)+3+4i#5_xpXG=f3aHaNqmpS@{#I zmUAqpP$;XYCyy#oD2r++l!YgMT!NpN?%B8kfBj(dD^=x3d|mzV(x3QucH3hbw#t@< zwhkAp4JbwymgWY6Hu}~E1{OBPmbU$i@?`L$9pptvtPL*Mnpj%=qGDohK(RHj{$-EE zFEl%|U-s_VyYCkf2`LeADSQ_cAwi-1LZKc#ta9bi*G6YsFNdbNxtcqhkMd}}_qcaM zDtueYt)E8{Z1VTpmE3QV%1*hZCh%G1NR*&x*sfbK&MMB?x9{vzi(4sZQnEiqZu1jG zl~mQ=<#>F<{tP-f+P_A&rcrR&h0dcP-l@;j#yXw)dz|_kcNZ6T+URimlo}>4!}U@q z4f~?%ELh*UAK22u!TN6LTE+tMjnz;8@&}SN_KO^{Dk_wM4mijM2M6!@V2s2xq>NN zQ(PSPAe;5EDt^zOAH06!28ZM~zaMK>XQi!gC@#KjwTk?1-op0ApGy{%lJEc7^^IxQ zxZ|qW*w~o%obE4~%+!)<)|*;zw`2w2$6amjeE0sHlsM;n)kc0cCWg4ptwS02oa%ld5V*H_sK-~%YG zzby@5E>!;dfFmckw|~YW#nkR^;}VBoEL!t1XeT~aIZB>couK6#5|a4sn^oyUkt1&7uedi)9-JEM_rvuJ zIDL3>439`zPVQETq0M#ucE_pSbhZr}B0acWGfG6_n6dc8uJXr6+Ff(?VoSz8n0@iT`H|HkDy zexWqzKzja=#_kPUWL>p6B`m7&l*BK+{b?2FzSDXxBfdvwCZ?w9rCnxT-&{v)&dt$G zF@Egs?!LURXs9F5vfJM%m~ZUkZQhM`Sb*#7)h!E_)+&A=sH~(! z*G#t%?4nscxKv`{XM~@Vwc&F5vXxp9GB<4`sF`wd=^8GTZ2R6K7X{o;uydcJ%k)6u z;P7zOeLm%)DlwCaqn9sVzIFSyvd@k)_X7g>shT-XsmDDxJ>GJ_{yw`_rnQ0p(B^|z z&dN@IyEZx2uh;SMfuPp?hYzFeda6x|9|R_fMVE|irLN)-JBzQ(LTue9%hv5Mo}He+ zF1s(n#l>ZzzH!~UuoWAHj62^4b=M|$o^d93bav)pX2B}o`}cRr$mlL!ycoMND`L#P zw@%fxhM}sa_5d4#jhkB-i{ICntv((cqZ~z>8tv80bItjA-MUo$!qwPp>Kb`&bFL%3 z_3B4FR;%LdJ=lFtsZYaEW^URh)vQ)YS6A1*wX-aw25UOf0sygA-=X0kS6(_Mx|nBCmWSPxmkiY8n)r9lQu z!Fc7dMu4AR&QDlR?bVG{sjf3l$BrItsF18?=)_%kb>m%4jr#ld?`h?szf~vdB^s6n z_-eTgd_I<-k@RbvdZJp0j0+DnPAx&HwZKc@`Hj51yoQE`w!S_KiS61=A4jm=RSj#? zOARaE-rGL%HMuMnwr!u>Utu0w(Ry(AjC>;G7@Oa01~3?(I{`#wHl zWIbR1xwa0+L2&U`a!`uizpu`lo3Wgjm@vUDI(SDX^=8q1r@;pxH^s^U?Mw0LmSzJ5$9m}+cnwCk#f;O}3Hi+$01;oSbB z_U`W3j?%z5Jcmz$;`}rS6h3oG*H5i$L>q{;qAvgqbqu3$3I=O zzI^$^ShJfjb@}Sm4|uZOURc@KROXB|#o?~%b0h==P9apQ)~z8Avi?NI@d)Ita|zXQ zHF|-ncGl14*s#XIB1fEaaJ|KE-W;P+sRbJL_G!6OecAP2#yjd#O`|OHW~zm%=4Sfc z$O_)4)1TbGzuvs*`MC-vZ(p)e8Ka_yTiQwG&6_vJqb(3wut996$1jIPW@KnY$pQ!;U0riOx3;nl3J=dF@pI?SNx4j> zybn6ClY0D@mCKhaojrTEFS}nx+_Fhqzwi$8_r)vPdV1mz*^GOt6DHey6pC^zu=3)x zs$FNWA#-zcm#IVn^u3lG3MI* zb!z|R&wo_Mp9^cA9Zeq`8mg3Nz8dzDCH7@#XxIeaaoxH0uP6n@!j{q(x&P@K|zAI-)u{+-HktpwT~Wc&JMszntpom z+u*lvcd;!l>g%hjsHnJbkFUNwJMDOG_lD~p9#}-(x@?EUpMU<@)5DWmQQ*bNWau2w z=Zq`lcGK>yORdzh%;BYy&4P7q@9$6iZPC(L^ZGOrMAZ^BBgmbz={h4AI2YI1l7CJmcC;^N|FheGCd?bva=rKRPT zmzS!oZOSb6N}O(vQ-NbgSx6$eh7GrmAyBK(XtCJRJ1i|N!#b|5kvVH?YwKTYMxI(~ z;ib4PPMtY7q}m#iK-=3RuyDo$pWo2K9fvwhM!(i-W?DxebHwD$^jc0%50%+rmpM8) z$Q<9&82#F|t_ z8gMz;qnzL+FM)0a6;p-&zKF5rLn@u=?dkmjjF3k(X zO}9-`w8V6XB;@ad+zx9-YQpe0%ns z?a!ObEx5BKuAxDj1mymlvEZcLqsW<2zaBX<((g8>B_bjsW&iadR{s9|`{4-3RC<$R z=ayr~j&+R>v@Tz>M*ZT&C%F?HoI#Q{1t0I2;8r;#t-}q=f^|A>=QqY6NIK1o)L4>c#;>Yf~1fjFMuVyZV4 zw7Y+71Fm24(c9{b;;NCnz^3Rtjx{v{0|Qc5zJ-qVH|uq1I5}nU;CYu@9K=J9iHSMx zg?-&99bH3Xc?lz4q7!=7YngI`x7SRess+VQE?!kEf@n zmwPjfGh}Q!Ja?Ra82bA4YtE)llfAWb>mHRF3W(b0^q3aK*MHH*0Hj@)yN^iR#_|sX^C*1Cx z*KD75*iB(c$#d-N?Dh(7LVSE5Iz!wT5vTk(>NjE$Ii2U^l#U<2AyT|~{rYfZPXi;P z`chq^OP8Wg`3W5_ofzsY)|M3&t?WDKJl2URU?5&vpHm9P`qF23F?>dHmZ#^gc)IlDazn6zSbFWc3!QFE@7vpGAG$ zmorAHEfYle0%z1*DK?$*+xMqiHtPxr3E7LAn3zaexBaQOj*Tq>r=zz%y)3#GA<){! zrk8eSv*Z&Lu3m#pG29{<)t=Si-OR}4FDGT8QRaAd^S+uJ+xwEvTR=5?DC)>txQeJ2qZKZKny zMQu!HSiBKG|AK-@e2V@gER1dK+O3G*cuWFFz{-)QKcJ{{RN7DC+@}XsnVi4rT^%?E z*g$K@xQyyW4OtSUs&V?#fhWgqw|)H@k31Xu^yyhrBoz4ycu6yOO-cjA5naWD`DGBD z1$DBj_Uk6Sl z?N?NRS*D|RW|41)2aAOTBqcNRoqru~{DJaG=%q|eMt#+Q3&ZmMT=!_>=V&%k`PZkJ zBaZV53kxG2p$ZpdtM*1f*x~c;$2|IdU*8DiK!Jk?wNaIq19gUd{HW31(IM5)JUcTb zZdkIZbp+1?$TSiKhs#)_1KS~NH^a9S-f;<@x`Ps zz`X`CZi%`%Rndyu9i5#K#5DQ%_zDIVEL!sR(%vrAX?Yi9$pZzv)7mf_ceZja|H|#T z$x5J5;4XDmVV0noDtB$^nrc*Z5!DI3V@eHAFI*e#Ym5VK&K|JoeBXu7!Q%Y3ZQC{= zm{_9_7c~Hl$`jl!ZRLdnV`J&gE}vg7OT`WcAj$ylZMr&iru30ya>=;f?BGUHpb;sLC=0Cf{4SsW%%Qm{2MoJ+_K;55bBYtuF5Dj z4i0`Q2|##Yaon!shjGw+cyIx+D08Gx86prh?d>!;hvVz8?7{U`i&xLA8WO;Btmo!V z{QdWZ*xKx^BpYe$4v?~=`PY8Er9HJcewW6nimDjpRF^6H1id`nn>TOPCFy(6eS8Gs zoAccANcg}h2TOD2)Pvo84)F&=+-5UT-JI*IZ0Rfw+^MIhC;jckq5x^_r`gGxJc5SUsuFG}F*1giQSWx%522hsh zojK4oct%VTnMr+wB~$|Bo*eGd->QtigM+x)$Fm_WqyGDVDmu6Y0XUn#V8Pg29awvN zbqhy3DW8yv=ydvUgH0KMos$5KJXBOPn$IrXRAXrK`E+%hx;HpDSe57I24MY#sjips zv=2y+pP&C(;MTE$0g=%iiL~q2ueS@`4BmhFNtRttBx+znxO_lW9)V@}g4%%2cwdK& z*@K!c`{Komq$YCt_Hyau>{LIt(5Q7HsoKdA3fyO|1gzr~!LvqYJhX9xbp^9_DlvCb z4m&IsU@rLTAXO*ZISlDTL3SpEBnDny-mYTdJQbwUk^1IYHC&8+=Vb?n8f5&~&v6TJ zzgFuGmalBLwzCuGGOm7nWNB}6o*wq)hr#yZ5n$I7^iY{>G4QPyhd6*TlD_%-QO4x&)&1Fk7{5Pj8)r8@8sT zq$Eh1jM1!=l&VeHh}ZJqCBPvyrf`$pc1OR1!=MCw*p7RTcN5OIVS2Y7u0F&qal|e9K~LB zaberEDcVoh>D|M}2{dV^;j7$?oyZw15};Mb`zNX7Dul?8s|t3Tb@bwv)kWA0IN-o|6cp6U2)citM?yj(n|G8v zk=QjY90reM-E^^ZsP*;+meT6e5&&}$Rzl=LwxfDQ?di*-?>Hg8Qz2LdHF1$k^Yp}! z9mp%x6DLvT1srsIhS+5}mE7`wstjZwAJ$tNY~WUB$t(t$BhP!O_nqu$jI zeL2V^)i}URoM9`iH%)?dYU}DcXV58a*L|NbxR$vS$5Aar zKO0gzZfF=uSQ6y6(2|*kns3?5Pyqh!!nmA%A;k34`>Mz@oH?_TJ*v3|s`$_XPj=eFvuqfN(yND_(N(2~W$j8g|@!&h+-W%Tbwf#40gURZc$ z=;1{MhK5BTpXBAIwU*HIkmdG@ijtoL&zvA1aF%U6&`umF4iB^%;m0>1K(oG=vSjDZ zosBkS(kxwu=v-Fnzi6wAs;jF5{QVx$60NMQqjQc*gbbCf&>*Zpn&N<&X0ue5L*f^@ z$uChT{{<=NKNpMs|MLpwl?+|*@2Gcxr-dCI9r?fyYo(cq+#w_#iu1lG95rB1#<`Nc_P!Mep7v#Yuv%Te5odt~G1cASdwc+oy&d z&KzqYkDlTX0k}=-71Y0WpI$5=#h0}6XccNryWua2VUd>2x!}-6U6F~+9PVUf>?Z{i zsRmL_{e*O%N23gn4iA4F%`S75#RA{G8~Vo|KSYFwzrC2WsqmDX97(<;`PVRXAhOgT z#fmu&wsp__Og^6Cz($lnru{k@85u&|9rxmhxVQZT|IVE!IQNIFb+p=3}#dh`!uAXEoCsmLMpCr_RX1BdXVrowak;J#pyZAS?^ z7`~Hq5SilEZ6}cdz#qjaMtCK^`h%JSwsrFH3S28Pf8nwOfUhYebn|lX1RQTe!Fw0C(+eZN`mR zb{wSQ?`w$p^UojkW72t!9Y0Rw1XQS%Q6a9epypJr$Oo40@adYHwVh~pJD_BfBWcrN z4DQyTKvY!J4p@A7GtnYd#1-gztZ|B~ zmMmQ=uzUCEW;c-he7km?#Qp_>NDIt)|Mu-!KOr5@fT7MZKS^~2SrM%*pha?=$Lh9} z0lWh>`?L4z1<1H$0Y_hY{=T@Fr10sPnaoZSQ^kwAI>RJ{7xw+q*B*a}aSGytUQ3eZ zvYMLYS~L4o32t2||D-_tkP3;4)aiHo_HDqHSil20dPPNrZQI+mByiKiqoa2LA49!! zar*MugNKR?sA^%sSoQ8PSh_D=cJr|+Z8u1mMnyO0R>0}Ci1+Sk^`vvX(K zDmaSPoK_@0O3NF^Nos%x_sx@?F~dV!46V| z=HoQd9pNjW9t&FfI3S+sdQMK$gQ{0irV&l%@^k!r5=6eZMZ2I&Bk;zfSrk;QK zd%zT~8($9BrIN}MpZyL9h>1N)D7?(DmCoJEK5n!Hl8W>_I{j zrNNhWP+P*HK?BK^3H`+usgT52RN4C$4)-MHF(jI<909?~1p9?-7LFax(tJ-qMIzel z`1lnRjHifpfQ+M}q;#=rCAW+=m@3q#XHn82HQ^k2wgS9#H|OOcouru72)!vRG(~Jh zF>GDx!cMNlJ^S}}px$ZO%ec9@OG9M!2vGd>*I(Cz35CEx&5}B`1(<=TE&%mKrKP?o z@}Wsf6E6YXwy!}Fp@KHlQF;rEQIdY)`n78xL#2dS9CH&EYAc#r^+vfz}dF7vLw-chfHRIi- zxu`$>k@_$77d-*I!Y>-^?;is?xA`e4jzjq!CsP@{8{GMQL+&Zo3pSJshb*?wL4IK{ z7__G6nLIIwfOVjp;5fM0U=$P_%s`|Jli9xg)kA~TP*)Wd?-8-tFi~{xUP2aybs*jv zFarJkO&o&%@1(hBnh7ADV}AercVYlYHLZ~+kRTjl6Ryya9zNPzPt<2XEhb1UUg~#t zU)OEMj;e$;K@I~iDkD8yeuPdkFTaFD&5xgOa0xwz6zTa($is)bfa&n6+q$=&OXSDn#V&e$(UO&CVPN3l1FyVx?OH+SDj3Y>b0MOU^dSgdv&8P1q?xWH%i@0ESVaB^U{k8o2#u;FFMo${?AKRpq<#DJ zl2FH~`+g*^JEWr?W934eLV0szb39aTm*IO07cBy6js+Sg6r=M$yb_-VZd8sAw-RxT z5z+$@Q|`=}gz6k5>xWFP^2De)JbgwbdHmMwzRqPK7F`uVB<{kzQ+2_zb$ zh9kTPxjZBT&{9eDprjL45vq}LwnkDt^2-{x@oQQ3z3|tF5xJt^3iKDTOKhJ=lO~Q=u0(@OvbLVgOHZ+A)I{EiOY0zSM9EmE!mphO-35%MK>;3|mtYp_ zKa8hlwLJRSutA{l1$c0H}9?KpW|1`tYnm2?4>MP(nN=pdb;Q z>QYR6!sC+rq1^Z(^Mj_0Ag(yj-%#A0hDs$YyKGqd!ZqjgKIGNpCskD{--#I2rbB>8 z3l}aVX-sgzP(yQ&Y0IGk+n64-+) zCn_MG`5BaX`}+DJrvTz5LDuz}1a%BdN<+CpDj#sCm-YpEofo8H`5yoa!T!evJPU3E z_`JemR2uAoTAV6CbT^b@f7b-HI6(3NsqZd^SR}6cSuU)(yH@+rv;v(-hEXi%2+ppw zL!WleNVS$h0SXH-!;**ft84ZJo&m?H1d4-=o!!K9tfxl7TYB_iety0!tQR@cgT=5c zMgi{wAx5Gy$;`?ss;{r_y9#KIddZImsNHYsX)0u%>;QoH>{je^Lip1d5=QSg0}{oR zT*1gUo%3mN_bT2LYI_Er!j>spGiIDV$2Z5MD$L97H5Yfg3J`zING>)f6 zG||_yv&TTr--Vap6vu#kRce{Qj&7_N8GMk`EMF@8Ppk~PR}g^eWj#4zY@SY0;Tx06 zmvVt_Cgp}j}R+oA36?_XeM2fYCrt*OJo0|&HlPz(%H5-;1>_yh)O z`CkUTC@HB*tGB|#2&Gl^)+_&}r8du8sC2@^m;o~u(f6RPf}c7HU|LD&a!6LqN}^aV z;pg9emzf>cM;1?LU?1qw$*?*M&xj)QSDqLm>cEc$yRlaQVjw({uYasTMT1(HiNos| z@V(4obii3j=rJ0?Gg({IO3f9WtVlZTq}pxL0;&Rsb`pEK{p(e`XW1w>6=WvDo1A(Q z_SRo&e4BB(br>=dG_Xm({8z8MF2h^~qFM)1ov6|KDv}N$g!h2m3J}r1As-@>fKc}i znk-B8>he*L`2cC1*qh)&C0doe)@8^&3PikyKEw;J5~!0B3jxMbt}BPI-TqeHfCrPF z{1WEdbnAB1b+A97)!$OA8qE7XMO|0t*VY?Cd?H?HKy7u92J1I$urDrEu@xT5?$3<_ zK}Q@A-ZGQXxs%-jAc%JD->(7rq8zuOPK@&-u(Vd=qDOFVao^1n5U1B<2f+LFrl5d^ z-9QfqZD|5~CghNNiZ+k}u~p=;P(DC-H`Z`77O&WFKYKX);2)+~<0_7P@xFa|53F=h zn-aIzxn#qFm7|ICEd!}Hth};Iw`^tz*p-byITM515N!RS1g-Ra!9Q_va>ju-RC6gL zM~BjI7#R~sgMr%H1Pr3RH&;LGVldrmnU}I+r`68rs=7YnVFajy^**^#x)Q}96z8L< z;y4AU2ZKVDL~F23m+vwX?MLmrock;buDnP@9bkud9qEb+;Ydte+)aFB*gMQh7QUl6 z{3$26A_Ue`qB0YSfXMlD-cz=qTM1)$z@aa`k83`)SMrP7G@|M!?)og-E;^`@$k(rr zE?%;P2BS9Nn(W5rtBN4CuF}~06q7L3I5kxxVRDI-hI{-gYXIU3B0u7v5(lX$liR*( zcRgYy@fSJ~`35bBGo()7fj#pizGs^S*`x>Qr~*6LH2yN{eY$Tgj|xnOH4tf-nq^{~ zrkPL&`vl7f4k)Qwzm@eeJMHT&SwIQxvku3Ic@Jz^A}xz`7Cx<3>f($5R7`=z8Cem& z5Gwf)luKUD35cV>+rQCF*OT{g&x%!QnZ&u`IC$kLAu_qRgs77cPud`|Vu!@S700am zULc%{7|nF;@HeW;vNMGjLJdy!Dwu6Y5JP4;< z!X21iv}{;+(69{8o3NgQPGm0wxk&1n=X|XH*vG?mqX$(>YbQY*W@hm+lV7%z&qRV| z!Y2|AZXB$1;*KfOS3wDVA7gdy3N*({d;L90g1}}IFF=gtF~Pm^^YO)W?3-WYY^+j! zx@F=!(!YbcNndpnBO6G}$A$IpR)w)umHE7U0HH*@2c*Y45>yY`RnsszQ)-IEAcww> z!^_re-X98^xCvSzir_FPlbwqv#03&aB_2y$_6gQocBjadYQuc3hz(J3p#S-Hqs}V8 z4XphP>vyeo#;9a)?K68wSbYG?Zj`6Q?}_qYq(rprOWtfxUhKbJli&(HIB>8xSTcF0 z^)Y(e=g&`I2a-p{`)}ngDQnbfKZA_vf)a?ThX85Vbk7FmG_D16fxPb<7zp_!alZE@ zZ$~9sgBF}k?NF5puD-?Do+sCDBgpDDKxB(ekz_rSK3m@`79|XV+VkJjgVFg`Gw_@9 zBM;GsNrMPT=h&o3)-PuGsBB`uITZ=J3AG~rTr!y1Hyoxc^~(JK^}50ew@<$<9j;PV zhynYF%Oy4=sPY;Xy#k5PmCXNEcVpEN31Wyk8FF>-CLTkYS?xBX&S&jAd~VKP(70;c z9&PYk0%&%ix3#r}6M1?0k3U8y70h3XGA_jpltxO{6zd0{?x@CC2~u~;KPvzH<~9sE zBBYFlLuK8L(`P|f#3r+xQQd|Sm)||qS6{M)^`-P?SDzB-?h~&lMEOh)x5c+Nq2!)gpyG~z$D);Wyt0Uwvf*0$h zf$%X%(!e%0z@H=yG*;-Vlk~W1^Zv7Nq~mr-_<>U>wk??zUR}3Xa4>YVwTT$ve_C~j zCm%&`H?aUi4Q!Y+gz*obbTFM8(snfDI%LjGcMyvtwX&%JIP0|NQedDp4JgPJHj( zQ;bPS{dkM?R;J!a$goP!tWW<8D^SJUblDu~R_HE&d<_8Y>z%xT0?zbCZOAt~KzAqT zq>4w0P7Ku1s6hY625N#!U=9cKCI0}`y9FJ zJ*LEtAGH7Suigg-|Afy>fbELFUhIDJh73hwuY*(8%ya4D#bCJOHgDc+<_SylrsNX1 zMX^LmXnSe=Y6ssl!otNWcu#Um?4T<4zE5`O*UfH)?hPBekZ71oDWU;T2oH&$X*Rn)qu{E?iQxNQTtNMI=K0iVtkEjx(p z%|T50z7_?LAD}akwlTOKAHyk9x5e@wNz)3Zb1nh?VCJcG2zfIEI4B6p{Ev^eo` zI+fftKO<-C75VtdkB;Z8Q&R_i{&R=*?L#f;Zo}Www;Vn=x#;}@y|Thx@(iq~u%{QCVH1UmjNoJ^(uMicPoeqwzrs}=`|O!g#h9N-RAeL#d9v=gV=^&> zfA8oaGzgF6=G8EY(qLSg1X@EXk8_*tbR%9Jal-K`q4FgvK7|Oc{C>f#Q3;z{vXR?qxk#)^|RC!9W^fFEgm@P)hKwucSS*ILCeVJ$oH%kXe(!% zF6kQN3)1u8H2ejvpA?tbiO#GZMs^As6NxAT?&yq7RrkFsWhC`9+!)giOEr1fpK0Iq z?3)Ld(@8kw#L~_81=37SNe6AyVwR02qc_v`+&FNcI4U+$_%fMH;^`t5v)dbmyI~wP z?dD)Tmv6>G3EE@b%u}R)6cX0(moV?xCr^HDE%a7L9PEQZDw09k=qMELjzMR}kt0V^ zty=OU<5|_N`-XSzR>(l{sPc$Tje3{3NYEd;e$%Fp_eWSC@o7LiYxh-z^6u({K;mM6 zB-Det#EJ9{Cum8a88HC)UC&b=PR@a zZCQTTxutC1PCLL&YWV(9%(|WO#wgKQT1GcKg-lCy0YxUx`&HH$imrSWo%jmUB<``M zYj$qYN3+pdvS zx?oZzdj5L&bpw0n`g@<&nz4{FT3AXWtY!k6r;q1Nlv)z$-qA60cvOh>>lU0syzS^3 zRGS<3o;&%Jg;_(ngd*C3D^b_bP!{9FJ{*6$eKBMbMA&P#fH5rctTiRCeB z7UbqW>O4qH`2m;pHy1Ezywt(5aI)+aYa=s>(k4(VH00L0+5+#TQ}1#?52og-*d41D@rbN`sev8T&BIe zhS06(s|w6QOA)>_dGNWg8wF(j^=SL%9RZY@&Wa*CbU|@k4H=o6?VsDHd6@MpDPp(@ z@TF0`(1+j)Qim2w`U?ZY_sqZLa=ax#Opm$S2vZ$?so#zZvhcV2g96+O@`aLt}5h1}$*P=2b2#UHR%PybmC}64Bz4@b1<;j^nlB)0Q20 zy-uDqHxlmQ#_BDx&iy&*)aCOph#)V3;rEQrQc8wJ(}>=xA5%0=t->y&PnQAetwyd>r%QeL92hGDBb$f2qBuj6j9 ziO9oZaHQMyH|dbxJL12DHTn={2$ zxKxF;zbS%##R`PI#Mj98BL4i^@1rGde>a6_9Q{7|VB2@+cE^rgCw|cc-JHoR{o{8wL!MnYWP~mq z`5Cmpk+cbdnly*NT`T}aR9sxV;~~-tT6*|ErDER}!RiFvpjHoY6OUaRRBK(1b2@(Q zK2&A^4T)JQYeQ*@hjh%oAnjBj*H_(1s_#YLZ$f$^SW}dKOww0^Spui%;4B5rRGgF_7gKw&iLos zsI!l34`yMgP;t{ytKM5Ux}<%r`(siVaROuaDS{2Y#{+8d3(^9;2#0so?^s@h=>6Dv@+2^WxTdv+rOB@Jn!2$Hh1t`i29fzm|}#}7q4=ZG!sQ331@`$rsLy+Bi|NMV%` zLa1hj)rFu@5KKK_QnR~;7F!K1{M|V_`-jjEzHDrn**{POf3n%8$*B;srnsnxQQvvF zg|zqVoohtP#t0fB0f?Ef?MI_MOV2^})t*bqGjkz*uH%*h`V4WzdT>3(4MKu~s_;c( z4oFEv+ab|`EUT(UUW(WI#9-Owi(Yu`+4&@OV0hvw$nTn7+^0(dNO`;7I@LmNZa*Fn zxUpGTnjnJY<7!X|k``#m4q_0!V+RG?CRGZ(jo1Y?-NHgb)$PT8J@LkKcmJVSY%*IS zw&GReVv3iy-lqS@Em*66|hdA z>A;clqhAyZ*mKv($`F1?9c0qRhTR&Wa99wgnNCh5hEU)r2*6ZG(C0iWYbqoC90{1T z;*#zXM7mS-Nco^xz+)&(a&*#)3`ZIt-2X@uHAs5m5NS}4UUrlqA?8!0m~(~xA#W%MiXW7W7Q#9EFFKlXvxo{ZU zeDi7#pk$4KB>}|;n-2W0X3SS9;;!|A?D1FFrKbyEe5nG9KJ4yth zj%Y@3u9Eg75&Z(<4v^PJ4F`+nFCMS9dc1lJt2W~7JChRNH!+ZJAYB9?<-oQU%V1ap ztYhWO6wOjD2IiF5HoJ_OK$jjF6$YL}(ys!%Nd-Vb!07&Pf?M_6Ie*C=XiSG2K@C$> zNHh81pf%!1Ok`xi=5=VA!NG+4nsh8dk$`(<&y*>RMnt(dKn<9t$*+O)7k3{08hOBx z?B+vpmr!wA)qB*Nk*r7BFsRLmnuG<#CxM^$8J^*{J9+ZY2cTpjr32C)1^ZP^(iRzq zLnjmlI^%wY8C(&x86FXQ7Q(%l2J_*=hk3&i(iWP2ucNa)6}&qkz(FFwAqQPHB7!s- z69xjfNL$L|uM$7w`5MaYj@kyD1?A%pvdpEHl+#5>RQ)#b^zn43;+S^GK}RK=bCSAh z4b$t1c>@kwDjjXtwiAN}Q6g9`(5Bv!5Fw+%z^n=pthl zE#tksM!nYT{haTfm0|Mrsf;bk&Es0^HVM3^9vFicv8yXX@!Fp|#dyiFK;4E52%Su@T6~>yj6{dBX<7w2f?R{&6KJ_pr~1gkRZh2df(l zoR{uMLSPSS4Uu(g>e~PBvrIYMZ)g+RZ9{}qP=Kh9MbSrJ`no6o1LvBBnw#>R7#R!Q zIgxNoNy0%|Nv!WwWa(0U+UKW?6Uv(Aa0^ z$Y-PY#%}N4U>`a%__b4;lK#g(W z-UJWgGlI>7k45_biU+7znW43)$!1l?g8t#7rfn_#BR_n4{BYfex+~*XKVHOtV1$00)!&S4P3StDcZXf#@h2s$bVQUa?1G;<%VnyA?gr4{BspyF;wdXg12LLq z%$sv{l9H0(&r!#RkeLG@5~&s3v)`7!dHwp!$0U8Q`ewO4EO4H3g^Qix%`MA-jzMVH z6DLSFFChYn1VN@!;23vk0DGX}W)~V<;T=;;&D7D< z(6pfv_1Sv8`?TkBU1SWNBne-T$#sJ?fiPPcm-xCuE z9U9B$6a^$;j48^a(vg~&Xl5icYu-PMo7xD<7r@R=CL90kc-V6et=X84pb<5LNKRCJ zZCOQ1`tDyc(z8!dICRb_-oxMJ3t6djc;cH!`gXly>>@~sC2nIF&<13Xh4ELZ7hWwP zav1$mvgF*&_j^30%xkjjPrJ10NWTA%DIytyb@``U3WrC{^)LypA5!+Xj!b{pNU``s za&3ZFRG9}te~v{+Aw&ukKyk1sOSw*Gd@nT)dR)KmuaDSX!u*PPjH>GDaTysOO-8b! zqfbJz+gtbugz9^vAV?DLW5-j!+Gl^YcMh_j@7`-0YH_bFv~pN*s8xUU0;^L9XGK>g zuYNr160DP%n^!0at`snc5Tej3U|HCgoCoLy4~W<`8+fbHAskcyhL7Z<4$RNlgk81^ zhN;cTa9rd2hx%lrQmlLn6ST`%+JlfwmbDO!}E50b$~BrgDUl}34tp#@_5i92X#ptJ1WWM~3N+d`jwR?wCx z#nM1#8&;SbRv<%~fH&h&!_Yv`5y;GA($|m)f?VaV2Dt>t3yXjF;bEc1e=&C$quwla zenxK6UG1gerX>zEjkDOgs9m9Da2YQV^K*Re@IykmQlvg5-JPtbN14Z}c3JPd?`pon ziGRiG{1xrF#`r;wx<_6-RomV7v-@#_@yzUn6!j$Qd1&;%_=fWIuV?lD4Ay%vTxXg{ zy1PkZIXXH?uZ)8qs{}SaY%ygu^Zr2pF4X^d)9u1UAtG`q*w+fcmcG|(GRWzTRU9tb>E zL{Ue%EQXebWEa%Y90%;pa;84^+9NMTrNQK|&m@S_4JvQZ+zIgAB0CqB_SUiW#98#f zaRLcO38@>yJ`)kNav-Mzo7^Y%A_u>D&SooFMVMl`0MtIL5PlwtJU-G@c>*x4g^NZW zy?ZfUB1nZp7s{Ev13N|ge0u!|W_>U-r5AzY#dUu=XLj7B?5BBVS7^xEffbf0DYpM) zm%0~fK;i>kZIqiRimD!)%iJmiP!Zrx4hev%%#Vm^N-DYS?u>}_Vmq^{Y8NKgfFOer zI>w;{R%o${uYZh6L(3b{0AbCvxI^Adi42Xvcv}X*L$lTHp>3aZfi5wz_Y2A?Vxq_= zJ>aw<6JX$5Fs%rulHs6si_%IBZR&EJqKZ>{DU=i*xG}52PWYmE3VSZ=IX5mw28+lQ zft^VFGCVPEbF-vT%5`q4xd)IwQFI?zNC0ekb5N9n0|SZ}U6fq4f^tat?YnpITi!$1 zC`k(l;2>kT$Sg&`-nQP}-jp@uM#+1Z?pcZR%d97pWg$wuh!KP?Onmi;iIz6^mr@S- zY=U=fHMh*ys(ok%YJ>;_e~NCT8s#QumqnAo6)=a97Cb1Uq*5e8$Cvk&5IdiN1H|4hj48zc78o8358x8dk;hkn zhecEa91f7(#Mny|DcFj5sYur3OiAWqBa;?MSB!e>fn2GIZYCnG&S3DCT@Ad8wn79mH$Dy4vDY>HzoQ}m^fNYxDyQ%UK%EM!MS5vT{bKRWLu44 ziXXy52Ew>!PGV%Cl8@^W$_tx`jBH$lCg{yb(p3yQm?Avn;C>o)D$H5)T*3O_(7US< zH{PLje5l#6s9^qK2<_{$wlH1o@d}(aMN?7;($1$P#ED#MmL??&?KP)Q=79azvv@Z4TUI0TS zpI02`SLcQ$CkYI`l0@RZ2J5hS=v8sP&V)=dL_7=ud53YO#1n<*2<@m=4=!oMq`>BP z@TzFf;#4QBgW%=P`RAX1k|8q_aM8!XfKR&iY8s8uX-y_o^6+7vA~qJwO?jEb8qDiW zfr}pAc7Bjiq`X9mEfIl6UR9&#mH^#m7k-7zJVU;)=-x~wpTT(oAH@;YtY36(6N#Tr z`f^EsN2P-6;5uS%Abtxn<`|9Ol3Vm4>yddZApFr#@z`yq#*GPUdkhTXU9e0Lxe3`M zcV<+}&-(!A9~Wq{PT+A!!HmT@1>3LfWCL-)!_h#7r=z(coQ$4C@md9;q*SEYbt*wE zUgI&w$10PS2h6sLK;ux=RCdOPF~{+57l3^-rgIp zT8ypopck+wJ|DbwVbcwajkY|w7qcxbN4e3cboWxcW@=GH055yqBr~z37mVpE!(?`x zYVnn?_8`EVKYIK)2bj8!hwz3njWNX6@y?MOD@A4@TsM8qgKf8^zu%IFhv%U=mhJ|R zKKMkTE$J9$^lfBBoXp?zpFs_>7wsQLC66R4_pE$HhU#8OuJy#CNCgpxohP(j&Kmpr_0EBSuNgS^ zWSB3Us$`xfl}v%dF(Vu|l{GmVX%}zQ^>Qh9%9Y5u#K-+Zzb)k|M?hl2QtUJakv58~ z`J$z(AI9}TGB&O1|AA6I`0bD=g<|muV3*}CMk&;k<4pQTkmoxVy^Q~YN&v#Y4k8or zsH+W;>r-}( zU#1?xvr49HfwYI|Tmcel%~XG0B3hVF(jh7Q&8r4Pjph4L<~k~h+OBc1e>rZLSPU|N zm{)(Px6ET=pe7n^3}-#LWt#?*Fu^MVVaf!tnJ7NUd_jGQ55kdw{3-w43UM8Vkx zKpl;v^*~mS-j~t3n)qMghs**mKMa@sF3edaktJX{10|aZM$cdzEa{eB#dS~vrHHLm zIUNleO2!Qesc#BkP-x|TJ^z9w%r70qprXlTMO$4F!` z_&CYjHfTwiZ<^OrLdBo5OE_1;hhr57DsLCcBXl^_AkZ}E$cM$^^lx!!c-%bG=YTe8 zTNVoOB=yNRD`I^IAIHRO5;8Oh{K4c0Z`Z0%&S_e=&_qavAad+E?+#IO7@PVe9cJO4 zjv}BKN$m}*(WOoPe*t8=N`D!$4ri|o85n>vJQ@SK$S77qP+;%A#bA9<)L9m*upKQ@ zI07neu?eu`EzD>om&ee_#`ylPeIOn(_~zf4G%oN92=w%>Sh-RKjvk+yvubLQun02i xJ2CJazp>8|kkb8o?|=dMEquBEYxZDv&58Bniob15I7VJbJtlvY@$32D{}1*Bk8%J2 literal 0 HcmV?d00001 From 0c81dabaf2c0f91272d50d82b5a94cc4b0d15bfd Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Wed, 8 Jul 2026 11:36:49 -0300 Subject: [PATCH 15/16] Correcting paths in the incident report --- docs/incidents/2026-07-01-crd-id-duplication.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/incidents/2026-07-01-crd-id-duplication.md b/docs/incidents/2026-07-01-crd-id-duplication.md index 09ca784..363dd2e 100644 --- a/docs/incidents/2026-07-01-crd-id-duplication.md +++ b/docs/incidents/2026-07-01-crd-id-duplication.md @@ -66,7 +66,7 @@ In practical terms, a stellar row and a non-stellar row could be treated as if t ## Most plausible software cause -The most plausible source of the duplicated `CRD_ID` values was the distributed ID generator in [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1893). +The most plausible source of the duplicated `CRD_ID` values was the distributed ID generator in [packages/specz.py](../../packages/specz.py#L1893). The generator works by: @@ -107,8 +107,8 @@ The fix was applied directly in the `CRD_ID` generation path: Relevant code: -- [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1893) -- [packages/specz.py](/home/luigi/linea/pzserver_combine_redshift_dedup/packages/specz.py:1950) +- [packages/specz.py](../../packages/specz.py#L1893) +- [packages/specz.py](../../packages/specz.py#L1950) In addition, later work in the deduplication path strengthened: @@ -248,4 +248,4 @@ If a short technical statement is needed: This report is stored under: -- [docs/incidents/2026-07-01-crd-id-duplication.md](/home/luigi/linea/pzserver_combine_redshift_dedup/docs/incidents/2026-07-01-crd-id-duplication.md) +- [docs/incidents/2026-07-01-crd-id-duplication.md](./2026-07-01-crd-id-duplication.md) From 9634123739fd3950df526917a44b9b06c922e56d Mon Sep 17 00:00:00 2001 From: Luigi Lucas de Carvalho Silva Date: Fri, 10 Jul 2026 17:33:42 -0300 Subject: [PATCH 16/16] Removing docs folder --- .../crossmatch-deduplication-design.md | 430 ------------------ .../2026-07-01-crd-id-duplication.md | 251 ---------- .../instrument_diff.png | Bin 23122 -> 0 bytes .../sky_diff.png | Bin 34568 -> 0 bytes .../z_hist_diff.png | Bin 23970 -> 0 bytes .../zflag_diff.png | Bin 26139 -> 0 bytes 6 files changed, 681 deletions(-) delete mode 100644 docs/architecture/crossmatch-deduplication-design.md delete mode 100644 docs/incidents/2026-07-01-crd-id-duplication.md delete mode 100644 docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png delete mode 100644 docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png delete mode 100644 docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png delete mode 100644 docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png diff --git a/docs/architecture/crossmatch-deduplication-design.md b/docs/architecture/crossmatch-deduplication-design.md deleted file mode 100644 index 28f3485..0000000 --- a/docs/architecture/crossmatch-deduplication-design.md +++ /dev/null @@ -1,430 +0,0 @@ -# Crossmatching and Deduplication Architecture - -## Purpose - -This document describes and justifies the execution design used in the CRC -pipeline for crossmatching, graph construction, deduplication, and final catalog -generation. - -The central design choice is deliberate: - -- LSDB/HATS is used where spatial semantics are required; -- Dask DataFrame is used for distributed relational operations; -- Parquet is used as a distributed materialization boundary; -- pandas is used only inside bounded, per-partition computations or small-data - fast paths. - -This architecture separates spatial and relational responsibilities. It keeps -the HATS spatial model at the core of the pipeline while using simpler -distributed tabular representations for stages where spatial semantics are not -required. - -## Executive summary - -For this pipeline and workload profile, the large-catalog path uses -`LSDB Catalog.concat()` and `write_catalog()` selectively, and stages the -heavier non-spatial transformations through Dask and Parquet boundaries. - -In our production-scale runs, that path kept the full LSDB/NestedFrame lineage -attached through crossmatching, neighbor aggregation, catalog updates, -concatenation, and writing. In this workload, that was associated with higher -scheduler and driver pressure and made serialization and worker recovery more -expensive. - -The large-catalog path therefore uses the following strategy: - -1. Open and spatially partition catalogs with LSDB/HATS. -2. Project each catalog to the columns required by the spatial crossmatch. -3. Execute the crossmatch with LSDB, preserving HATS pixels and margins. -4. Materialize only the resulting pair identifiers as narrow Parquet datasets. -5. Use Dask to aggregate neighbors and join them back to the complete rows. -6. Concatenate the complete updated rows with Dask. -7. Write a distributed Parquet checkpoint. -8. Rebuild the HATS catalog and margins with `hats-import`. -9. Use LSDB/HATS pixel alignment again for margin-aware deduplication. - -The scientific semantics are unchanged. The design only changes how intermediate -data is represented, scheduled, and materialized. - -## Responsibilities by technology - -| Technology | Responsibility | -| --- | --- | -| LSDB/HATS | Catalog opening, spatial partitioning, margins, crossmatching, and main/margin pixel alignment | -| Dask DataFrame | Distributed projection, filtering, grouping, joins by `CRD_ID`, tabular concatenation, and lazy output filtering | -| Parquet | Distributed checkpoints that cut prior task-graph lineage and provide a stable input to HATS import | -| `hats-import` | Reconstruction of large HATS collections, spatial metadata, pixel layout, and margins from staged rows | -| pandas | Local graph solving inside one aligned main+margin partition and explicitly bounded small-data fast paths | - -## The three data representations used during crossmatching - -### 1. Complete HATS catalog - -The complete catalog contains all scientific and provenance columns, for -example: - -```text -CRD_ID, ra, dec, z, z_err, source, survey, -z_flag_homogenized, instrument_type_homogenized, -compared_to, tie_result, ... -``` - -It is represented as an LSDB catalog backed by a distributed -Dask/NestedFrame collection. It remains the authoritative source for complete -rows throughout the pipeline. - -The complete catalog is used to: - -- preserve every output column; -- receive the updated `compared_to` values; -- supply tie-breaking attributes during deduplication; -- produce the final consolidated catalog. - -### 2. Spatially projected HATS catalog - -The angular crossmatch does not require redshift, flags, instrument type, or -other scientific columns. Before calling LSDB crossmatch, the pipeline projects -each input catalog to: - -```text -CRD_ID, ra, dec -``` - -`source` is additionally retained on the left side when neighbor-saturation -diagnostics are enabled. - -This projection is still an LSDB/HATS catalog. Its HATS partitioning and -projected margin are preserved. Consequently, LSDB still controls the spatial -search and boundary handling. - -Projecting before the crossmatch is important. It keeps the spatial stage narrow -from the start, so the underlying graph does not need to reference and -serialize wide input NestedFrames. - -### 3. Narrow pair table - -The useful output of the spatial crossmatch is an edge list: - -```text -CRD_IDleft, CRD_IDright -``` - -Optional diagnostic columns are: - -```text -_dist_arcsec, sourceleft -``` - -Inside each worker, the selected NestedFrame partition is converted to a plain -`pandas.DataFrame`. The partitions are then written as a distributed Parquet -dataset and reopened as a Dask DataFrame. - -This step has two purposes: - -1. Only the required columns cross the materialization boundary. -2. Downstream graph operations no longer retain the LSDB crossmatch graph. - -The pair table is used to remove self-matches and duplicate pairs, monitor -neighbor saturation, aggregate adjacency information, and update -`compared_to`. - -## End-to-end crossmatch flow - -```text -Complete HATS catalog A Complete HATS catalog B - | | - | project CRD_ID, ra, dec, [source] | project CRD_ID, ra, dec - v v -Projected HATS catalog A ---- LSDB crossmatch ---- Projected HATS catalog B - | - v - Narrow distributed pair table - CRD_IDleft, CRD_IDright - | - Dask groupby / aggregation - | - v - CRD_ID -> new compared_to values - / \ - v v - Dask join into complete A Dask join into complete B - \ / - v v - Dask concat of complete rows - | - v - Distributed Parquet checkpoint - | - v - hats-import rebuilds HATS + margins -``` - -The complete catalogs are used immediately afterward to restore the complete -scientific rows and attach the graph edges identified by the spatial -operation. - -## Why the neighbor update uses Dask joins - -After crossmatching, the pipeline constructs a table similar to: - -```text -CRD_ID _new_compared_to -A B, C -B A -C A -``` - -This is a relational join by `CRD_ID`, rather than a spatial join. The neighbor table -has no coordinates, HATS pixels, margins, or spatial metadata. Converting it -into an artificial HATS catalog would add work without adding spatial -correctness. - -At this stage, the natural operation is a distributed Dask merge: - -```text -complete catalog rows LEFT JOIN neighbor table ON CRD_ID -``` - -The merge updates `compared_to` while preserving all other columns from the -complete catalog. - -## Why the final crossmatch concatenation uses Dask - -After neighbor values have been joined back, the pipeline has two complete -distributed dataframes. At this stage the required operation is row-wise -tabular concatenation. There is no new spatial relationship to calculate. - -Using `LSDB Catalog.concat()` here would require wrapping the updated -dataframes back into catalog objects while retaining the lineage of: - -- the LSDB crossmatch; -- pair projection and filtering; -- distributed neighbor aggregation; -- joins into both complete catalogs; -- catalog concatenation; -- final catalog writing. - -Calling `write_catalog()` on that result would submit or retain a large, -coupled graph through the writing stage. In our production-scale runs, this -path was associated with high scheduler/driver pressure, large serialization -costs, and avoidable worker instability. - -Dask concatenation followed by Parquet staging is intentionally simpler: - -```text -Dask joins -> Dask concat -> distributed Parquet -``` - -The output remains complete; only the execution representation changes. - -## Why Parquet is an architectural boundary - -Parquet serves as a checkpoint between two distributed phases. - -Before the checkpoint, the graph describes how rows were produced from prior -crossmatches, aggregations, and joins. After reopening the Parquet dataset, the -graph describes only how to read the materialized partitions. - -This boundary provides: - -- shorter Dask graph lineage; -- lower scheduler bookkeeping and serialization pressure; -- release of references to earlier NestedFrame graphs; -- independent retry and inspection of staged data; -- a stable, distributed input for `hats-import`; -- explicit schema normalization before rebuilding HATS. - -## Why `hats-import` is used after staging - -The Dask concat produces complete tabular rows, but it does not, by itself, create a -HATS collection. `hats-import` is then responsible for reconstructing: - -- spatial pixel organization; -- HATS catalog metadata; -- catalog properties; -- the configured margin catalog. - -This staging step produces an intermediate boundary, after which the next -persistent processing artifact is again a valid HATS collection. - -## Deduplication architecture - -Crossmatching records graph edges in `compared_to`. Deduplication then solves -the connected components and assigns `tie_result` and canonical `group_id` -values. - -When `max_representative_radius_arcsec` is configured, each connected component -is deterministically split before tie resolution. The highest-ranked remaining -row becomes a reference, rows within the configured angular radius join its -subgroup, and the procedure repeats for rows left outside the radius. Ranking -uses `tiebreaking_priority` with `CRD_ID` as the stable final ordering key. This -limits transitive chains without discarding their more distant members. - -This stage returns to LSDB/HATS because partition boundaries and margins are -spatially meaningful. - -### HATS main/margin alignment - -The pipeline uses LSDB/HATS pixel-tree alignment through -`concat_align_catalogs`, `get_aligned_pixels_from_alignment`, and -`align_and_apply`. - -This aligns each main partition with the corresponding margin support, -including pixels represented only in the margin. It avoids assuming that the -raw Dask divisions of the main and margin dataframes are identical. - -### Local bounded solver - -For each aligned pixel, the worker receives: - -```text -main partition + aligned margin partition -``` - -Only columns required by the graph solver are retained, including: - -- `CRD_ID`; -- `compared_to`; -- redshift; -- configured tie-breaking priorities; -- homogenized object type and graph-inclusion policy; -- coordinates required by optional geometry diagnostics. - -The local partitions are converted to pandas because the graph algorithm is a -bounded per-pixel computation. This does not materialize the full catalog on -the driver. Many independent partition solvers execute across Dask workers. - -The solver labels main and margin rows together but emits labels only for main -rows. Margin rows supply boundary context and are not duplicated in the final -catalog. - -### Merge of labels into complete rows - -The per-partition outputs contain only the identifiers and labels required by -the complete catalog, principally: - -```text -CRD_ID, tie_result, group_id -``` - -These labels are merged back into the complete distributed dataframe. The -complete dataframe remains lazy for HATS, Parquet, and CSV output paths. - -The default diagnostics count rows missing a newly computed label and classify -invalid tie groups by aggregate reason. Optional detailed diagnostics collect a -bounded sample of offending `group_id` and member `CRD_ID` values; this mode is -disabled by default because it requires an additional distributed scan. - -`tie_result` has the following stable meanings: - -- `0`: participating row that lost within its graph component; -- `1`: selected winner; -- `2`: unresolved hard-tie candidate; -- `3`: row deliberately excluded from the graph by object-type configuration. - -Rows with `tie_result=3` remain visible as isolated records in marked outputs. -`concatenate_and_remove_duplicates` filters them out because that mode retains -only `tie_result=1` (plus hard ties according to `tie_treatment_option`). They -neither create nor receive graph edges. - -## Scientific semantics preserved by this design - -These representation changes do not alter the intended scientific decisions: - -- angular edges are still generated by LSDB crossmatch at the configured - radius; -- HATS margins still provide cross-partition spatial context; -- graph components are still formed from `compared_to` edges; -- object types disabled by configuration remain isolated from the graph; -- configured priority columns still determine winners and hard ties; -- redshift disambiguation and missing-redshift policy remain unchanged; -- canonical groups and tie-result invariants are applied after the same edge - construction. - -The pair table contains identifiers rather than scientific values because its -role is only to represent graph edges. Scientific values remain in the complete -catalog and are read by the deduplication solver. - -## Fast paths intentionally retained - -The pipeline retains `concat()` and `write_catalog()` where they are a good -fit. - -They remain appropriate when the graph is small and simple: - -- when a crossmatch finds no new pairs, the pipeline can use LSDB concat and - `write_catalog()` directly because no neighbor aggregation or join graph was - added; -- small in-memory HATS outputs can use LSDB `from_dataframe()` and - `write_catalog()`; -- large and lazy HATS outputs use Parquet staging and `hats-import`. - -The choice is based on execution scale and graph complexity. It should be read -as a workload-specific execution tradeoff. - -## Why HATS is not used for every intermediate table - -HATS provides value when a dataset needs spatial indexing or spatial boundary -semantics. Some intermediate datasets do not: - -- a `CRD_IDleft`/`CRD_IDright` edge list; -- a `CRD_ID`/neighbor aggregation table; -- per-object tie labels; -- a tabular concatenation waiting to be reimported. - -Representing these structures as HATS datasets would require artificial coordinates, -unnecessary indexing, extra metadata, and additional import/write cycles. It -would not improve the correctness of a key-based groupby or join. - -The design principle is therefore: - -> Use HATS whenever spatial organization affects correctness or performance; -> use distributed tabular structures when the operation is purely relational. - -## Operational trade-offs - -The checkpoint-based path has costs: - -- additional temporary disk I/O; -- temporary Parquet storage requirements; -- an explicit HATS reimport step; -- cleanup and retry logic for intermediate artifacts. - -These costs are accepted because, in this workload, they replace less -predictable scheduler and worker memory pressure with bounded distributed I/O. -At production scale, this tradeoff favors bounded execution, recoverability, -and operational clarity. - -## Future opportunities for a more direct LSDB path - -The current design remains compatible with future LSDB optimizations. If LSDB gains -a large-catalog concat/write path that materializes partitions incrementally, -cuts lineage before writing, or accepts narrow key-based updates without -retaining the complete preceding graph, this pipeline design can be -reevaluated and potentially simplified. - -Any comparison should verify: - -- peak scheduler memory; -- peak driver and worker memory; -- serialized graph size; -- retry behavior after worker loss; -- total runtime and temporary storage; -- equivalence of output rows, HATS metadata, margins, and graph labels. - -## Final design rule - -The pipeline uses the narrowest representation that preserves the semantics of -each stage: - -```text -Spatial search and boundary alignment -> LSDB/HATS -Key-based graph operations -> Dask DataFrame -Bounded per-pixel graph solving -> pandas inside workers -Distributed execution checkpoint -> Parquet -Large spatial catalog reconstruction -> hats-import -``` - -This separation keeps HATS at the spatial core of the pipeline while allowing -large non-spatial transformations to execute with simpler, more observable, -and more recoverable distributed graphs. diff --git a/docs/incidents/2026-07-01-crd-id-duplication.md b/docs/incidents/2026-07-01-crd-id-duplication.md deleted file mode 100644 index 363dd2e..0000000 --- a/docs/incidents/2026-07-01-crd-id-duplication.md +++ /dev/null @@ -1,251 +0,0 @@ -# Incident report: duplicated `CRD_ID` generation and leakage into the clean product - -## Incident timeline - -- issue identified: 2026-07-01 -- correction finalized: 2026-07-06 - -## Scope - -This report summarizes: - -- the original failure mode observed in the pipeline; -- the most plausible code path that generated duplicated `CRD_ID` values; -- why stars could leak into the clean product; -- the correction that was applied; -- the observed impact when comparing a pre-fix clean product (`318`) against a post-fix clean product (`342`). - -The goal is to document both the software issue and the likely scientific impact. - -## Executive summary - -The original issue was not a problem in the star-selection rule itself. The core problem was unstable distributed generation of `CRD_ID` values before the relevant dataframe state had been frozen. In large distributed runs, this could allow the same `CRD_ID` range to be reassigned to different rows. - -Once that happened, identity semantics were broken: - -- different objects could share the same `CRD_ID`; -- the deduplication graph could merge or label them inconsistently; -- a row with stellar semantics (`z_flag_homogenized = 6`) could coexist with a non-stellar row under the same logical identity; -- as a consequence, stars could appear to "leak" into a clean product where they should have been excluded. - -The fix stabilized the dataframe before partition-size measurement, generated `CRD_ID` values from that stabilized state, and added explicit uniqueness validation. - -When comparing the old and new clean products, the impact appears small in global percentage terms and concentrated at low redshift. The comparison did not indicate losses above `z > 1`, which is the most sensitive regime scientifically. - -## Original symptom - -The pipeline was run in a mode where: - -- stars should receive `tie_result = 3`; -- stars should not survive into the final clean product in `concatenate_and_remove_duplicates`; -- clean survivors should instead be the non-stellar winners or hard ties, depending on the configured policy. - -However, the output contained rows with: - -- `z_flag_homogenized = 6`; -- `tie_result = 1`; -- presence in the clean output. - -That combination is inconsistent with the intended semantics. - -Further inspection showed cases where the same `CRD_ID` appeared more than once with conflicting attributes, including different: - -- coordinates; -- redshifts; -- `z_flag_homogenized` values. - -This indicates an identity-level problem rather than a simple mistake in tie labeling. - -## Why stars could leak into the clean product - -Under normal conditions, a given `CRD_ID` should correspond to one object. If that invariant holds, then a stellar object (`z_flag_homogenized = 6`) should remain semantically stellar throughout the deduplication logic and should not survive into the clean product. - -The leakage became possible because duplicated `CRD_ID` values broke that invariant. Once different rows shared the same `CRD_ID`, the pipeline could no longer rely on `CRD_ID` as a clean identity key. That opened the door to inconsistent grouping, labeling, and consolidation. - -In practical terms, a stellar row and a non-stellar row could be treated as if they belonged to the same logical identity, or could survive different labeling paths and then be merged or consolidated incorrectly. - -## Most plausible software cause - -The most plausible source of the duplicated `CRD_ID` values was the distributed ID generator in [packages/specz.py](../../packages/specz.py#L1893). - -The generator works by: - -1. measuring the number of rows in each Dask partition; -2. computing cumulative offsets; -3. assigning IDs of the form `CRD_`. - -The key risk is that this procedure is only safe if the partition contents are stable between: - -1. the moment partition sizes are measured; and -2. the moment IDs are actually assigned. - -If the upstream Dask graph is re-executed non-deterministically between those two steps, the same ID interval can be reused for different rows. - -That is the exact failure mode documented in the code comment now present in `_generate_crd_ids`: - -> "Otherwise a second execution of a non-deterministic upstream graph can reuse an ID range for different rows." - -This explains why the problem was much more likely to appear in large cluster runs: - -- more partitions; -- more workers; -- more opportunities for recomputation; -- more scheduler and worker pressure; -- more reshuffling or re-materialization of upstream state. - -In small runs, the bug could remain latent because the graph happened to re-execute in an effectively stable way. - -## Corrective action - -The fix was applied directly in the `CRD_ID` generation path: - -- the dataframe is now persisted before partition sizes are measured; -- when a distributed client is available, the code waits for that persisted state; -- offsets are computed only from the stabilized dataframe; -- `CRD_ID` values are generated from that stabilized partition layout; -- uniqueness is explicitly validated afterward by `_validate_unique_crd_ids`. - -Relevant code: - -- [packages/specz.py](../../packages/specz.py#L1893) -- [packages/specz.py](../../packages/specz.py#L1950) - -In addition, later work in the deduplication path strengthened: - -- canonical `group_id` assignment; -- boundary-component consistency between main and margin; -- tie-result invariant checks; -- global diagnostics for invalid states. - -Those changes reduce the chance that a future identity or grouping inconsistency could survive silently. - -## Why the issue appeared mainly at large scale - -The bug was almost certainly latent before it was observed at cluster scale. - -Large runs make it easier for this kind of defect to manifest because they increase: - -- the number of partitions; -- the depth and width of the distributed graph; -- memory pressure; -- task recomputation after worker loss or scheduling decisions; -- the chance that two evaluations of the same upstream graph are not operationally identical. - -Therefore, the most likely interpretation is: - -- the defect already existed; -- small or local runs often got lucky; -- large cluster runs exposed it. - -## Product comparison: pre-fix `318` vs post-fix `342` - -Two clean products were compared through spatial crossmatching: - -- product `318`: clean output generated before the fix; -- product `342`: clean output generated after the fix. - -The comparison extracted the objects that were not matched spatially between the two products, i.e.: - -- objects present in `318` but not in `342`; -- objects present in `342` but not in `318`. - -### Total sizes - -- `318`: `6,616,173` rows -- `342`: `6,668,608` rows - -### Unmatched counts - -- `318`-only: `4,156` -- `342`-only: `56,938` - -### Relative fractions - -- `318`-only fraction: `4,156 / 6,616,173 ~= 0.0628%` -- `342`-only fraction: `56,938 / 6,668,608 ~= 0.8538%` - -These are small fractions in global terms. The asymmetry also shows that the fix did not merely remove problematic rows; it also recovered a larger population of valid rows in the post-fix product. - -## Observed properties of the differences - -### Redshift distribution - -The unmatched populations are concentrated at low redshift. The comparison did not indicate losses above `z > 1`. - -This matters because the scientifically more sensitive regime is `z > 1`, where the reference spectroscopic surveys are sparser and each loss would have a larger relative impact. - -In that sense, the comparison is reassuring: the disagreement between the old and new clean products appears concentrated in a regime that is less sensitive for the intended use. - -### Flag distribution - -The unmatched populations are dominated by high-quality homogenized flags, especially: - -- `z_flag_homogenized = 4`; -- to a lesser extent, `z_flag_homogenized = 3`. - -This is consistent with the fix affecting real clean-output membership rather than simply introducing random noise. - -### Instrument type - -The unmatched populations are dominated by: - -- `instrument_type_homogenized = s` - -This indicates that the effect is largely in the spectroscopic population. - -### Sky distribution - -The unmatched objects are not uniformly distributed across the sky. They are concentrated in specific observed regions and footprints, which is more consistent with a localized distributed-processing effect than with a uniform global scientific shift. - -## Figures - -### Redshift histogram of unmatched objects - -![Redshift histogram](./2026-07-01-crd-id-duplication/z_hist_diff.png) - -### `z_flag_homogenized` distribution of unmatched objects - -![z_flag_homogenized distribution](./2026-07-01-crd-id-duplication/zflag_diff.png) - -### `instrument_type_homogenized` distribution of unmatched objects - -![instrument_type_homogenized distribution](./2026-07-01-crd-id-duplication/instrument_diff.png) - -### Sky distribution of unmatched objects - -![Sky distribution](./2026-07-01-crd-id-duplication/sky_diff.png) - -## Scientific interpretation - -The bug is real and it affects object identity, so it cannot be treated as purely operational. In principle, any identity collision can affect: - -- completeness; -- purity; -- local group labeling; -- the presence or absence of rows in the clean product. - -However, the empirical comparison between products `318` and `342` suggests that the overall impact was small in percentage terms: - -- less than `1%` disagreement in either direction; -- much smaller on the `318`-only side; -- concentrated at low redshift; -- no apparent losses above `z > 1`. - -Therefore, the most defensible conclusion is: - -- the old product was not formally clean with respect to this bug; -- the new product is more correct and should be preferred; -- the observed differences do not suggest a large global scientific degradation in the old product; -- the old product likely remains usable for many purposes, with the caveat that the new product has better identity consistency and more reliable clean-selection semantics. - -## Recommended wording - -If a short technical statement is needed: - -> The original issue was caused by unstable distributed generation of `CRD_ID` values before the partitioned dataframe had been frozen. In large runs, that could reuse the same ID range for different rows, breaking object identity and allowing inconsistent clean-product behavior, including stellar leakage. The fix stabilized the dataframe before offset computation and added explicit uniqueness checks. A direct comparison between the old and new clean products shows small global differences, concentrated at low redshift and with no apparent impact above `z > 1`, so the old product does not appear catastrophically compromised, although the new product is more reliable. - -## Status of this report - -This report is stored under: - -- [docs/incidents/2026-07-01-crd-id-duplication.md](./2026-07-01-crd-id-duplication.md) diff --git a/docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/instrument_diff.png deleted file mode 100644 index d0787910769ec81f6f8845f8b59f055fdfdc9321..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 23122 zcmeIa2UOJO+bufAmT1%@iUmZXVgm&gMw)=pD2__+J&LqJdhaAriAvE?q)8ExUPM5e zfQq1k^nsBsA|PF)_q*RA<$u@ze)pcczO&9b_kPFaNEYQ7B)X{(1%e#q8jw)%f2s+v7Bqukpw2>x+Nj`z>}SHSClvjqDuH z+Za-eEiBCq57-*m7#dpGnpoNmEiIJ7hjx<>ov<-HZ)a+0@uP~Vxgo{Q(B{X3B0tjY zFa3D<;Nc@b9ug5hL=``D@cAEOS`^BU6xz=}sW{#1Z*f&owVz)YX0`MExHadx&)%5R zdIv2ZUMy5p{%NG1HRN;T$VEeX!{wl?G$Ww!@9@>p#xAS0Zi55$HlE|ILvvEbCdtmjQ~mh|j)a!)#sgC* zpVM=gc1zh`GVL#$vA;OCLx=q@zx>e7$^M7iUt@gmC+O#FGDMVTyjdX@kpleu{Iw0~ z#+`kA{5Oh~Y6r-VDED3(rN>7`9y@>jyp8ofX?fY~BklbS>8sYQOYJ$r{zTo!(Vkj* zbGCJk^^44J3JMAsDegHQ9v;-@HRR{dG{20DaBSOV@$p&24xLad_kGzB5j=}e;<1D; zZ&%RDkR{}U|NeL{C0TN6YHBeI1|!5`=|LwaCzZ)w@==O3PkZCFGiT1&eR%mzKtMp9 z9lzJ};^LF#A=KO6-ZHXTU*R*&N4{psT}W@Z6CAuxU0wZ(x3`e;A6oa5lFk`dhN~JG z#e@i(D)ro6OK#rG`6BbSPQHt3pn$HDhDOlS{Cu-q?q}a^-I|ncT-n=OmuTbYnA+oj zk9swIRU0p-q@~42+n;|j*>=wH%k8p zs&L^#R2uj9-`~HodJ|`prlw|ZZLGhWn;SFor{B-K;!{#mirRQU%jA_Gcf|}v_0#9} zB8+Or?icNdP%qU$e&>}W|%(Mr04Y0 z-1JN+tP;P3CBmjBE1uMCZh_zoP9RZGmQNxS}ZZoG-!m=T|A-=}7wZ*85H zYu6KX!h5aiO6&weBbhk#v1BKF}14hwYi48DhPE z{=5C?C&dRbj~*$sG4a-ySY6rnC|6wAFY-Vcbj~4_1i5`=ZmiP;KMP~tJki5 z;yUf(;*u+F-y8k<_30~@FRO^F-4bbaIw5~__~$D#yXC`^9zA*#+S_APu#Uz_rQbO$ zB;*@`@hY=ez*KYa@T9)l=6cS1Eq9F{w}i0Uq*Y!B`G*^u1iQ{(#N!@2Q1q7C|Ls=M zDvO!UK>hj;#Ww;%LK2RcH=fSjcyHI1^d^pmOtU~TH?D&@G7R#jwACB;SN*zlCG-9X z?^XW$_U~sZ%S%3_n$^E%vDDY^k=w7n9!ulp%b$1c-p#jT$MG#&wlJbQ%L2__{_%Zd zuDw}K+QYMfcO@!u3A@=z>#?D>!sdcCTu1iq+I2EfJ+=Po8b5W;=C>2uX|1iTtJbVZ zwrZ;OCnzYGx264hiAV3pH@7?Qx;I@qB02r(fyCh>hu1ihG z5Fu~T^?iz>oQlewnwlEB_7}^O_42iLp7N=BV%}u2@0W!hc~2z^i|Duar`@v$MnVkJ<&?x%004=E`Nu?r3k+RlR=wx=N~^WL_Fs_89gMaeI1z zz9lnGv7`_)pNM^7KiI6fa_v?H#2ZyLwcRwA`Kj))wuQMiMzmwAvy3m7Nc~D4x5z^V zC7WNlIy?8&JUpw~x)aap72$8#zV}s8(TM|E84BGMVOYfbTkKi70RpKhr?K%6#iBiD zJ9w~{0ta*Z1Y|>x$c9Nc*6S}!vsU@1HC$4yo0t7&)hhB%E$_O@L$P`IRZN!&1Uh%;x#>I)1-{PY!)~~p+zwfl9VR5cnyEbJp zjs%^^zIx(FsBCd@v52mxyINgw@jR=nuC6ZV{#-mB>ou+wA0J<$fjbw>PxnPsrx|dI z+f@bXdzfzO)%TpU$9JbqO-;|7JzG0H*0)Pn`@{T~GC}p8F$B2;e)W{i_j8qF<-UxL zGBKToOr0E?lY8XDRcd2o2mB;$KAwKK?g~k^3PKti*GiDsimSwa{%l2}E%p{RimHvx zW5wIIZ$H3wdJxYT7ni-Ls#3Ue^=fa0#E{C3jR%B{-fk`r6_Nk-*I%bU7p==guy&sv ze#N1e7ruZ^dx|nYFZPm zBhO$ybsphrof-Hvb6Q@Wb)AEM^x4vNd$ES}-G;C1J{J_ZmRo%D(7W-Gk=onxQt_s> zG5d}l)xm_-HRsswq7}{!So)fU9d%Y}bDs!vnHbbP*|v-C+g=gL(#IRVUWPBUMZ{waxFQA&aswARNf6h+_%5c@AXz z-94ZkvvQkLWQ|!RLRI1XSSr<|>c_Fc7VU`^#|TXG$2*ll5=XEI-}=H{I~vcadX#tVG}nxxp$uOz#Wy4}BFO`goFi}jiqXi`UVv{w=lsYgW3ywvdc zR0_cv?@O|~|08(!0++*|-QTCMk=DU${~2Ub*WckUt%l^N0&G^BOfFow_p;eLO`ws+ zOtU@G($WYx`*!a>#dYLTT!zQnphJ%}_B;G4vS*J><u%dHuFn@8jp?z2M3eghbXDrqj9p;W5%3n-l&4)FI2K)$Bv6>8vP#gldq)j zN;n`S(OD^B2s5$RKy&TuJhRWY_4M>ir!GtjE?>E_tg=!bOAo=iyPv*d4JX+*aeDbK zbY|SG4;>x$T8l0a~8&19|2Qs%2~fLEI2+2ULTxHVkD6(Z&C zk~2&o6msR^Cw;6`%Pi0N*(sN?I(4e!ppJb{&B;JP{rapn5A7a%fJE}}d4suULq#m_ z^3F|Fde%Stbtws~n*z-|J9-<^V*$IwsC4FOvXyts-HU6R?GkCsv5n;=_#WGb^QhCK zR<{vZUEpBEGi4o}dlmP@b~-vb(rOI+C1Xe`DO{KvvYebE%SeU$M!8X5=IHVhe{YzN z-QC7nL;q}<7?-i$xi&woZ7KXeV2@xP=pSF-DtYJF<{m4cn`?qm>}@M7{9*n2fxDIq zOp zJJ-%M$BbLl>KJa4Wn*hQFrPIuJjKrl4NcUa>s8mvvIxUdsrGcwrQNx+0o%56Pz#A? z)z@EF#mI&-^Bjj(`OkcM;6M8Q_jL)`*&2g`gN<315rYLY7GdJ{XWHA_8PP@F>k<*W z#64!SLqseEXr-@TE8t0LYHRIZaXa3TuzW~1eWXSF;DY`)qS_o6HyP(HZhx-52M(wL zNRbI!zuv$nFd$%`l$74>-LiZlp}N6+gdR_tmF1q2Yx?vy5txqmh^UQxGWX4)emOio zH9bx5t52TpmR=y>ur5_!pG(AoN#1g<`h$Ambbws=QOzF5l^BFb`ag*74r+tgaF;U# z(P2k#r!^LO7V5bUP!H(ls%*T`@$fVPCfNvBoXJSRs#<~J;VBsEye(e<6i=Q!8N6_e z1I1c`6>SuWr!L>s5fE1n<;>;z@vK5J9n(Ofyt<*8g*Khn^?+l5fa1+qDK5w?W91@k zCS`XIaOm-iifZtS}8WPfwC>f26K>6_OH_ciCF zj}Nvavh-YKE%V3jO3m~O85$cW;x?z;O-Zx3D?5MBkLkoS$nZfU&6Pe1(xV4(7_`Dllik!n`ZN zk3aps0>zIiFnQ@1Fjq9NMS^Og8u$uzIz8dSg$pFA_W;l%RRs=zmod@mDnCCzKg}wf ze+ZoLT2x+6PA*(W8U1A;L7wzOdh$%1ZT1 zmy!q$sg~gpbF)$dg!?!+n1T^A*e@b7aW%E(@B;#Y03^AuPNNn-!}vZSweU$_Vrs!* zx{Zy^=-fnGQ_F--aAf3KF>c z0IU(9H_?8w+>X`#(#7dOB0M6RJc1P8-n|NlGfym95NMhq>^XMBU|uEOY375GE&OF%-ikDup6)^H6s&T!3H3&t82J=r@A9Vj7$=> zD#ImJu~|?CCID$Nq7ex9QK_nuuH$#==ykwGRF7Gwj=5R}6ZnMWMMU}9CdWjJyy9wW2$yutFs|Izf)M^?VuFPnmq@4$ z05_4lr4v21GKiNower%*#^KJV5ZefA(A?bIpAxoMNaXDr$Cyy=-LP6Cf%-%ld;v;) zM2)%>T?t^+L_`-6cfYxr3A2g87RJT>f`x-GudToS#~*Q@vyr8GQP6V=o7{ABcc-3z zv8=PRbFRPAW+(7C=i!Tgkie*!ZbV}om6Xi1={!>&bm#=O8zD$gsXjzhYU0jwo3ioc zk=Da40x@s7jzcM!hCIU|MBAaKBYxZ5M-^!-qg*8YgnRe4p8WaePZ(bo*14|>LQAq{ z`YDov&!4{!ej>WSeFmZYjZ^7lm}6@^DqGVMpDn1-Z~a*yGBK$q%bfo;s!0;A~kkHPUV0 z9p*@HD{$`|>@M(Ylci8B)<5m$%;IXbUBi^EuC8|J3{X#)4N!ga<_)K$i(TeQv!2FM zFOfF4^qOA?4t*v@P;SQeJ5Q-hiAA`5dBZ2&U7p(=?rMi>i1}^|8_P?d&*kMSfi-vgdt-=^MwGX|$d^Eg&e7hXQmF2l7b-|DDYX|*fuSi^5tm^IOXPtBM z^2&;@6dFW!ff4LM&5dEW8_KmXMd~rtD+`VTycm36PHN4y^?UBzx}^w)bH%1jn@}nJ zyz$1v7;UoUEx0b5?fLTgv%!-#=Hn%YlFgaqgv3s92+<8#_}BdRDD9m ze_N&ms;;zf=dLYaZQdZ4{=Q;OO|X!0KOMLWESg|q9baNy!sMirZno9)lrCnxGN-6j zB>2D$h|HHSF9T;GI#k3SG7IfGDccs1Kp-ADrM=hj92qB7Qi!x?0ihG}T_^a2g;m_A zM-_ki=~u~YWH$f6fUhK9CW8nXw1uzHgKbEHx_6(#3s z2B}X;1k9gktp!j42#MWw_U{J zbI9F$_wPRkgw|X0@P<*vy?BYi>=S$B?kT9L-9@GAs|7a1eyCMf!f}v^hs*F>@X*b% zDNfmq-$Hz@1DC?FX;Uns^c#96=W{JauEPMC6%aCfpnvlm29!Vu`TlZ%h7y2JK%gEx zUL8p9Rjz^p0&1Aww6KT>K^h?dL0Jd5%V4;?b_(1L8KlH%WlAcG^`>eS*&{z?AmpJO zRYJh6Ju8?W4eY3nMFMW=E^j>OS%+7`;49d)W>V6AXD6rEZEb}*Bu40xw$V!bE9zua#{PJZSP$Z%B$m`M`YxFNHGfrUtdVud+3m&kZ}b`79cjLwq56(U0uy4Kesb@v5;=k_3@4?R^8b+-|Non0^}k<)XpHQ|cP=5#N)mKD5;t@tP6MPeC=lvI(tP@q`9NkJ z2x8`7_JOOXL@uAF7OH{>Q!kAjM|E60WuN*6k&@eSQ zNr>ZbzWD}e*SdA(;y!&s=hYc%$y2v1oO=QgZ!9uj%AaZiqDm{nQf`2j z;77J6O4sFY*56xbCuoh5ML4K*)$ZkjdQan^74#xqGD-4*hG^FQ;>zQOOJG`*ao?D8 z!8>{y_uE6Eq#9Rj_1z+L3e+Q_Nml)DyaK1MX&iHJj`6%}`b4*${jmlq%+qD|KQ?(D3aX0jT} zh?dbqccsKm{d5l(Zck57E55L>M>Mot&7wcn zLrYn4t}=cq9_zCgdLZ@52-|3o^8WoVcZmoSsg>jA$+Z4n!P`moAWbW@dGD(;b(>Qr zhT2k*uR-g)+01|7Kpngp{G;O{ib@O%rl3s9}IL{~@WadQVO3C5RJ>K<$DP>R%qFqH+(Ag8(l_qc_z}F| z8g6kF5MD$&yepu~2X5+t5j3AOr%xXY=aYgg1kLgFyP@9R1h9%fgXUAsvS|6(+4&GJ zVAlH7xo+b8;{AN-o|R(*mnTR?7FqH++^pH6chWV*Y=XN3lR}{|m;4nn|JO<%j3O&t zz?QRt0$Z8@mbJ32?gQD}YuwC!YWpiiR+3Jie>tFEpu-?aF{v;}0kLA;wj`8668S%| zKS!7FZl8t#FzvZ8v_NC1C2Kug+^8qY0tH!psfB~1E7;={H?Z|;A8z&rE(FFM1|td; z&uds|P%o&SI(4;{i^zSH$S1a&Q?xTo?!!!?5Gx;153UA+13!(#yNHO0fBl%gQ~PqF z90k> zeXn>tb)=-EdhyFTI`gSdAxwsV;#Ll2S4fcpR-01-QJi4~+H_#mlOfn zw~8;sKZ#*qCGPB2sj5pOUFBMtbk&^UbyAbZz{uL3QD*;?xod6F7^GkjmMRuUd3k?E zHn#hhpV?z3hyrQx6n7-Dw{i&KBq)&r)|*ohWJkdr1#(Hlc>s=>e0P~yj0WD`u&0sduUChTGGAiEAwAL?9&`1In0)MJ^X-uUAWLYm@beYadwvg-VcYl2I#( z|1}q1dH3ZJb&a@EutEchFB~d?q@E1_L+lZfv?#}P-&=J}BfG5n=9_&YdKwxgo1U;| zXY!%V=9D3*IPQ?&O2)^I&HY)HSM;&VATRM3?T!2){pmnPK>C=AzMRhYzEiUe4#0g??`TYKQ zX=$kvyehmsh$CuNR*$5nKkY#okYkP>NCrr%t6kI06`D8aeYD#1a zFvlr=+ZL}{x3xlt2bK$vGPhCoAL<$!N`)CilZbc!>~c|CZty0y-4y{`l@%3)U%&U) zBB!y3vTZV0%r@qSAAZQ7*YS~ClF|{p{~ho|Vzyl>j)jXaEY2aY+7e>#{*5=%%+ni$ z?LH|OyfsRvF;=YI8VA%#H(5_U=+&6`!Ybde>~8NMdtG~p(p ztF<0qyawR88$koAO5gA+p1?Y~y@TiC{an^YY%H*BlY;uI1OJx-WHlmGlt)PGgUn_T z55xmPs8A?$s<#lUAvt$qX*C%Ewjn zK-ySXScDvPek|@d*w5@@|1@s(R)N(MFmwPHmHH9{1q2{xO6;)|uiTgCv$q|&6hzWI zHdZeg*)&43oPcqqEU(cTAM2ul1b; zi(W|_&R*SKO}`eoE^@@reE-e1z;U$?W6mccqRtq_PDPqITi5t!)Wd?9 zewSuIG-O`F%%jRks{bnp^SNm`VX2M1Gw|;zm>IkcRugZiQs^L6DHOh7#JEKo&UYLxinB~|Hj4o{{5!Z7l!SMii(6ygzRCM-oQ4Zzy`2? z!uI{MU$t}JK}R5x1Ku4fkvup+a+S0;qjXbP&X788L(g;>{d6B&qA~0Nm>yz~fE(^( zN5=ywSv>$}4pi_P;7PEo0_y0hkTL{xvVT#xg(|sbtEd99BhnkOz2Gq{dPdlf|6|L` z+81S|rG!@7waE-FQlrZ7WY|VbLC6r-9~BT{=dr^7kYKg;14vBP%~Jzi)L7u2CxG&5 z$NhGP6lgi3h1;jFA9MI~8m?E_O^ zn>!19PrIjI9kFPR!}P>sM0@L#3B+(1Y>tQP&jdCV=*yLrbRc>aBDF3G{SRR9+>4qC zDEX8x=r^JV0s7P7jUifaicZc2_4OM!#=r|i2Y*UPWXRJOGqngWOA;NFHA3Vf3gWwX zq@q)B3uR}Eg4#7hg$#R#@fAsk8Hw51@7`5ZRtDOcI}|1*^&7jd!&^M7Y(M^_#60GS zF`vww*taZV&Lb*adAMX=^W>d<$`)ceJpY##%0+(&VQ_L1?H5P{)7iP7BIYqu3US7 zqMGodL|%+`_I;kV6rJyZRk3BKiTTh5=A!v*ED)!%H9&Ox;6dIbpi@QI`U@|=CXDAh z%d~e3-&7c-2XM{AN7SpREZEXL=*YP);|<2X1E{sHczZ+X4l-=V9plsl+IZ}BmNB>M zsg5QI8po@rwyv3w_?89H5pYNuoMwxYmR4h>OI0wQ9IVB5TJJyNb$NEBINldE&*9$m z^mJd%F7O$UE%u6tuq?*KB_y;VAg3zY4mRi5up~X_pSC!Iay8GAMr=|~(b*^RV9o#q z8LSKYMFu~7*s*DocjZ-*QxfC`3M4tQ4cQEO<@pKFyEUJx9}sTB9q&Ol9oe&RUz(uI zis8y=xCC1nrbqY(O(}aC0XkT%KhB z3O|IhNmyT(zT_-k8lqr52q9=ByJ*)T>%(h7yx0(A`QRPH7Q!bVXtc7$w`{|;MVZAH zs-c*&_2v~8`uc4V_GthKM5#x${8%fQxWga>`Fcmm!kB^O#3|vB1a7$Q;n__Oqod#H zTK@OpiYL;gHepUF?*j*7T5Wx=4~G*rj=33MH_38)R@x5~7_h4vFLAGbtas!$DCcs| zXBWV;LHwJkh4Z<&(`;ZAS&?%-+W~G#uDS4;=(J07IDl&rX3ovPG z+iZ2wJFqYnu~1DcOz zDl%-ofWp)ind3_V3NZue0(FSN+i>YfER`*C9@G7BP-g*lR8pElSc!*YlX&$Q(FCEa z-+j*NUx7TPr4tqiSX|#UH8pMK0p!3$g#A<7xu(ERo?+c_3Vx(_7Do>sRzk5WXSN8% zb6>Q-wTJfI;xlI7=DCm$jpYAS)sSui#1xSy2R7aS7!?NNfbyv}zMMUF3Y$cAHcF9y zgG`BRuS}y|BXR+j7gu7<0Cd8a5Pnd`Z>zP-?ZtUFLJd+|TT2WI zJ2lQS;MxgyAJ=>s8!PcowbY)AGz%tEBWD<2kbLzw5R3+RA+J3=1Vw~Ygs=R0)KR?b zqQM|NjS)^jPEnT+T)f$ZjiKd6Gd+y)*QDR2)@PqZCV3Z~9voktbb0Eo5PN z&|);`xl1GI;w&%a@v5bFy{bOJhw&X=)ANvS@<&!6F1WDQcl z8h}O|c#^7<69qdGpO}~?95psti<86TKqa1S-NCnYfh+?o8dc{L>IMQ9&2XfwyZo7D;<+ukV zRFkX$cn-nXlD$ftLA9KV0V@U6PONOG8n-+9^9sgu{eRU1)PM>d2&V5^%yhp;*#${J z%~^Tzs=ETN_Mya7bEyJa8Y|(ld~K92_HZBjhrJ9CIk32u`4SEv4G{^%2ItM=q2W1I zrvEqX^jx;Z)*bdFLURr@X4ZmshOZz7#b7+lYZZe8Pf@(IhOV}YLu7?bP!Z**9eR{V z0<9+YRBZL$-akKw>~4R(qrTMh8qWh!kW*^eJT4 ztWRQ3cbx-j24t|(va;I|5&E)O7!@yl#y$~`^mPi|Z{u6my>$Q>4a@QPcuNL*XK^Z; zf7B7w&CTF!V!v?YU~YUYyBmSxRr+r=)c;iWGhVA`3@>U208S?(@n6x8EZG2>P+G$JX~exp*XEfpEzlZrCx9I#DOGc)KR&_}&X z=pp5DVwy-)7ewBT!%BfMDgyy5_3az>VouQ-<4j?pB@bP3N^ppiz85&FAE0~!Q}`kw zSjs*3@#DwB1pCd6v4~=Yq-59s?oi@xG*KbbsY8z-%q3DYsci5^tsLxbCL{vlYQ^rW z!K%9x9&Ry^nvkFbi=I6?BZ-oQP)d48s1E(_rkxi9mHHa*LnYb0#p}4mqi^2)j(E)o z-Qsfy{x=1f2uK>C6(%cr!)gGkj_Bb?vK!E|0oVcmvMd8!g!De7Z1lsZK$;;9p1^5{ z=cLnv5sfeQ!7@OsB``+9-$=AG^mOe-TN~p}SXlD&-&aBe{2A#Yu)px>QymyEF{lLZ3F&%l;-V=Kun>|Rx#uj{NZx%AB^W9S3Xxa~aPetV5mbq@j9t|M zZxRDAY|UYG^a)@WYMZ}%^b;z)#%${caQ7#pltPle1Dff`rFW6<-f5UOXDgzj$owsn z7icgpD(X170$dx~58AF@Uj_GoMiWFkMs#m4T~%G3Rh>ton@$K<1E+fM)$n+Mx`?tD znTJlHDy>H8vOwxGMl>FdrUZ+nvN`_aF*_`iSb|n5h$)bcw(0BWB!PbBOx^Os4(MuRB1{nWa?!VyGz^<776KpYBJB=1 zTk832Q8Z(wRJF8vGh7JhlnWErq)IQ$+WN92K|j|Xg{Ue=+*bA=I5_zmjh$UwN^5J= zOR{0qYi^!YBTJC^;Od+c-{(B)=sElN`QA)d10TXik*1*qSW)5eH;bKvo1c{EKzeI{}t$7-pZ)5iZb@(xsB_KI4E&S(A#)a2Lfk zXu;n3k8EhDp6sHV;{gJg&;KL_D)xcj_jnbI# zQsC*dGKpy(RVpzRqxXs+CXACZ8! z5EVWFaG=NV9zK~}%!Q$#e9mK~mzuWz&t~(`Kby_p{nc##U+NN9uP!}<(*{Dwxd7~3@XP-m z7s`KTjgU+VwE`m5K4g=%-Y0lGXVYpu417pW0jrc=IJrsFX3Gs1pRfuWChfWeR zF+V$fj#QRKL4{VG&Y>Ye7!y!1^?`ViqoP+Htzh|tdWGCa2kU~!^7QZ?_72ESs58r! zLL(rNP@pIh|yq`%S2qHwFMefxq^vL()ktV$WNZOiI zWI=)tiA+HfH)=m+@S&N72?=K)%rIes%kY?UL8n9?Tmejko7CpNFvMB|AL}=(CAP8~ zmd6M9d$Lsf@WSxR68A&EXcM135)KG}7)arX2#R31h&lnTmY0U2@;1DC#IBB1w-@3Q z>^(^Cx6!WrBaM84CM+Ol0T}a0k&GHp-0Hm-5-BeYuR%f*=qB`cIcjz*=;;Yy;NY^> z9cSBF!kp^cd)Ip9QputPiev|jV@6yN6V0~%NV~Y#W5XMA5mEyx zq}?4up+!X(1xd;rNxR}vq=Hv;OfZFE0pIU~odJ;?zf#5_DNn-Xv%1)8iCm(fIW-t) zy6zR;29E}oKy?EACkm)Qd_&S-fd}P>|D6vjN6mR7PYKvt%I z`S^xfLZB144c!bP-b4yaJ(*2U)TZ<@(-=RfXIUwBdu-G&=RB9a77YJrQ-Y0?Q#Msk zN$EoJVeczf;vYX&0&6ap$U;gbES%~HoW`u61sE3PMo4TG@fXn51M z%vWWoXlllTXE4DeGl=P$bW=i7AlZW#JLEJmP(udVMT5$O++GXPgY*ELVJzlTqvfBW z1>}#`oCP8D__tZmKWL7RzkK=f>hVwLl9!jSJ1Ho*ke_{2PvG*toT@&ntF`sv&`AD9G$4NEr=TPcJS1}~NuF@j(@ zsG{)CS%LAZRqnL4s+5VNJ>O zBd#U1AyMKD5Hs;yQ$hLp(CY zD}^9t1b8e~Y3kbp-7ClHyoQLf?p5|#l9kVZ30oHDp_a;%(&p5$hV(7tXdPvUjG#=qd+)NBs_$DWH`{%K zY)6{b$$1yV$rBILKM}POwb@q@-odiVu0Wzhcq6*&J^=waNOo$dNYI<#*IU1MvDZ|t z6dXX5X;`)qrJaQDo;`cau&)Jr*}8SrW~#OQ(iGSl3geI}@87>K*Ma{)5lI;f`m0n! zBN!tZByR$OdMOoqtv~T7J1=z(_O9^CCFjVH;7lqyUW0SVGpP+$?stpYts0M%FgHt> zArwPNxA7eYt;b(O)`Zpe%!L~qCNGUvX6OE1W`lYhWj||kp;3`N%sXHP^$Y@X;ruM2 zk82X?9_<%N1E>|b#cX~?f8wTZKT>+-oSk!!cr^VL}L z6OO+0lm@E;uzhr&*fvSD1Vh63`@N|cvI&gPJs`nZ0eIj6R`WeG*Wlk`d=KcYtnnXa^M zfFimQL4flLenSg=GXx+x4(S2L@sl3Xpv6P5?8I4XXY{#)iv(%mdbvm z%$!{6`w+}g)!w79DdX^o=)tEWa!6rj5Y;##&tRA-k$PC8rYLJe;O_wSNc3iPJ?E1+ zyvoYn>>1Txe)Y<#_kb-uceO35VOY~`A=^`dKYY<7_U!N4&RScPy)KTV*|522um86C zPq2eO)Nin~@)n53)}-pk`Y zdW*C|*S*nMtO=~c-gchxrZr0GWyC@?+lLonnE^YrA~Kiu%4jV&NuI z1dH`DvmC<>;xWZyDMjIx23|^dZ+A-eDH$2F(mVTzDunu*Ko`iBt&i!5S47@~+?=hT+B9^z_PBMPztxYlq<$y*d z(m;!2n~3fYXJsb5Js4!tQAeCV=>U`OzyzF${vt0HBO)9?j&$oPN! z@ff6VB~{fr!%BV{R0Bd?!Ayi^duarCaz2n8)gdKTWkl{W9O-@HVKcVlk_UiWLAt>2 zvIAYl#1l&nxlooDES!Ce^8#?B9XXK-5nj2l4=ww?Fa%2cF{q0+~ z&Lat&BR_x#%o+aAbFddF4}je@C+A(n>cNWi7GBD7{C+v&v=0JW15OM%R1yR=J_zj) zLKk0uH-hfoET!f~l{*Fo%v)Xh4kX5) z7T+x(P?4mYSAPkf4kXba*k^)3^kKZwd?}1&NhN2#z}<=BCFv0I>25KxhHp0?eAfRi zt;lVjXD+EJ^Z$xZ+<-;vezijz(%@HW%Z_x-G1aU7E z!9KV2{}CD%0Rk+h>m>F!RL*wV5x6iBrWk~{n)!g09@%$hH!e85DKGM z=fM4%7to|AC_01|dEcV;gE+~eBq-#3zsYhCpe70;tB=q=vI<4pm)rWaw6!lE%wO2bhs zRncfrFz|s2?0@SbId0Ev0o&;U`332PDS3mpmhD=M1{{r|DufgA$kZZH8u@}=2m%3t zSwTNf3^@{M4z|=Hx%TNZ`ju$>tc^OkIvqzvVV=nuRKp`9LTG}wNyx*YH{~eju!TVC z*1MFW5K<`224h_n+2S@*UhgTfbuF~{cnAkImRZu4O;2QDry^1doM!Htwx{405*2;t zAPr#1h=yZV%$FS4!Y1%BgP)2w>8rpmn;d~dmXO$b$T4j8dg$oHu^w98n(7!`^u5X- zz!5}@j0?r9$S=dE9jUB~>Xhv(>T|=4@6p%S?{gz1EW!zAR+0W|oIYiYL@F_6Kdyro z6arCTVt(F8&al9CNL)!nCwjfFNUMoCBo#j70C+zgIcFqC260Fb5j~YtyDjzK^aTSb zASTHHT-M5pv|P+@#atsse;f?~9;%-~Bd&fQbt#1@C{^_>h9zv_yaBQujyGdOH(BJ# z;zOlx-c;j^9!p|!!&0roJX%$Yxx2a1eNgki+1w%QFlhTMbF9NkR6x|>$ty%AVvRxH zQR*ReNW}tuNBh|R1!!DYfoHK5Q4NP601G-d=X8?{gu_zm<8qp%HFWxQDhpQ_Z!t*P zCLrwH;#}$J+vu04#KO7{bNO`NdlX5VAA7cE3`09c&Pf#Qnl*mD1W1qMY&zs0;=sb? zw>iCELVB2CZ&1WR71$ZQqra+cS`WzyJ)UNHkcjYZPiFE4o-~nT#&9B9*SboaaGn-?`QkJI*QK!Sw%&44Gk_oVNGOZX7NDe=8bPYk8Phml*-2jd+% z%%tA?OBnj3Q|irDP~!BPH?F0aWFX8xM1v8s#aZ;=R?iF-rb0+DJ1K~nCrci;fq>#t zokLF1grr7HgT(KJ4fxTn?+ecu@h}h*0iiUZSP{04EC~c)0arhG4v@K%HPVg~od^p= z(D|I5T?Qd*&?X0;W3d+19>yW$e4+qhsAB4yyy2p8Co7vg5RwnO3t+355@QtEB#{`M zYh(S19RAyHUlBb}UUKXX;DS>_x7~I~A>^l!EC@B}4$0O9nlm{{2yU6~>lG3RaKw*~ z?r@UhF>JXBCrEz zb8OjSn*3DnHM~dmcuFFaLLV!{ag=?d=Z1KQL7#(P4bDAv$7XA9xMJcEj$jJY9p6e^ zI|Okz+g8Hj!(5fSNeF}!HZ{MLlpZ;r1odpD<)^CqpoEz)U(h8TzyqTBr5>IJ(syHq z7Gq-49c@alK;2S@43vuUgCE82iL2idA*b)sE?0;E9+6 zl2dP>doy8}&Y$WEB}PO-804U!I=1tfc{_z7d>Fod#q;OemruyZ$RyI%vzqTg1}s=T xXjYdHfOXcJhPu6X-CxoZxjXsuA8Sx^(l1Tl4fnbvDUi?6PRjk9a{QOy{tuRD9KHYm diff --git a/docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/sky_diff.png deleted file mode 100644 index 6400317a72e80e1e8ab9fe78fd76b53d98a8d835..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 34568 zcmagG1yogA)HZz7tKJJL*8mg|R6uD&KpHH>LwARCN_Srr0|ewy(%mH`9g1{HBdK%< zN;m&pAosrC_}_1QXAJcS=j^@LT64{Ko@cJt5@Ld<@ksD67|dzx$IAQ(FQ-4XU%~$`SqUguNt^0g*=ktmVDwB(pXxAJYFX&$m{{tY zS}h!_;DZm{Kp%Qwp`&4CU}|zz#^9+A#!AQH>Mi!GhStVcZ{NCo=PENh7xNvi+jq`d zrr*b4u41qc@5$JO&kfqE$gJ$v{#rY7q3y2uF=heFEf1xlx1l%QO*eO{G!O;Pq=p#K zeDEI}z-u=VQVq@&)-5pFkmxWXkj}m@+;LyD#Lz(A97~ZW(R4b5vx5KC>$7j~O;(C5 zIq;E9o?YVA<6GinnF;4}Dj(K*muP^74zHhPF5?jgZ!sfyc!~#af8t$$=|Ha$LTc?J z`|m30ByWHg=74wRj9k89S7*7i zW7*n}qleF6p?Tbk)0Y#wGnDCW-I92rF*Wk>nLtf*U0vI#FMsB2W{KFxj~_ifJ=@!2 z9$2sc{6^QZ5@9<;vm9z}Ztm&jWyv)UUzqt4_cB#B?MBtD3*_WX*v*MB(+ig`r_su$ z*4;Wqb@QhB>dNZEa8_mI-KRsPMZH=lPLohp-RjKM6Ay0DSeqXt;k1bVT40*h5J=Z} z>r{q%MR}dpY)=+b)h)cI6p!9eFRpuEW}j){UXW2xc<27?Sy!RCmS`kT-}F1N2rkYc z0t_ZyDsH^2ohTMT8`eBwrDDMMpo5joc3vglWKbz4CMH#_tXNY^3nrO1X>|2R zf08F5ogq93)x=+zNb2a)^O&m{#V9r-xtQ#1t=gHitn&7L(~68&um0@HR8_{YYS;Z8!D;!3-09~(pFVw( zsc_k8tU>qb$~ThFHytjo+ewqlXom;BKteKOQLbEI(gpuPPhdUWEV_vD)+sUePrlFxx$~O!=K|q%N?%l=li3!7B->YYaD=Kb? zg(nBAIn+Ph2<};_4rMcvNf08e)#9)CRI$ZxGxyGp36^b4g!yO*FJ1B>=U4N; zDc=3+EX6X+*Ua|D(q=9HZcHSvqhVi;&P-2M6lDt5{{{(%S#*uhm9*2B@2K37$NYSJ z1LNEE;i+=2$zZXV)}65CUrR<=Wx7JQJ32b>d2I)atrICNhf3|mM!kp_Zc~MjyR3gB z9M605(OpwLH~Nt`NyuP{EfXu*-+w>hTkX|iEbE3lieM*Tl%4~wi(}YaFzksugN~-; zWa?Z&UoysOhIKMV#hlLp+KGvYME2=v=B_Z1T#NAr#wg{biRNZOGW*X7T1PNuNwJws ziwQ|dVLd9A-W%Uul854&B;y`&{@{1nE;K0U$LGz03+g5a1vV{BwyH~tp`GBeZ6}Sh z-(LT9PHTC7us>Zo$!B9WtD<;0^my6c?iP#&rbBGB>LmgD&h~cGLd9PB=ZgZ)FRW)e znYnG|rdVy}2RL(u0%<2yZ3o9$%dKbSq;W+C9f?wt1ekH}6c4wPX_9d-jvfuaeA8<) z zIu~Dn+3gvu%yjvD*xsf&R#a3J%Io++YuiG%H4+&)EO>t1jyUg1fMHK&#+Nw3U^~X+ zMVn+nHKq1TRufH^c-ICj8L#l${d~fpl=nDEJj%wA5ra_?lk6ehctJp(nW{=sv zbqf0Poi-NFX1JR;OhOJ(_yE(hkt!x^|<%*#;1vgHhSuc1VuBcRElhNqmCu}{{b}vJ@p!@k>=L3R*5{F&3MEeU&d+?b`qvbM`rdMG%&n>OZ z^~Eu&7Egg)J4{B{F2i-F;bFftGF3SBO@iYQ3S&2wd-(7<@^#H&>_%W6=qa_XcskA( z)iS$q!Y-ErKj}%&K6G2P9Ui z`o7f4FAvvV-dP)-O$;-Ux_j3RTx!BuibvGk+$9oCg@qj8TM~OyVlbFOue2UwxNAp` zx@*QI7TthNHYsUo*<4-Gu2k9CoLc+M6%WdM{V(@-cX#m!2zs6o@G0m&{lZkfem~KG z_V#Tl0-m34nMD&M$B!SUVPwqw+!GWZ9!|QkvBBvw{f-IZ6mI!P!7!174_(x^cg$LO zy3HibQT*zPii+UPqL#sJ&UB_o3+%QtF*1s!$!7%_gi=kM!bCo2gQ>r5G4^<8DQYi( zQZ`*d(zJXdkqkT=ohRYne@AzA%A6+UkYFEn%GKh5?=}W8&{_SmLN1`5Rn&R4PtRv1 z+F^MrW}-PvebpZPJ(ya=(PPJC^9^Oe4Yy=7@r&8n6?WK3-ezI39;rNXnw+n+tsJeS zFVEtuNS956LAz1!U~kKObeTI(zb0!`y=Z7?Xva|rFE4k98`b(;+3cy<9p{*z`VzrDV)ca>Kk=!v(mRGYEU{L-b1BP0@yJ;jE z!6ggs`0IzKVK9FbGz(?8#qPeDfuh_X2IWK+y%zoNpI;aM_|U)HC9gjH;<5?#otINn z`hCWC_O~v#jnx~|fkH{#GMM58v^W}0PNfj8*$g|W!E=h>WHOZ4V6FvcAeo4Hm!Xv3 zKG&C7*HN_Mx`{YY|M?9yP}d znC}VdA&ALp-Xv`8V9QL7dUgLCI|-==mW0o#kceI(I#D#t5IIAEA+MPWmg6ic@1Oir z^X5_L(XfeEAxZO5Po8I7lSJ}s+(L1e8_l(uHn1M$Gkmm_ts%$CI#|B zTs-$esWdJCeC|s!hsSJ2-So2Q8%vW$a04M>VQDZhN^HT9LmHROTr%W~w49uZV2Mnt z8x`A&Ue~W*$MYs;)AfKMVQEW)deFXC%+;N)BH-_Yt61-D+hbkx2BIMGfo#%e)rsyD z3F~7P-KI7eD5*y}%+l;jau;eVzg|eXs%vYFAaHvmTAnz5oU`RT29p+G<(zFbErnwE zWp)#okEHO(Fn|9zh~23W0zHb0U{j>I>};kW3xF`8`u(d6Zc)HmlEldN)RoqN4?omau)2Z829Kjx;g>x^4qv)=`by!cb(9cg&zciVdt}|Kk0S=FWq(rZkEF>|d+8klStVJ~cZ8<B)6RT~01&l;;SBnl4yPeeP?%FhoP- ziaKo5D4Qd=8i-RehvkJ{=+ z1h;K>Qj|+-lE-Br$wE*7dv$EIFaK^~Tej3t=L^VW=(xDPHpsw~jJ3~tfi)G@Ey!VK zI+A=L_C$1B2w}03QGCvw*;;}b3fZwTs%?=D&XVEa{-)r=W}9TD{U0(i%T|>V;^R36 zx5s<4wZ0Zv>b;(|B7{uY8McH7A+1asBt@BZb&ppThATc#s%~DI$TE_9Zz5g&YZ+fM-V20;39thV>v2?@BhLtye^wE2Gq4V4D6@lo3&WQMOWM!3i|G@*V=6bMP%ITWRrk&CEo>}dz^l%e7x`JI@ID0lmFUq;w zJUl8YDp5Qt({Z6pzRYp0Tg#VsKX5IM)u+O$pF4Lh{5#2unBHE6ByK|A%j{y12}&SO zu>Mn{Y-hE96RLnbsNIgAJjv~2xx$hzpA`woXA-C71U2LxwnO$wj4FlE9{9wD9f_i$ zJoXkbDN)KFjK7Go4~|WhAH!s69Hq83t=RcsKbuhy3cml=v7<+uAjb{O&d#2w*j>rG zUke$0D9&>LRyD$B2NLuqsQkj9@MJ%B>{v6no)E~$qKX{XOoA7`<{9|+WvX(s+`85F z=GCiPQT7v;EL*}k6De7Ec?Zo^jE#+BdVD-RZ#yneNm4_a4&H>ZY!^b7sH>~%5*RTR znexCavKd`7v-cr^f#Vm2#l@LcdEa79QF}SAA&dzhA4l^s&kTsg#G3=~M2SY{oVWFjMAZ6BknB8-YPA>hH zmZoMCSmI&*AUj1eyFA^INV&ATwO~u)3gu%HnifJjIaZ_2WDRfxgA?Gzm8aN*mM%qd z+qO7?4K!UgDg1cqufLenWYcs%KrZIAP_`ygGN?wK1$9YCR>f|y@WLgG-MK1OSM|!{ z7@j5wO^l z5Mk3%g*e#~VfZE1BOyzc1pH4AWTIAMb!WM9Fm_kE>yD0OySuxynty)?PT+@UBI(>v znKBd<`PY(xV>q2k-2TN2we1niR}}1>ybm zm!HPP@rW}Rj7|%yXat|LlAoX7;?k(Cirlx*W0;Xyr;X52Fu8{M`fN&@YEMGK3m3Gp z#^Pg39_d=OJ}FQnk4FHAH$GqIM^U``)u@r%o1Y<*6N9N6HR>UqY>krNIngsUc`A(R z7sd7K0g;jOEY!gj zf90p;gyAdBdrw?(D*uh0J;_(;U<{kyeOF#nASZupe=gj5Vkr*rv`S5z2Nr7<3pV88 z30Og*!#TV18=xb~t>3vKGe7@x@r2dC_*|Rc0V?+-TZP5c(pE(M!Tzr{L)`QE=%wTF8QEG7}veEv+$#iaz^oj$FDh8-fz{+EAh8WSlVgu2CrzylxFq347;U+?5$s^h+ETAix?ZIH zJ{UG%vFC!70*uOPs_hyT71eaiOEM{Ng7Kv->0k4WxFdr@LKKsQ)1m>oJ$~YZQWy`Q z)As1m+}w>5ua$%?+}3CLVd6b`8lG$xin+xlOR?7BVRakFp_dS&j|n_{s1InE&kn$H zU*BsyGbtJmy4_5;Klhk!^e9k$Ob5!VNtQXRbfzn^I4?D`@2B3UE?uWF6+#u{x%1}{ z;LBz*pn!56nH1cKQx_##`VO||_BHt<4c3&d;+u1rM|=1#J|8Mn3igh?1-Yx9b>b{baIPhf-ot)m3JXgjnh2d&yyVFh}8k z8GGL;epS|i0Y9C8pV5x;Klfy`N!PP%YbjFjVhQhXJ}T)BKl+4uUTKQ#fBAW~N{2+I zK_Tz|aJ$)Fi}G#*CB0h$1&@}_9F7CV{Gb2t5Wmw=w)4=&zk29kLNUG%|D7(z2Ju~! z`J2VmhY$BZe`$=f7%Y7hBzmWyX6b*656ofP+Lo65ekI_B4k;g^68+7_)@EsZrqErq z$(*vR4vZ9&_1|5MZ)^L1TYXpl0&GPsp@La#$J=~A++r}%r2kGswpU`*aU<;wHCNn1 z1oMIYl%9joRoZZFDR5D2e>6frVz6#e>Vr{p!6h2lXk!9|2&|hhhNLmujv% z*0LE&UFJ156>HX`{aQ@#8?5ZW!UCH)(r@s+`67y!5 z6}*PPD8^#@#+88_A03|6Hw2^{(S5mkY38Ff2)0Ey06@*NgSNwglseklk8wa9MMM&l zkCSa4;!N%~+)m#wie* zWDrgW75@*ba1D;ulWd~7Z3_1w4Fu{ZK#SXA>=o4pIKJT;NPjSHTlRA#icq!7L4nlK zyV9ec1%MzH2Y?LaxV0{@KK|t+ZM|k} zSR-;=16O?J3YR|A`eIrDiAc-H1W=~jdv*errw!(bbj#AhB2T00#GpkZgH9;j!6c_o zW?xZ+6t9b+U?BGM=ZAw(cWnY<%%;~u+8n|nC^Z!DX?@*}>c)*!m@mVD!fbHnFe6l_ zFW!m{&pj~a^=Hk?{!w8)d5hI-IXoZU5`sS&Zw#ipcI_V2kW$Z1oDK*NPY3ji%f}uN zSZiP!yh%Bw0PQt|l3hwl>cY8mQa&LE;h++aOt}sh59xU-B-_>V?57@_zai2FhWJ9c-=op^4zK|L|j8m7`0Vd zcLqsXFaM9!`CkX!r~TMOEqt)%(6zhi{g*Zpi|}uhfb$=o6<;i4L3B2(o&0~lFG{@D zVYNh}XyP+@`hRn^xr@2lbi#r@q>_L<0Vr=>b(&ots&1Xl&2!+{qh&TpcJc@4w_MZF zHnuAuExG1e0A0Mr*Q8Ns`p{vWBz1@nYuT~rC zj)?fdN_O#La%E-Z{aV|FAyve$-5CM~jn!&OVjp})vjAJgaCNRvNJt1rsR~;iC}P?B z_uU~YA+n9|Y&@L%pPE40$et}ZdG}BY5J)Ras_gP5h1CLf!PEXb@cU+iQea3o0d}Us zCK}8@9pC-5U@#GC`4XzN_Wu@lQuKj^rJ$TkDdZ#6tgM~PDl+VHSg~-9YhlfaC;|Kh zf9-n?U<2Y28U)C?S$ftA6o1bC1Bg748_9j0PlE(4`MRlYUf!Bnvw z3mglOH!e~@9IgDSC8q<3AVN`{@bEHxSAma7wBMNNfg+O4Vyy0VL-O)rwoqxf0V#xa zywVQp^M~S&^n-G&AggG6(20ZLXY&kJXYkg*$2#?b*%s2Lv#b+Am}CcYyEyV09}cQmH-+C4e*qAnm zHN9r$fRFSZ;`OpELbHrr7Fe;1^B+lVmA326fAS@Q^vHEZ9t*NT2 zy0Ybo3kZk@0Sd^+qXtjt@#8iC#7X-BP#Fkff%=RO)p>UnvBJU~5Nuct zJChg1sx6L;N8C04@U%a%sYwJBAzu-~y}U6QWdzI&l4~3mRzCGjP4t}!vJ#**fua6a z^0^5sWG50DY1{iCju%NJ5CYl@IA9!_XkZ76=Zj|%-U%*?1S02(0B$)t*OUKMC3C^wE?Z~kP3Z4Cx z#VFK48I@`Ge~q<6DF+Lw@@b7Ld7wEBPZ7jAwhqP@U)&IR7yZ}y8v(*db_FkLZZ+B8 zyXU8nT)KhITG<&UU)8xQzkmJeFnod)TWN2MsNzq<$H~9+Ol6fn+FKOckLj5g4-po* zIEqGwLnr-S$>mz(U|2k;Mg#-}jllbiJK?(HAXj&~s~%So6eirjy$?MQJq89#Utixo zN%F?Or4dLHxa=3>5$&m?tJ`2w!2QGV=f{((p2#~mVFne@6E<`p_7UcN2CQTs2HTc= z&T^`1{5}!TJ^N!&Xz^UhE={7CO-YJue|znL)#kDR68J!N;`0M|6l+dEp$x5U?L3(DF{Xdo~!c$nJBfub3 zep%(v%jczuUz%5~S zwV=WP3JAopB5@nG9X`QkGpB&4D&#|i|C{cXb!Wig7H&DN8Y89*X$`dIGE3&$PqPn1 z-;A{T1gm>kEnSjY=df@8&b zu#hU(bN#q*UDPW%w zgQmqjh)I1uNa4RwB|ju{*Vbh9Ht(~v_5xFNAlAO8j=n#95c;lUNea;EDXP0{Bq$Y_ zWPrYp`v*Dffd8=^%>i3UM;*RsEhng|(-aD(#YS&%eXbNWCOz~OKxUcY_&H$087^XBrGp)$t^A7)W$C?q7b zwc|n2KuAoS1Q}+q^lv1FS_ra8kU*sXivwm_$LzB8qW~dwjI~ee}eGJ!27B;kjwTv{WIlC0m)8eIpAgG%5AjcNYYn>VLW%4;qH z@d|jW72tD;E>C{)z;9@2vH1#@8`wt|M?UB2FP($Pipb|tbC+KzGeEf^JnILFw5uJg zDUOei3EPblnFObmI9GfMo7pL%@tf6#qV(rvSGiK&*#PU z#~T7Sx0hP2fobBZ0hZ#hz-Es8KOxd!5Q4ZBPQa}SWgzRxGyC=_>Ef2R)1(U1wu) zT&cpP4B+#Tyo&$oYoxNoN&%H|$GA6|MmjM-@HdVjkgF+5$D~9dj5EFy2Kz5_p?pgW zbh=Pw%Ok7)B`hxKnSsj#hlTj?zk+)0oZ z(bCZgg8|CI4Bo$gzkx$}-^$z?>(qN12ARt~zt^Zj2=CLuVp-tHaY)F@yZy5Y=~?cW z4$VoC`Z8#T=+^I;ztGSJbSdgLCAV$?xa@TE1{@wLB^d9C2 z5pDqf`24Wou`xWjFlLw3VReoMTMTMxos0Gn+8d2RQ2I3f>b-TU&-jDp=zktab$%f_ zr-U=o1M~)i=MLjDh91(`*&vQXSC+1;^q!-Hm|=CEk$x57@$mW=alNbfGq&WE!@p(3 ze?o7CW>(6Qu!7;9y8olVQ7i>GYZ`EOpW2sau1Hx}91 z`x#Jj*s~I3Gr*JZdtYR}e^WeC2H-qdAY>yr%~V+686}##24cAwZM^+AN7w#x#TxLIT4` ze(PI|n9ptXB_e z2&jllO7jB8?q5p-iR$e)pG@CqTtrw4?0eQHl^6h2JI9IlGdIjn8aa{U%kC*2P^brL zTm*q2WQ8<&!?#W$&pK{LMU;bl1B8`i2pXvLhnqphu`j0rOhc*EE;~^qBvzb%M+OAB z)PsEoGd3bl2LfxrIzIH9bRabm$oXo^ksgkgRtP9z##*}Uw}eod965FZ1uUQHLZAha zCJlnAGi28!%$m;;DF&3m50l?Y(kKEG^f*{~~f&>5`X)$-F-bGFODhHku2C zD{2o{6qEhr!`4T2xW zTAJ*)NVeng@}&`|DmS4Rq{6~ZBH^}4M{|LdDli$es=Z=cxE*_N5x#1{p1KBkz^k1h zi%5lTh>mq|v9e_j*1@9z3U!NXYdwYLqpT(a96EY>!Z?66B79Onm(j@d8a%97DrL3H2#B|z6%kD?Ij%7DdsjPtp|o!HhUp=#O8Cja)JcylB8 z^~)z65u9jDurn_F{Z<}fO%AYTmeY87k{YS{ZeCXOYIbz6w*36Q%;2qg=mwxnMml)nOQz4P}Rj97|z z-C;5Z4HH^EsBoonb(=|TX-VNTYR)UN`_lk}$>}rps6kSC&>93>|6KsD$AaMY$3T%Z z)V1mEpt!Ix-WLn1yQF)IEu;|j-9rUH{91T+VvdQ)KDJup|GK4(KfCkPKgVVTfKa@TJk6IS`Y z&Z)lJJ6@yuW%y!J0>0rGFNf*W3nq~t+iSgPbr*wObQ5+&_MiG;)dY-{jFiTf^3xzOeE+aaY4|{W01P} z@e@x%^Fce{D$adcLiE~5vNzLTkOksV8BlUz+rL%4+;8>)t>0(L^bdBwG+_buXAku) zZM3?k0#k|1lwz2lj*D+sWsy7Eb7sAM&ntA;xoaPGay`B?)F*>`wP!3BH*+&hE=tBCZ9~H~y46y(Pt&JFQH(MXolyhAzIO%tYa};3VO_NJ15}J*f>< zooZfUGzh1VX5Eg(@x~Qk2mz2)b=_H+87wI{dUSC&s{pz>5ay$!1#TVXs^vSY%FqBJ zHtf1r2F+{$WSfA8X>55)PlvKIP>HY^_uj&yM6VkJ&yZu#(a@xX=l(Xrh?iLgItl;J zR30K@eChE5RD%N6s%l?DR99dB1yIXnC~&yFZD+ehp<>tZYeg+*Aa@+GQ!zX43}dlH zaxcsX0XSx9aHmMu3B52_94IN4LHub1{efu53Y&Yv(m;lYQhN}=so*U_4rwq_`47Y+ zb7to?4v@Y=HG`N^gw`S{^q3{o-Jq8Hob$NVB3MkZLH%LL$nuRuHhW(jUs`P9KoQsR zEQvXq`6xMsvDqo#X`8cqemR=`wA#;>E_~ijdAYHts#z{IcA#h0eYH+%okaui=GhmReVO}J4Yb11diGu_!-aETrL;%QZ?3=^e0G@igwW3eAR;b zAH*T!y?`N05z#o*~~Lw_xUj2N2h65g56 zXa^?UjB=6!!KoZHBOr_qkrgiq$WiZJ12@}&zlaJ94V_tim)U$}`3Vxyx@?RSL+*=3 z8Gb0cX(kB96(l7k$0I_O16c)VT**5Gc1fF7{S#JQdFG9lB>`$MINFqX7@A zqU4q(6?X5b+`W*V$ExMUQ@!LLt_joCbhcNIwGY=Nf~JG}MQ10$(x|G74>JYl!pRbC zADbQf@#c4%FJx}Y7mwWBS#>0SPCQPqZLt1CP!8k_{aOyr*MxD8d`MHEG!L>RIhY{o z8tBPnbKP}<=wl0o0YFJ4x3ynEIQedIs;VjK?Nl7nSVE^I60QN3HXh;Q-)L?8{?&4n zvb}TparJ@<=1hr6854gwk4Vc*o8!m#tR!z{?rGf{%?&*LBV{_tt6q!L)yFa{uf6Ci zp;f6*#c1K1RF~rUWYLo|>uDY7BG~r!`uq!K*G*mLd`gE4$qX+>?{(ejoDB3gnW|+{2Av;Jw#6R94HbI=(^f>gxYt`nVLHN07zZQIkBI|l`z=-!n7Loc1Kb|}F^QwXgy);EX z+Jo-sMPBU-*bK+vNyUo^$SJzeVG}#E6=>HOO-xB}{Wk*p68wj9mT6P0Q>@ZG0!FeM zY;Iz0u=0VAu|pp&PziX@}$*K)i_)T~!YcaPhmzAvm6E$t*&~!{xkB&I#i{x3mmx3#qPqyQecJOV2B! zrU}$-2U+%s049|}3Dh?wmwvU3;li&ICxZ^P04-v_lr&m86e#2!cb~qeoE~}h;4zbN zO^cYcQR=M9lD(5x(B$$v^0#;1Sv17$ZA~64{(t+z9Qb4P1HW&_*GH-NpR{F-1mfn z6G|}~=7G5I_SC)y@!GR_Y(@2+bbhH>%lakhZ+B-JMFL%Pr0>y%*{IH+ZYFORv%s=v z&~}WU&p58+zaS#tPhnD4VEDqqp($9KK|*~>^yG5cyg?Wqm35Zw5!+!c`5wDLr;51h z*N-B6k6^CGs^kl%31`UUh6gOm(6oP}RNZ(jSfcCvG%z6?$73#~8~!3e zVdbf@zexgRO;P-PQ|B&oDf65FDF#)gCxQ0HB)t>w`Sd%)QkQ*PZ+}l%d`PnsFEcHA zPlU!llRcpmkEX*hvN_&-xW=^N+i(q;H2Y3L#(a@r{xrQ;a`Nxrlz$!VyD1xX1A z)1=hRL{V~zEBjjaxAN$DNzvuE6EacBIP?c;c7@1lCSZl62nu8fv&QmFW#ztnqP65q zFRzuKPShF7Ng=jc!Hl?bMu?4%k54-eZ>in2cX25PvDW${iVsT>?*TeSl!V*{z|y@V z@0s@ftlK-?7gMj!p-wZ@#*X?dp?j6fY5lz#BvvRQ?QLf5eI0RVx$Gw^B?aZ?bEWZL zzkbm&GMcEQ9Td%)eg@&M+fM05NNPfpJ@{(=ooR7d)u&JA&ytg8?XCOnwYeY1CsAp& zw`CxO+7Xl~%g}WAM?fpETF~^`4K+Pu^oAcrK*^1<<#$6@KbvAmfKN~Z}^N$`IS?R{k3W1i3EA=D&{x3O!nx<;d_P0$l0cNXN1T7NunMp zTG|S@kh|fZPvicRjZIDO;q_7fOHzuS)^R&Mat)D#47ol2*PMq!g^5XLp7i857U(zU zUEAF%iM(TNHlf#UkQ6W_8fV!bKXz`#CP05J8{17Hz5f%>JCZGqb*So{Q(J&^?i1`4j@v z$K3?M=2q7}t`(OS7}gwDV;g}(3v6Rtdp_CY$A7-MprjyqtvT|NY?gtdVo3S= zcYNsVYh)}D0b0h}VcVV}hsb1Kz6U5?+uvlz`3Z0J zdeyT31ELC0xo^3XyAHjg$0FS!nd2NYLMRY70-ACpxI-S82~JMmsbSH9f$ePWyQZU!L}8u2ycpz<2MGA^s_iZ>=rJ0|#(< znR0*qFw6`mZsw@jeLIfQYbczd^Tz?YWG@iQJJL4XdNU%b?M=q3RI&5JbOki_&Fc1Q zYLOteN&)N5_ftK%#$9?4I=n=l4vT~07a2B?-KsJXlM!`*=J>g6kCLZBl}gLR)PwqB z-QABOmGROM=W7(0s|rm;9ooj5-kp0a*!E1}uR@U=D<<|T3&Y|k?xri{ zCq|8N#OD~kq%^27@b|HLt3=pH?k*8Y-zRcOoQG=ZUOWDghaA4{t zsfYD>?Yl6(_SDnP@ue5iMi#G6=gPo>rNs5qavKJJf5=MYHuYrR_{ty2iGta8G_wz< zLm#l=&ykNn<)U*~2<25`K!rUk$iFH1Y=(34VNmVp7j!NJKq0E?&MSeZsw5DO-2b$| z0F=?UPf<`u9E6d(OKg@CA^<)(E4_bmFJn=CXGZ>XWL-{|?Ms8uVkpwt4B9DCvoX|b z-RL&ZD5r!Zvrz9cW&Ylyul@o)umgHii~{%iD|W(G;f#=EZn<5UX*!92$ABUf&^33Dy!)&-C$f_2?UtBlA8n1|mh!B4L?o91BEk%`rlhjHrc#{MfldQy*WIP4 zkl)fG4sz*_>K(M6-F48And1V#LH#tS(G0|?NMV6m;@5^oz#DbJ36nGoxPlA9w6H$kBLx5w=I6naiImq5F zf6<1oN4rdT6JT+WS@q291FsNGeBp-BtD8z-VWIZ!m4JIT#;iQXAj)7$%xA zKF@=^2mS2N>nsliFPIgnz~UuC+q&5oYv$h_Yi@5}19Au2HtmZw0zK9LX3^DV4}-Kb za(CgJoL{jpp@$;sKd{s{B4hQhr=_L}p^i~RWbf{5(SZ{CZ7j#=1@#{YfedFZ9sxYvl}U@BXd ziX$(8Q#BAD1>y{?yBi>j$V82%A6IVq9j^$S@Qe;Wh>M{~z?P%}U+C?k0_vu_ltHb( zMr+5-I7n2)ABt%^1NjunVIGShIl$79s|jV%^GQujJ=)UP*eDIU03dduF)|Zm8Qok5 zyYlMX^{dy>=`PTBDGB`+r>fcjjx~(Pb=t51fJGE=8c=6+12xgHUf%ufCI7a@UCLmM zi;H$VND%^u8-P83cW)293V>GY*e|tfD{)vo`VW&psyk+6Av%M-p2H1A`P$V#P&404fHrU^KYbK$oQtKT;}p zDni}0(5Nj3nx{^X)d2l5>18+rh`Tj}d35Rn47nZ7gGol*BjgL6aIO`2n(G$VXHnNT zOut3qqtZS3;olB$rt zD`x~FrXA4TVeijo=z)kH&HD|}P)e)=9EJm$4=&!NCKjL`cLnmZp@jfo>)=S}y6*d7 z1M@KhLd!1!G|~V-ZJp>j?{{9&nueAZ?MsyqGiK&l1c(zY6b1`hD^J8Px}8fR$xgX3HzOjV&HxnhbK zeD8?UMbxjCh>MCMe;d;MuDZh+1V4-g{i;TpR1_2xPa<`o-UeNeuZ)sXDijTdz|sR5 zb@y3h-23+uuCRT8P3Hp{$pWx5n|-Y=M1L1r@rf4qZovcVT&&^rg_`{&4?z0mUtocf zp>^3+Kg?I-Vt!;5Ycp4)Gn`_O5F2}a&kvp;yamEi=fMp8`Au^HmT08#?Q8m|+)0_1 zNIhsXj{uL!9T*sh*wfRIpHT-q+7=)UR|2pZIbU>&4l-~60u9Z!LRqiR0%F*8#dg5F z#S&P^nY|74C7fR^v zv?(c*3+jin(^@!N+a;tEMZSo;ES}#ZQN-o@4a4bCaUjgmhqInerzT-<=D>j&fsv6z z(U!iT8;0#o%gwEfP8c)(P&twf${qta0F29FIoVb6J#d$e-^`##lzZy8Jr+J;yPoW> zr1zGRk`jKdeEE(sinu%D=nZ%)rs7?69#U4>YN#t<6~LqGHoZeChm?;i564Iphn<$Z zGb?r-pc_UJWJc`}-zO`WSXiWS`MnD8h|n#`0X(<_+C%7|Ie`6gwGM(zr6PbURzFAX z?fn30q!hR9ybRbcVj@vT7V4OV&9&2KZf|d24Cm%BZ5D082?(IgKuYZ+<473x4FdE|@pBEs>5;6X4BIiGt)6r^$E}P>HhC!VNN_SToiQcW*X}{C1VI z^SYR`&1;UI7VtkAr2ElfOHYDfnB6lbt3S1v>G2}T{PHyu&~`s~d1hre64Gu;12~-w zr5jerDMKo<`NPyTEl$uENe1>0Z3Dryio_OVn&W%+z|)~F(lcZ{$wrw)k8sQ2_l@8P zGn4~8n`IRlZjyKd#kL$MC*g3zPv`2Z`J4V((z=fC7tJZI?XgvIOYSGvF^r~ax(o5< z<`vhqqai+@qr0EV$knZ=vrw$E<@XswZa5j{0Dqell-#p^r2RWWVn}o=-HrgY915*# zfQg@*pICQ>$Z#*q|8@E;BOlkklkUF<{(6lZOpozVxO+u+$+dba)LeJ=lAm=9&Fjfe zhnKsLy5`X#T_9Cqw&(dL%@Lwc{yl4mFql`5(_cg0>NfG&gm(R!qGaC820GRVoTf6g zjKL8V-lwX54(nY!abEGjOTRjW+RM-(W*}PtZ2gD(tj1yN{d{ge_)cyoJ@m}Uxs3>2 zVo12K#B2H({hAt*qHiYmMJO`RsR8Jwps3QBjG9)|$tLFg8A zfM(vV5?fQ?;7$%yXxHNq8W+?Pk;30aVj0MPG%oWoX82PTU=p&kZhJh#PlvUxKmF%> zs3etNC*#$2wuv>Wc9(^-8%|Np<};au6QxRPU2D8!mgQ^8naurrmI7-;4YN_n9|Q+* z@dYPpjGE3X+qJ&H&cy1)U+d17NU^1sJ9jfdUV)!`Kq^+4E9iQHeWMt6y3_Mx_3bo9 z8lC-Qd~&m8=NT_aN495&OQlfXYoN`>tFUMwWv?nWoij+A%O$$K5J6V{;^^H|myE^? zyL?Q#y+<=BO9Z(h?QSs9@vF_q8afl}IPfP`JiV46Bb+pvK-49t=%M;Gka~roVCF4l zNx>Og?i9JHp(kyATY&6N@aX6($+J!={_$U1c>q*I2Y_1 zHlN7Rl=?7TitPn;zBVxYIt>>6#gRIz@{6;5*U}sdLNr_BB+`AD7;u0PeeYYFDv^| zrrgdC!;qqiQ_7dWus;3=|FoJkt4t=dGF_8Bm-FCVtW={YxqiCe>|VHW%ggAt z(wnNzNnG`>^PY8@r=yEg77m5WI;BJ&x)-Z3 z!ginifOLoTouXD1b_wyZ{32oKXOmWKr#$JKJ0stxkmB;xs@S;)mp(=kxReD467cnJ zq-)h+uvU_qOkL|?y0=hVUd<+vKcQ2JSx|+bf!7?$#u6T?Wf|UfT~*V?@tEJ?rj66H zV#W=mVNZ0Nzfl~lRM1*G9-^eD<5=e2DE;z8vh=D}W{^ZniyB+QR91?pI{rNFKp_F+ zuSRfjuhhll7J3mKf|OIXaK169B3cJ~K4&$2k(x#@wjOlu*7y{}%HCnfCEnbESH~4O z>gDS@a*Iw6508ivE*jdzNN=LJU+pr(NicR+rsc(8zclQ%=_fd~TyqN!UP%6ji+eCt z9)=`-nl13GL!t+=6_W&IuwvMUZi)DZRZ5M~J|(9EmSZEch_)R9iAilNF(0C{Pt--f z5LaSvje&I&thKVL=>dAuf+m*rHvqdT4}XjS&QU;p)C>Gduj#SUq; zU4?hT7xYUL^BsEGu-E9$N{fH4oTHzao{St`^g8$UgU2B^qgu2NcdrI2hwoB2T5@}&*WKbGDlzdaA|^jvUhLr5Y?VT8FJ7@ z>I*nyROl4~C93vZ@sUtWu13PSHzR`9%6z!d?(^IMwLWD6dnzxR2Y;8)+M-^dh>{tTjal=~`h@DatmfH$r5;-%B-#SI-*qqUJ8IqL9rR>_wewFCOp-R+ z<{R*NE=`uB6K%HIcqM(U!dp92sZE?3sS5VSTKZIk3nz~Y350@JB4orOADHOerESuj zjTau-U3Mm@C@EeD_`Txb3yt`#P3rcPT?-8dY+4+hzUR`hOKIZ`FnkufDV|{tH2Q7# z(|uc0TugsSYkF}QHrdEZXuq4IpU7Y$j}cw~6RN5-%hwyJc7ykQdjos0`zoKNRikQg zN((d|mU{LzcP(;`e^f*Pd$~uNc|zJ4BKTv)oXh3KQGv?A3;A^AO{{PFdW}atJLD1` zij39KoH%-en-+t;QAnUW>aT0X_)3)f=p|g0o(@~jytPfA0Ty?acWOL-B5pUH3md8T z^3olw4JRQJ!}(n++2%)eR$6+DW}L&&tyiLZE=Y1=-JG3`Ux-LWzhs&o>|6ru)yZJFxKognz-(;vZbty=XyoR0#QU~s{ReDzlf?)ul zb$n2Ff;CY=uElV32QXuC zmnZWQ({4~wY|*Z*o6nFYEq2a|8%~8L52KsGmYrw@u z+5P%xp*N`Mlu*7uk!LdVHjXMQk5%)wdpY+bD|yO}u9h(zmsS1gQzAnXew2KL+=Bac zjhDpEu%@Tvjo|N=cB?jO$jID7MJBItk%9E;@4CYp`^a>@W6=Otrn%W+k%77$Yr4IK zB+t~pcY9h<+ahq$@2l;j(G_YkIMN!*r<0+AtLiAXfhR8N7>bE3vVFapCS~epuqrz# zG+?(cn?|0Tcdo^G2_C1NFc`+Ed0SFHd1}A;lS9hf97=CYSJO3JmuX_uyX%Q| zg~qK+*hqX*l>Uo`1X0dzUUs)Hvlq`K@vpu;Xqeq|sCizk%Vx?mm7R&>a_7Cxw<~Eg zi{4NkF~`Nl%-Xl_tQ1Vz+H|A)S^S9$N?aja>X^sQl=B~~ZDgFkWHwA=xK-bkBrFSH za!wXDu|+y5`5$19JE(+n3e5~TP?angFl9!R&@KqNWX{V?p)-vZ?!iekHks^W{{$DgL_cWy9^s_+f`eUa`&i5til$_kB}SiI zR0{5{8)%|R5V#<)C-}F8)NWm6x=egf(Jy}?mAIgRki93)hq}{0wA1CJJ+8Z}Fx0Is zqf&{RCqw`6o%6V4y~OH@h zdjI`8rHyHGDgSOwZIRIcLlv{gn4w?8x4Pr;8SmaBbwJ%ei*eW zC`sdkQ}=4#{<@cFrQDdrk$yWaXtc{@*)k+Z$NWZzt(!A9$L*?uzSd8kl9Uu#f^Ht7 zJQEghP^1?1cg%m`V|Tfx?2*CG$wx4I+Z5a2!w`b$dBid-$}8y6-^!@Q zxnDO8QKM$Na{vs8GFrdykD;Bjb~4_j&Ug`f-7oCDRp6CY`b|))r)UDHILZo1UIU00 zh|GM;!(J=b9uPAkS2uX&1_v9dtalB)sfa44ntQ(q4{}bm4-CKL5m4os$u}RRc%~0x$*!6^+$hIU(o^u$R;?op{qqF`eUTU_K32NR_3s&Vy476RWThDcSRx3@@PheiHQu_6# z;OmQuB@a%fT3x>U3{9-j0zo7nS-TJgpda{JZ4U(S#D*F=y+CrOh4Vk2I=-4)D~+Voc6|xcW7r!VMtRmO zfn@Kx9+9KZsPFR(NVF;M%wRqoH|_|MP~xFYpJVsinC_)GJ3nwJV|K*-(ZEz$ibff> z92cEUs}>_-{$ApOwG}V+H5X~>`&T7y$qC>!5xO3>(4L5)!}er_TsWa)o?OlTxLS#A z=2k~^kN`o4yYfa8hduVhQMJd7YSD!=BBDVpVSWfEsAj7KHI{ten||wD||-1lSH~*&Y{jeP#Uef(2!-UD_`4dp)ty# zSyLNnkJm$=e->sMGSJt*YRS9yDd88Ju%#mqHDVnjnR~d#&?jX4eF!)(!fK5_acTo0ThtnTAonN|kQ>gl$f*Hiw2_b@9 z%U>OH8$v@21TQpW{HhCN=d3D~#-xnSYw$mSPc_91fFb%WfD_%#SZh!VeFwzf&Z+(G zl)9=BdM=;gw1{$oLmH!K-~tU|S;=1hx%A1q73J5^JnZfzq%r^U!nF?(-!QDV)8>HQw7nC&L>^)YFgMl&vIIL8 zh}qizdn3ENrgVL(_U+S9$aNW`-pZl9ww~XPvC_NS?SO$QP{X)D)&ja-EvSF*2evNI zBBye0Twf&An^2E68&veGl1!B=;dk3Wi{+~2$ z7p=E-70|oy|NKPBb&HAC%=A*DvwvRCGQ9H}ixqUKN4{sofdqStS=UZ_*7UdM*IRvk zw$M|CFYjZ}kAB`r*zJ5ed<(gQ%wpkkKZ?q7i*6Y(@6RF`{k4^L?nJt-ALvPx4RhW?P*c2X(AT1^dbWwND{?`k!w){P||d?f}jQhPzH4+OHdkS=+@Mch&yi zyXtGrSwX2Q(D^C(B;1GfW)8K=yzdzC z6^h;YiSngB64RI#7zbVj@h&II_$86PzwDEAS~PEkMEtvb-#! zs2X>_Q{1;&_kT}zPg-P0u0IlQ+Wjx|=D#U26JSaLWIm|z5)nH9Da0X*hSitA+`b6n zqEi=5unoW!(9qRI#JMJYXPC-@gVkoZpu1O0*TTLFCij8I7LkwKTyg_)h~J380>8Q$ zsWc;V02tgO@f`>7=fSE$4GAESO-!MnP77>7z;rd~&jF1OF%YCdtnXl+KfG~RDVp^j zn-6zjwE-yvf!+Nt%Ym*pIH{xV17ztIpoZZ=WmA{{=3G2s=@*&3W(69zc0$kt=$M}57jfz zV78euGcz;&3dGfTAAflRIwUp~fh;&oXmt$2=o#iYJj1NxwrU$e0C4f;&G`OT_!OoeL|EkK@% z;!#r`8vql~_h3bpn~*-8wPrD)`?e;eL?H*q%lfzAy^^l{b*sp~GLVgxo8jLOMgECn z1Kt%>+sH4brX~Nx4l&mB`)l_Z-u7;wDZKdnh646qQGP(8o2|{UjP8GXearvjeK#(O zLT)Shm1~zxzVWZICEX?J@5v$gulrc>OpjqV`62ujgl{vgJt^f91vBK?0Hv@NMBSGc z;t(JPVB+GvFAEAJ1_uT>fNu&s_;8Sgfv?X^ChK?YAGMMTW`HwSYHn_B2hTo)5CI6v z2(n^=NQ9{za{^`j%diWY;XI!KQ9>fPZ8QQqn0Y8hNa~Cmj0Lz-*i_%Hee@EU15PG@WRMDX%osj`&usKx$KV z-4eie*MLC-FgYklH$|6-mg`~i^>6PAp>*(Aj0RwvA2{4ac}zV4Y~O2R@Bl)3l}JVj z+W_#IW^iDH*m(hTP|}bTJAyhR2&J^o1{_b|y@D1$oP}tyc6&^#1_uQpUnXSFhj5i3 zPe(!yE0{_sX+h{iNkydzP)&wb9 z1gcY_AW04&f2HJsr)igjlF4Bgz!T(+`hP~tYf3Ub=7HpBaE$d&ac&YjR}9x=^Cujg zQfK0B8i=Ol@M}y<($;*heWJeo!G-I4GK%0=$+?;RjQ&9UsZZ&+ay$+oC!=IG7za4XAnO z4f8-(8zvI!AYlO;a26ys=ShnI&T0bW`RW+pyG9^WQf+WzCZ9yIeDgMD3D0a0-Vb>N zEi99~jE9#I`4#z80LhpL{hF42cP2KYtO{U3I80;GQA!I2g?H^Y(@gI8@$|@Zptzt6 zfYN|4|62N6* zaj!h4u!#CxCo%1d1m8dnYKz>Jqf#C^AV;=aLHiKT(1~=Unul3Y9oS`{^T!Jng#nau=xbz0yc>knOb{@I4SD*t2VyJd zIVNHV;Q5057m;}f@O+m0nOExpD1ZC%m|ap>RK-+sQdDRty?XI^=rq9Zehw~2gAQlD zbiF(--(#9eN^7^{bXsIeTn^z&A_039#)aVLMs>S-EX;fO31P=2#k@%%Y28u>MBG)Jz-H0`g;In*R@2|5>BJ9`3)Kr?-W zzIgs2!WBYa4cSBizuC6R)*x_t))N%5vbwi}r>E*iv2Sg2$eaxdcyvAjLNyMAQxGJU za#*mO*d2IBq}67n=5*apK;8y565tQ=OPT=$?L5iOzqMo7!_zLMwLr{j+;lIeQ`u2p zF$VjRs<3lDUAMNI?kzjwef^a9v)K*UE>iuHTA4Rp?%gh34|^B5nDpbZTh}NFkjE?p zfDqmy$sG{&nV`M13?`kekP)Ii$h`a$rqBxx4Ei2dflbpKcaa%=J9T>H(F~G%M}C@! zC4zME041JBh>aL<%8$SwrXTR|42?c)S8%G=a{|I1TkneH=pxAE2?7EN`H`ZN6`zArHbNREl@)kxK@!2`whwmcTSqBj0JP30z%J-4DNEx~J7(!AL^`gGDM*(QUco z*XBCna*gc)F^-gjK;IaI?k?3B*d;jic!n}k@M!J$=tMD4o2 zIDKL(Efn8p>;;bFeNThlc#?8_xnf3IwiYIrk_Y zu%tMX{=C*^y98I`X;|4d^{-{{@$r|ClpnCspeAaR=C}UZnva&!Vz^?y4x{~fU-9U+ z`EjvOop~Og=2MRky7$8-@)e`bHMz}NUC!@&w09B8i&;N)kyts6G$lK1b}M{N(b^%#f;|` z7P?-hx`G!Qa;ia;eFPXrf<$mo{3hA0T_+gEX6PDE53$k}*)cl5{Pt4%=~s3-si!86 zH7upwA0C{7JujCMroyXo@nUF~ZK@EP$7oxc-f3bj7Nq>#=Ejo&)!bS9^b$|OHn_PU zV*zIuIC1Nc#A@YXk0HG^Pf*;$y)7mc&zGK@DPCPHb_0|yAc7UWT{HF~>Cb|g?4{c|iI_)u_zk>jmR|Ee8nMHXMQD(YYC?~r-M z2UBrosMAb_`)F|K+54(CKaco-;&V{#_a0~Mc>A+Xp7>dT<*?v`*LiIkW}|hr7y*Th zG`7RbFH{29U+H|SELFm%$#Nbh&UDK=wDrxX2S}^eiPdfQu=_gKKRoR@eecWDoSx}r zkRk;6q%U8~iA4e0bI(hvUr9fCK>#r*|Bhv*>i+g9%&R$x7L4^!;DOW5m>wUn=>jJ?}RU1pM@|J)=0=48nT#PT+W8ctbMM5u&AhLW2mrswq1{{ zp`jt&jxD3GFtm(}Y@x69<`C#0nu0T2&aYoiL=q_nYQTt&v1|}3V=S8b>F4VkRyTY) zs_q&tDIuZKLn>++0M!=4X?kGSX9?||UtkHSV`rC|Y?$8yB9*rLm?Xjy(vO=ZwJRzp zT!%&C1kx$_w+DS(3yrvWcrw97u+(F0c6K)8)hl#0!h(Va&%oBEKSUF-C}q?&SuhNd zCLK0+B?MFx5ERP+f5Lh=N0Tx#8f$*o7c(DD{N>U;?ec8Li!VI4Zf}3_pyn9|`l~(q z>%H&1R}br2Sb%g{`tU3FuOKXx3>w9wLg|xG=QecO=0K)bT~(D1wRgw1ZQEprnWa1| z_w3n|4J}AzEH(`&+ocPOi`OQ6PSg=+!=t^wK-m9YVtw@J(X4$^^BsQOhdWSRQzIDHXa)91Lz6D7BR-|=Q5qQ-n;SRw z_&q82WKJE2&(Pl9UhnJ#`wH@3zJnaKe|@?Gr2qZX)n2s_o}F`DFZd|bxmlvfX-eBU zvV+d4dt@PAlTBojR>>B!{J-~3K{6YQ#&V-;bnJNj#>nBU%Ijxaue5`KUq# z-4XDh2YwkkupAv6EMXJx??-dJ!Qo*9mKL*6~1O0%la~LV7upj<2byS$iW1 zMz8~+=7rD7DzWzttWOviUxt3~8F6v8G9eL>L=cI9PKvdy?W-BLuC6YWi9m&r2o%fe z;c0o_$;D;FdPQ_(WR-MHnLZq=1+8)(0|O#a_lZc#H>yV`F0E;qV6r#Jy-m={=TKR^On5 z^j41ub*ctGgNCy5Q&e|B8)_NECh@7M`~1YfZU)AiO>J!q*((`!Qi)KFRE;?$CMT;R z&M6p1oWNS&xB(e%OY{iLI2#{88{M1E*^JWrl7T@|QIXU#v>V!?Muqebsd3c*zQP}$ zoh=MaqQ_E+DJfd|`Vsd(UD2>XW*pU_CtIP7b1n)@m>+@J8R1zgQ4h>D8rokXeY$h`rf7A3o?Xd(J!q zx+*tz0U{Py#0$aIK3X<}4}9&~H4VG%0vWf;B|PRYq3)6tNOXJK!~6oYnQ$0rkH-(g zB<%EK@~2|ZK6mZ`(oBTOgtFVr$7b+_Q}p!oys8KpSJS>LhS}F&z&Mfq7uOgx_aFIj za&Xjt{CF2RNAnsLh>il!5;0iNvM{ojl#n<=6#-3s;Xvq3_rHPtW&q0x=#A>8rthFf z665c`*HwZEmi*(RKvafws}TakafvzTp(jE?1Pi#5G)IJgHNwdEOIZOV#;{_Fk{PDWPk2m}Ik(ef%4S~5Oxa~Ym~nU@i{ zYX!*lcR)u%!RDf8a=tSOweA;yp$qGUe!=7B%}bERP5t)JOEW1|FNTgG|MFVo`M|d8 zW2W0u_!tbDE)oi)qAo%*n8jyYl+@jA1dj%bgYH&xeHe5&;W|_ac@z{^oFLYm_4ok> z1dT&bjvZoR3Yqy?E6o>bQ#JJKw+FJ=n}K8uc%W-Q0(JH0;HMi?9|rl@L#ByvsNMz! y1~~PBz*7Sp3kq-Jf5Yj#70Y^O>i-vaSGMYNNk89K&h3qU9xH!AE=Bgr{r>{&c!eea diff --git a/docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/z_hist_diff.png deleted file mode 100644 index 1429e4ab7a292153979ace11664055fce6f54bc6..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 23970 zcmd_S2UJw)wl#`vRuFB)fC}1T0whQj5L8Sh5(J7QK}3*@5+!2*17avdlqgE9B8Z42 z2}%nsNDwH3fT&0+k`g8V^TY0QPoH=0c=x_{-x&WJ{~2erg1dI@{e`vWnscu2x|+%n zjwRen*x1-OC`S*fv$4&kva$WLXwd@v#_z%JC-KKThXWMNMfjh`qBGa<^E$^vI*uB4 zW{xh#_NHuSZS8DKcR85Yo0{4>nAKDH%~Q8By`pTegvGY@6692ls1U2FRimIF%)f7OCyq=R>2> zhTd+EtpEMx=tp-}T}tEuMMcw8oqG=->VB4P3s63DCK|VUyx#8RiT=C^+cH}A@3%I} zirarZIye=!$Kf8P<=wmEo}oP8V`fbAoB@q=*&Q?S` z(-D+RaqrK~IM9?oJuJL++qROrI_;>aC~mn6PqTK(%Nt5HI|jDDd#GU}@8tWCYzgTA ztvT}JpBql5xaBi+xAxVk+9keE)bU@vOWdT0J>8=EFzq>pfP-87sFCMz5T<++MaF$3 z(Kc_)tQn8QEqeN3Off}8Ma#=8PyWZJxxRjWmTzuwWp_llKX>R**~-VK?_}8a`t@t; zk7rFy6O@95uo#k@6L$}`muXm9TE;YPQ;kTBkKf@rHC(6t?rlVcBJJ?awNYmyJ!?Js zySuH@JehPV_a6J`s;Vl-Mw^VcBJ8~lX~o;xUoGUV*>YR%@lK7{Kbo7TZe%gH%`UKl zWiHK5!DSO2k=}+&R|{6YuxWf3%fDxjF3s`Bx6k~y)111scIiCbN5+D$OVxXvjL4Jb z92e~d&G5bMTDRrL<3GCh?;V?%u=?`*b*|Gwxl<{orT1m*zh2|(DwArdd1_qv+@d;m ziHXvE!Gv>R_-Oa1SC_K~%VhRjp2p;g;zPHid(;YvHmhX+#P4+HFhXe%C6QXquKdQljX{c8M4x~6A}}TcXf4< z0mmXrwrn|Rn{`M)JGmx7BkoMPUQk3tGL81@a8E;8iRU)ukm!K*QkKIkre2P7lBCUt zXoDPQ>biWB;$_4AEk=rd>?fR^Ul@9gwH+(wAMA>Yq}w#Uz!H7BPT~BQD`dvZvzqx{vJGxs%dS91ku;!@oYK>2u*#gdS5dv$cLa7e39|JMhOA)`#?NuCo!zMOc|}T1k34ckfnrAMIn_ z+o>@$TxVp}ob5!5=JooqpLQ=Kgx-*LmVtS7>xv1kd2)o4Y%t5(#D}hlMpG>9^pMcp z+0e))`{vD?onm4dajKEEbSgvKx?aTQG$W#m!A0ZSgj8Y4OaE zpXd>LQ(UZlt)iqv-NGWF?50kdSs8!RwARTR{{C&PttMiUb|2-qR;@BGxwkXjD^Drk zikWJ1#do*UG+ayf!^3&j1lAC^-16*bC#r1y~izA3uIXdJtLD-H9vxIM!=1I`VOk zwyv&({`?t38kv5|GYkFP+}z=AqZz#;w|UYVMzJipR`1Z6!g+jvr5C4)!-@ zsCHCDN`)qnNQyTgaoRSkZ8fE8Y_P-W$2a3QzyDs}ljG8FI5j<*pThL$*{|WWOF()@ zo59Y`CCY1%)^^Ego>R&9ax)1SN!Me~TE`TQGxWL@a)Ff)=Qz=uAs{28=j-cxOjWfm zRyp+E-Mhv`VKPp~@b&~aY^5w*w&4!np~HvyvF_@OymLQ1w|H{m$>FXyYm79H9=-O_ zd&V>AWQGc_=g^6m4x10p6Z|-NlPQ7QcJlEZT(M$Bh>Vk^Z&BilCb#Y+vgb#78{?1( zj?o(48H5D#kqz>p-`SvR@Q;#`s)4prr{aw+JH*8`Tf8QcwjF=)=*pFOgX?tX%$dV2 zVRphto;W0pIcB(UwacI9u-aSSsm~~6GFMgnjJVJTco+lUQE-YzjFiFE&7UMlL(c(7N7VOmij=WHJ zxom%pScTjc7Z;bo+F}w*k+qK@1D!vg`s>P-=~%+^7cLAfMs&(xX4thXNz)#Fb-HF$ zIV9ld{{0t!nL9TqGSbMH^49MZLTG3xF;;ZqbRJcIbl{5Qwu zFSL4@bBNZ#>|0fNn0v}IprWDDZKNjwV`0_Tltqiazw0DF@YBwnJ898jVH>t?-)@D; zBOtD;I!<6C<2Yg;@R z<)%yXm*T~?&!0DM$EHpDC=|*#vMK#0FP5|5YX63zqg6v5-OqOH zdR$+x<9qe0_8zT=u7mAepRe*Ji%j>VRq)HoHhLgec|U8AcOS7<4ws>-%8x~p@Ip2h zAgNJM#~uWn@sVBu|2L00)=PH!3+Jg}{xHBo0PAAOfS2RN3e!<@7cMggAS=NT^?T2_ zJGdJgM-c?mot&cr6cZyqGwFc&i#N*mA~nal(NC8-EkC|=WqRC@IP>V(*w~mK-xdXh z{3s1k_CgWd=4MZi^^?TJMCWDF27aQtYi3@C*u+bu8)`_;XR+Mk6S|{fIpQ)i>wks?XdYEpb_UH&*TXHroLPJmbwp>hc+=WQE8yF;DxVui0B|sM0T0!4soqN<~rl%y%{IT5h?AdNCrew#&-H~472_IhAFi@N> zX*4SjOSp4qxl>=0ZiT|kOO$_QHr=&JfvAC16{ez{dp{%`@?TAszSrb;;6=fmJMX?Z zeHRF9PkU?Y7AYwy&*`y_pgspy&CXp$dHTpF!EqgL?5YS_LQX;jL9h=1PrCX0-<&!t z_hT~J^YEV~oSbz)_gbbZ5WOJu{BKofs#xor*zYuS>Qo8<{$4j5ATK-8p1 z=A&@_;i>Ppy=$7X93F@YljclWA%pClV2-`h@U_*yT!AY?u|9j zE(t0CDV~Cm7>Fkck+h*pSk-nS5hPb8PP4T0s{mjblRcJ~W+(nBGDR37N4ia;X<)LP_6x&jp zs4Ir?-{ZAEXorx{{oA)yeZQY?ElL1DVbvrA@_79)o|@|St|J3iDx7~bTUyCwdeSv_ zphSSwrP)s1u}DcLTfAp-C2gB@y0TEW4I$rl7IG?RVs&9G<0mIQnlH2l*t@u7Tn`tw zpyiFT1d!bNwyzmF;ZBlCp2trlaq7$WW@4N3jsD%V!~|9 zjx1Tal$r1CEn)LPlcAj=X=lPX?%Jwm1RRShJwl+!sk>I0=J@T?e%~us;t>pKXT%)V ztX>^kS*f;huiNvI)}h`;ZCzd6@v)A`w$GoV9zEIw+y;2B#Wk>mgM*4EXC@oS0&iI1 zug8uZt3o74CN{Qd&SvThd7{K!vTRu`vYBc#D(?34WU)MbDl(z4T()N7IMq~1R8MgA z-jR9}n$UD*bZji5`7ux&EbMDvE-G#~o5rwFux{QLT%7CFQ!g6&;N?p>>~;YW5w(jK zF9Opf+i&zjK7O=~y-*G1KHpA2vKEiQ0Fq*~H8t;%q7SR;!NjUntK#ii@&*P5suuEk zsC6Y)#sGkAQB_q14k+7z$j5M|I3D5S|AJ*3bWE*XZQCL}Pn@{h)YN2cwy3nlDEr1I@KpDLf?}>* zxgyZ+Ai?sCC9@|v>fP6zqwP69^uF1pWfvtx!s@U8>P?f z14N=l-@CWU^T+3_2zuJsKLp3lnY+LoLAPD}Gq@u%Fs5hD9Hk+%CxE6Q*A9EU!Kj>@ zuex{mo^Ae=Gp30Fk`ARZUOI4W!VEV$aNvOT$oHWkP0%Z@3`3HpMtT{KAOE)NcCpB?-#9jtca{_V;EgAsc z5=$a`xb`I34jW~i&6_gMb4#Clv`4J&h0RGqmEc0ARZ4EGj}M#j!`+8SDPWsBT+3gP zapC)EL~0d8s1MmrjG|i`8Q7mYfoA-S%`tWe-@(%@GZWrqaUmi-LdC6zZ=4JkNff43e%a1S`4#y4>ogha)vtC{VdreGt_JS-0bK=k(QyNCVqMuwn;pD-Tn|^ zClL*N6ca4PRthK>>cN9ov=ViqP@C*jP%uKQt~zO#8}l*OU5N%%7u(rso@A77I5j?8 zJv}uMboXu)CT^#w=n2~MbDs}v-MY1ltXSunotT9Uj(Sx$8Qyp99a3jXRZ{J5aZ7_v zKz1c`VnM+^g09xBTgSrVHy79A{qiOUlqkFHwg|d(R&GwWZg^HTWq9H9{O~;vhp`av z<>%*P4X6olvL_$^dvQSHv$L~H($7*yB+|fcK$aFi{re*22)PEc7O55wjn<-o>Ts8j zJCU4*JKrnfTO~83unl(^zSQBA`{onvFLqKR_V{~%NW%XS9E6XEwR(DZ5CQ^)3=32W z&`Q?$Xn)GkkfQ^$pgSes5R5?->b=M5Cu6st=y{1R|FJq@h0})#m`f-qwoWzY!pH7Lj z`>nUF?Dra#MIcnRd->h3=r*s>XXktbrm}}(M>yqwoQfH{ckkWg-k*{BoxiQU{X;HCdfW@EB^~<=ev;$QKZhxJ8-hGpBcxB* z?|0`_XKHEBHppg1`D6af#C1#IpSyh zz-Z&MvxnZWHlT=0c{y<7@slSExl!B?l-n@Url$EGxj@NSyEp184}I9ga9p3kII&&~)GtMQQYZ-I8N%_ZUw>6M zj-_YMKg8YB(fP;SEqeK0xrD)V5CjiV_2yx$QorZBvNDOY1P@U`%OlN%Hqe`z^qu-Y zHs9QH{>wpMhX^^>XRih8V*tWzBHH!K%gV|urY~Ei>ah4CS+_a_>-y*{daSul4fCgd zvHtMb4ePd)5B6OO5q0h7-#=yLB)2I92Tf%}eP{{pmPw@xV%vTG{F(5gWvPF%ae$a& z#Tfo{*Fm#1V^&p+zuexvHm6anrGu9d+XlK2CBlb1&zyCGqy|z@Xt_xWSf3A-o5ZM? zTK_e|wcuZ^V?B4R_E%R|zjyzBEXp2mXZ*W%DWfLjmT`>d+ldSch+TJO@k)%&V^rW4 zpmG5crL^Z-*CmIO{k>w#;ZBg6L5}As!^k*K2MCkqqsQE&%tS%sgnHmue{~o}1sIk< zMC>;b0>RLw>pGnQnL(iGoz04xDDhDMS^%F(c8?1K3!4<(JkgTpxs`%=Rx>#^*ggG) zq^m<5nQRhjF(K3Xl1DWNGr_;|g^u7cWo0UO@Sur_A3w}dD1}Jb83u`#g-BFCGcDaP ztL~)7d%U`c%r!PXE_qJU2O`aH6zzPG1xq7v+eaTWb8-ZB?D)guj6ZmdB*A^B*7^zQ zXKq4fSWQ2-`KK16sTqywM%rs=nsb)(rx3#!;EJ>w)P3M*{D)zSGB6A-%$P|R?Ls8j{_ei&YH?Z)9-<2y*kLN8-sK|D6b2A)fr|y}(o%kXbmr?GS zU)}dGnEb#UV?TaWYG&OjQrGS%1Fyu!#yw~Dx_=Vk`Zcq%XNd-Z7Uyg)4KmOAE0lv6 zgp!kR{1rd?bU#kW*3#jdPfgwRNZHd2nhsm$st~%>p*zvr1N_R#&*%0CYLbt#Eqg>X zA2zmym9szl%>BvV7R>oCav%SDeO!)PW{z}LHO)+izh(6-0h1d$7jGxMK7?Ph;?6y7 zX3^#02So7v6lP9W%kq(=KeLcY@-JIW#sJquWjBtS>ChkMl0iw{_1_$%eV4_l3R6RY z)1aVLH*elNFgU1zXpEpQ$maJOfPhfdw+F8}!#*B*gwZH{#s1*qn4xnQGso7Z-i~ncNR(%a z-i1ggrJb#)C94lY2Epj4_8?7)n_%8fBtDIRw6(d|`c;ulGZ`5=iZg>(_7G zPzOLPdjK@ZFCtQD9nY6=GQ$?+thfmo$$}HZ@*2ZH8|JSr&F5C|dKuxy+G*30XMkWt zs`kG88LzglUyss$`|UUD5&vt~DxtLUQTFWFgPKA8+_`hU4+f?j?Cn?a@ECrcMA$Za z*M=%ZQ(gUZo-m*?A6OvWq*`vn^SzLMceA(B?X#X+B_~yZT2yp-j`RmT6>R6A6u|IiT-zIcB zK`<}7^?0Pb0gxU;#0y{}T-GJ|gG1dyRNN>)*ZFOhlA74IZcC|^U68$lgVomC+0yOl z^lcKpgHQe>Z3%Z|WgnYun4MSH#>!0~unugbomegEI^dsUv{mcY?dkpr|=IJ_qcARIe6T znnbeGYX3uk_9HWDSyQ7GC1!n=(L%Re1L!A|KM%z-z-0a8lUX5Q3CUm_skV&KSQeIk zEogD;qP&SdSx6K|X?p3FF@OAVlt=|wAT%hS>QJ0?A&uSRbLy%hDmAcuxF7qv-zX*J z<+Z@=C;|m3*by}B({#@X=Vr%hWm+^S6Lq9hvsR})SX8d=?%HczFk0V!yPu{bC{Z3K zl>ACpKDg-J-H_L=e>WK@88BqMbEODA*1#YHA5Bm;;DE zH7J|l+OR8n2c*oy!$VjrY&<4ITX>i5vx8tJ%-%sRkpNTd5QNHQq2=`C*sx0~`8xfG zHzMp$q5{upAvqGM130tVBaUB6N*kMCMAf_e=}}kOrLB8)m12^y)Sp;tYDn+>m!)py zlsiY^Jh|Qft<&UxuO##T>kRoX5ATD@S6xE`kj2^%q;9)&p$JiVVJ!e|qJtIssa5}1 zsh3)a7~O5$v!Fzl`%EkyNd=ot(#UGQ~`hP4hInCxJPaY0?brkar3HIOuAh19U zq(d4Lr08WkCLn4*KmBG2igbqhaGHKa*3eSQ5Y13|J(gKL3u5qDECItU8_z_k_@7URPy-br}Po!hq` zLCGg!0(qASWt=!;o*WJ!Or#^|!q5l!6lj?f$WSNpy{4YSP%(eOf)I`%K0>>Jx%zO1(zs;@i5FP*9B-c2XvU8Ny_Qike@6QYP$9Rk*kqi?wD?ju{>C9Mm%M3m=8( zpP){dd7kc}^SAVfqI@NyGZ6#tLC%HqNChj75a^I${zpngzJLvT)S>8|YRfMYJX#g# z*IUo-zcEb*54W#5m65qhC3)MQ`d}gq{qFM7^$3VaYy+KezJ=SV%{HE5f*$!xadq9QoDljtF|$rG z*uLMvyJ3CPVwc=9nXia{7FWk?VUwL;XeIIAEdKrX&d&Ew(j!(S?@v61dei4{N%GYh zKTraYcoIaWVXPA1+7CNC3O7B8YLyy)7E1o+kYGabcH1_H!YJ z*;hCmR}xEy_Za4e_7+BKM@Mm?5BUfEK>EumVQFa{(A>0WP|q;ZXpa%QLymLrb0`&V zN*l4~|C2$$m(-qRX6>&V{nS;=^ozG=~j@cVO_NZF9#FE8Lwb|RK~I{eTiuYXJ23)CcPJm(&Ysl z>MAO?zI^$@M``gK4JYCxOrSgDK_R%Yh;R&k_^UHX}7 zJUnY$vd(?JLKp5N^Daw$Iw>1AGmXnb^!vpm1A}t@^!~myAses2Z43YI&!>UCl0EK^{HqzCp%J z?vF42SO7#mcKZB!K7|H^zypt5*#prhDl31^teLh3c%qnKBQISj1llmXAY2pUU`K~` z-Pr7^)<~P!L#5#taX)_i$S#?E6z+{Vc^XvhSk*`&pe#s@4(~+RA@ux*92Nc)sf2*R45>oMCyZ% z({Qumf37NM9FB?zzb{^>KHP1W$`QU9U#PDMP7cwV7;_-{RSy^MTd@ZCTvp@ z7rvb$sGR6~&}pIXZKl92GAs61DG4XI^nXOctOTn}WFDgL(VlzHctKn3Ljex)3E0->amSXR+`J57KTzOII|k@%t$N#R4O#XLlK zO8^nkc#~Jg@hE5|;)L4M-rGwgNOJ^Z8nGjHfya{+#fMn+sZZ7sCpH;bQPKA|yWp^Z zHLq-K<~1Kl^`~?%U2YL6^3p5m`thd`A&BSV(&fug{KbjYH~rZMBR98f=l+jL*m9cn z_X7jBP;Oki_H^CtpWN2FweZA-u1g;HCtm9Z;kAyN{$F`5Eq9?56);@u{7!6MlTmt_ zTV2^9;k$&K`qaB8je?^v25P%Exv5i?-)H`vC($rqltv*EU=G~Ay;|s0x+?>ED_PLU z#1=8B1oRkOpG5p2#uEf+0bbrn{#L=&ja~B#Mfh(T^bI^H`I6!ohHBFapWrG|jgU*g z-Kg%<;-4Xt@$rl$v7BbDa3U|Kn5t@V>FoJeuEZ4m^Ap`@AAP7W!3*Nmsk?` z;9LAlYqHJm+D55DY)JsQnD(816ND*rFoBynSy>VnR+Q=r-nSe)e_t*pq(fc1?$hze zvY2E|4RMvfSBC0!ta7`$LT{^Xp2INzU@yKmOGGe%&0Vxuvi69(up%l}BE#94q5sk1zQ798S?J$#!diMUI)ZLs6x zlVc$T9Dn%mAy<)@m{_>|LrXFy@xso{(^9Yokg`o-^4n$LBoh8%Z_x*dS_4&(C3A#a zc}@>_kJz<9AEpEwx>11rz$qL$v{OQY3TT+{PqR{3Kv8bH@Le#j4yv0z#UfVSa~QvE+1W z%@O%?Tya~=;=i2z>6~{{CYQwv;Dub^3$=ph%_)4T{&j@q8wkq_juUAu-oSZRf+2KW z>eU;Zs2Y#*ie;PqDe5Aw5Z3r&1$#>t~`?8L}N#S&V7F0^hB2|;R zH%T|`FbuMY95VM2^VHZ5ngp%DA z8%pd-q%#NuP^)d^8ACRf(=GDFpPu8HXwe|+Ne%cHx-RaEs@Zf!x@_W(G^;{<6NlKSg8m=r*z5`v3=e__oDsR#YCY+An*We59xl zTSY)X3Afz6o}U8uULBmpHuv=w(j~?x{K|0+Tf7rlWvu~HB#v+uje{ zI+}YlhEYgA@a9EiAn^smL5BoIi$)#XmEt``{75H8b!XvmARWlRDJu;O40~(7z@p<3 za#;tOh@Goq?`Vy(Gr8rbuXJlV0*7X#EBSHX6B71UuGIMJ$1{Mv-s4@{J-@$M12HCu zX_?+2iSN);{i!F+&{=T!pqd=lGIhKXAR!cxF=@Kg| zOQzP7Rr@LAl=fA45Tj9^2K&wKTp<(fU{V?8!mWTEMjl99hJ+E|J&v^^`>;+?Wd;M8 zmW{x}S4XiYmwf6$L_}lSbF(sT#88U5@6YZ3NzYVULjqoF(s|?d>Ec3`^hR>Mxjs0K zAY>Qx&tYEwf~X;Ml*Gg({mni`+#gmYOl_G#}klRQ@V4wKdQ|6b(A2 z&Pj;eyRs9z8dj&wYF3NMAnY?ajb9%-b^~*1 zl}GXe{7s{uRMe|8RzVe0@CS;w?`PjwrmBcCv2jIn3_v&O?m zx@l$>pwkHdMIsI1`H0C7$>=LucDqgoj-t5W-@DfUa&?G^@g-JONI3{2R0Qa>O>Qv6I*Y|*1*q4h-c!ECI((CbhID- zlbepTZK1B#M8?odJ?+c)mzz#WI5!$#SO8WrK;LSN({ryWHwp;{#7#%yH25#z(5lzI zONZYd;U$K;Tl%}n#MM8wily_D8WJ!?t97ar;EZ^1QOuDKL@8b+4Xb&c6Ct9AfGDr8 zk^~GZp9R7`@!TsPhcXZ8JswU+;=?CSL(=u~;DLFO<39*`f7H?ZWpUBb2cg@yPry`6 zp)oD7$&$W1)vg9^13dlu#aWkqy0k$dlx}&WC?FfXFn?O|lixmM=@~3T*o8tVxs|Kf_r~n^5+@A31oCSe0fwN5JUpl$Opsa2(bBx|KwLe)$uq zqijn?$xU6W0D~K1k&DwB-GvD45IKQ6nwq8?g$~}Yt0QJG$>a7aCI8uj>gedSNg`Ub zO!c`)_BX>a1uHNng*6TafOQ2Ufo9wU)Rk7G*A&DtA**fkJ!1A3iqH)_pFaDE(Gf$; z;p>|nEGRAlKifmJUnXaprRyQgpxsdb?p5xJd@$l|LO#Dbf(xSbl!cXz@T}wttRKHXy@bS)SNPTd|Gf# zS;n=RhgCNM>Z5kdy<~Ap?My63tPOg1Uv!#`lhKY$urDafY>j~&1r^?Y7DzmpAB$sT`C4+ZejwM}= z&fK@rV)OF)#tqAk{NeUnO!?7O3H@blZZ|p`dAgA=H?YXL00)hFuSR^}+?&qpW~q9& zs!pwfcZI@vn2g77LND@84fd>=59U zG)+xoHY~de4<8kB1++4h!134zpiir3Idh?4B@k`Z=qEr6M)dE$A0(Dxbo$hx%HB#L zE>jS(F=#5Z!ry7-b_w!;indY-WcT`fT(pi3b|HJOwuZ(NnBPJ9>c3wZA$|S;_#b*o zN^ogg6#%HFt}byMQ)u^vjgo6?wFns#5D-9UZx9c^M4s46L8c7D(iBG;Nq_Oe61}28 z6`UT2gZOgM&4e&WITc)hG6$@WleVS+`F8zVg_*B4hmeew1hf(>(L~9B8f26|Gu03E z`@uB5LLI(7`z-wdx7-zJCwwuPsP?`|gflj(KZ#yJ(^$I5-h^hd`=je`>7mI^~P zUY!Y3l^X6hqVxPp7VJ�c;3S@XjMXZG`NE7caE1sZ(tp&8Gc`OzU{u+N~$GSlqcs zXCtTp@3B&&A@m;d1E6mho=rRlm%6Gl5!{B*h*;X|+XMPYf@0YCae^!+|NNhg$R!M@eIHPOO)HG%z6q(s-G zijM(ht-vlh?pl(kPw{S6 zMpm<)<=x_3KpZ>AKefh^=Pj6bV7B;qurGd=k6XEjir>y&PX0W;>8IUn@s(NRFLF_B z_ARz8r;~sEKjXujd!o=eqe)uozqGcJ#yBMFF2wi!4xs5!enMm-(@Cse0D=1*Jme!+ z3mYaP6I-IaF=UTJY~J*E_oF@70>qeK+D0pFQ%ARENJz-|lmr>}wbHM7=yV2yv=fm| zI0Olteh@}d1Zui@?fAyS@MePV>?X#dtY%w4a^+pSc3s?qmR;BYDuMI~85}O}sV8Rk zPR-rj{S;iB4w@%VGD=F0fsCeOF=LY&d*+gv>)9pLq^hH6G`pY#7EGyQkV+E$6?CwI{1lRY0TBsR-AQ^^L5fnb`ALG5 zl|4xV!Z(94izLHrZ0o{_*JRDr)7nNgU2_d7>d+GT5|WcuI5;>$`+LUt1C0?kJZR6}2CXXLy7d=sb}Qb@@nD z`AFe&a2yL#1}3KP|6JGWo+bUGr6M;~(tp?Ay>Rc!#r-E;_8xM4k}{&G`e&GrJIb<( z=N9Ck-j*A(qZ$jnX=>E!E!veCEgs0I(nfcyL;#3^`0b+{0vij52+goe{A`ze&2Vt zE&wC>o>&%!nc54Bnr$s~V8NAGh?XA|39F=$@3dfUIvl`_R+o7zQ(oBpnDW}QVHw;Q{6DVpx4+IeNGgw7 zzv5e-)4vIu%hv6VZgeo_q{7yLPJXfy0cCJ@1COKpKk{AnXTSA)l%F||)GNCvSYlOZ z7to|qV^D7>(OBKxPUvSM#W6XtWDAxVI0M!Zh4H(5q`mllT{p+P6s0}kDtp4qRazk{ z-Z^xlKR4N{6suf#vH4=OsK^yqWoL+w3hm zE2HeCH-KN!1ULStl@g~-pgHt7gpI}T!lmpY?h6^vQ8f@&35*VGhaRV-3ZN5pp=GOz zcs9{yN2wyT4dlkBgR>56`IRGTq)D5iZfIy&`2&XfM8fCMo_mbhlRWqNwXdbMwI&P# z&OP-}pz;eQ3Dk!U6SCL+h2F`N5^&Zd)C)*TN{V#gIFxGIR;6EZ=3y^ufc)rgSwl>* zr}m|O9mp>|k2eq!0s(eD!vTJ|$0Q*`F*}{%Kty)9H9p=W76Z1uujj{+KS>TCfBqjC z)UTywvlZAY)3ju><%31E^e%`ewqYzbWh0d-s++iMzwMj49KSfY2(+M(5dN!$8^z;NQ%@kw0|nv#$Tn&k8XEbt zCtb14RjOBH0#b=X@dIg`Tsmk~oA@5~LSkcq%UKoZlyn~{DJh*o6IT?X>2s9nh-fFt z@gNYAYd(JDfoG$91GBc4{_L4L{K)jABg^vol*kOotTxwpUCXZ>>V7!7oVS(d&Kv!=#zJsr8*L*D39W>%Ii z?d#}7+vvo^c$=XpLUo~t4D(lqw;LX;NgjQk6Y*R2d+mv+xIa^Y2H&uRCdUdcIyW~r z9|i3D{#N{F91Xm`I1uv;`Y{|ss=xp6^IHJBw#)c5a?iy*1~`ZbMHxdTfb;{w#h3eR zO*QAFSN==G6hMpP?|h9Wnm0}jEAiYe4*6oeO~2pb-z1>!HyK+D_#YpjF8r6Y6Feag zpiGYUntdOPEjc>_=43ZLul(;5Q;2}1-}oCR^_{PG&XT}RUyJ&gu-L5 zFX&>%tpS6$)>@%Db8>RpB!abMQTb^9eutmg;#ENys&c>x$mdcb9e-wtcLZSgwp_)IjFwgb#rciI3z3&{V{c2(j?qg9q{O z36Wm6i_L7s0qf&(`m&V=qT^Nhtgm-m6I%=X2RXE49-4uTLtyEK z(>nb>4f`e$?tOc}su9x1(#D3I;sfC^xxT)>Ags>NT^aJ$;+|ink&J8=*3Z&7Zl%U( zxp!J|myWfGD$X!P>a-;N+XXqa(k{!3bx2pVyeEF3s`YuA>Xoj&y)WXfSq?FO7C2;` zuA+Ey1abUk`DDqJr}B!TWyR%o!=7Cg8i8E%$&lSyM1B8ctbQzLCD~OgeAT#YPV;4D z?+F%>3=1N^S{}te{=3q2Tkm}%4U$DZ$$5#*vi$$eia|ckk}YSrfqUcV_NM0Snsm5ATPwuB2AIeqy|fyvW_FYV)t0K5E(1_cc{yHSXhc^ice>i#)yGt_lQ- zXf^xAZVTrm=M4HRqP{5TP#(Ul7CfBfd_{-!pcL#=jDNP$`3EW2Uf)tP-O}@o-1Caj z&7Wo8KJhL8j|wT?Y5zN{kP6mB{?yY_hXkQGXW=pqheNpazB8LOH;dX zTPfFz&NIXNo}9~+SG1I0?WXd&zLHmm=nfjVJ26_36ISZ-r_=e!^WspfJW1>NN9fe> zXDF)brrK8rZmMI_)eGsGZ(eNdx6ey6pl#+DR65%_BP?RuIk|w37VSW@Gp*?E5z3rz zAy?Z=UzfdlSM4i`1Pvs+M>3&Ly<3DND`>-JzgC-R2?}jE;5aD zPAXg`lNrv_Iym-q(BKEJ4-f_hWC$lz?Eq~q=;YGTG_yYHZU>)g*s6>#p>teSsn2uh z)>ppv=J&TOU+>Xl+n?X@x}df(GF^#x%`w&djs&G?ap`BtCPorkQ! ziIp`~RgY_GYE(L`*VD;q1IRQg40WE{an&s9LXxA!ei()@lC$Y8mOV+R1WUc+D>FJ1 z_Wlj!L=O#DanYB($nC-L6R%LksUt;KUZ7IZsFbyn@J#z!p7tIK;=|!}3GmnPP(0nP+Wgjl+UwQJ%EQn5MXQw7$!+*>9(oS{28XO_m~ps2n)cC zY|udjm@-OPtE2djy~K|wMQcie1cr&hkOT-2yX2fCJp*pHE_D8cz{(C~_N>#+&be6Q zq(=h4;R&7258*(Cw3P$cCkk=o5<K0%?O?&%iV74S&2hK1` z_w!QF(L`)Jh%mYaOO8u>jz713`TZ~nljOjQ-h@=PY{DJ$y`NMoIdyk-K}&3%eFV zpzYYcy_jWxs)`&96Bs#p5amXC_Nt+~AWUH4pyFta@i)R=QBcB~ShQ2GW&2+~fwiUU ztir@YGcS676~`P>HR~IQF>`HIs)p*sZGfPPw(Q89dbsE8ypa%aU6Bb4p^7SK`6|a zs9z7K0WmmU!)`O-pVM`pv5ksAUs`Bq)4#lW# zBb5><(KIvyKVE3%AnG2L&pgHsEEA(HXmd;u^2C2n=H_ih1?S`_IPf57aJ-YB7NXU~ zAMr*okv$iM5X4VD;N_zb#SINeuszn=quHTP|EHMvg}{jw!%a ztM~FkDmmDZ9BYz|@RTI9a2p4YoDMlcB$^UP4$+0u!h$n3BU^BY5Po03Um$*;%uH!u z=_0_B9ZK zjy5`trzblUjyRq^Vm>k2U$@bvX@zuOW}iFm2*oBNo_|QIxOgKPljaBHgG(kx!9t_u z$7OL&3OWxojf{kYOJfji;=vbyT}VUxUo!@2(g`l-E@)1qBqbF%Xz!~ncOTc2n0w>K zk|dqf1LPbx6oSM_4WXxnRh9jw<4+Xb^z&@%|2ZAP;0Ge_maB#Dy99)VD_B*ifwp0D zv9Vn{h}4Ra;M^buh5<%v;;Zqc&SCrQZlMT!A?r6>at5V4={+JODmi0HR8+rn4qK*A z5JVFp6V#NJ78WN^%U)cJHFL*CY`1Rnt$jIKnwr}2-0R=WT5?JjzVWO%= zVu^z&t7EW?ZPS4j&|5lRgTMZHj`eS_5bY3=hbb29B;qdHrYfSybRKYZ&GH&%7y(w` w^cG?}__k&q+cllP)SF$zV)Oq}wz)FH88~vFlC^<4Om0Rwq;fF%z^P0B3*9*LM*si- diff --git a/docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png b/docs/incidents/2026-07-01-crd-id-duplication/zflag_diff.png deleted file mode 100644 index 73d8880a8edd6cd6698fac54bc699cbb3fd47c2e..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 26139 zcmeIb2{_ejyFb3FUG3)GE)|MhQ6VHG3bh-Tip(-rGA(1qkYVq3*%hfJ2^q>fEy+wM z6=jI9$ec_eWS;qbo~3u6_dVx*uXE0Ie%F7vuH)+3+4i#5_xpXG=f3aHaNqmpS@{#I zmUAqpP$;XYCyy#oD2r++l!YgMT!NpN?%B8kfBj(dD^=x3d|mzV(x3QucH3hbw#t@< zwhkAp4JbwymgWY6Hu}~E1{OBPmbU$i@?`L$9pptvtPL*Mnpj%=qGDohK(RHj{$-EE zFEl%|U-s_VyYCkf2`LeADSQ_cAwi-1LZKc#ta9bi*G6YsFNdbNxtcqhkMd}}_qcaM zDtueYt)E8{Z1VTpmE3QV%1*hZCh%G1NR*&x*sfbK&MMB?x9{vzi(4sZQnEiqZu1jG zl~mQ=<#>F<{tP-f+P_A&rcrR&h0dcP-l@;j#yXw)dz|_kcNZ6T+URimlo}>4!}U@q z4f~?%ELh*UAK22u!TN6LTE+tMjnz;8@&}SN_KO^{Dk_wM4mijM2M6!@V2s2xq>NN zQ(PSPAe;5EDt^zOAH06!28ZM~zaMK>XQi!gC@#KjwTk?1-op0ApGy{%lJEc7^^IxQ zxZ|qW*w~o%obE4~%+!)<)|*;zw`2w2$6amjeE0sHlsM;n)kc0cCWg4ptwS02oa%ld5V*H_sK-~%YG zzby@5E>!;dfFmckw|~YW#nkR^;}VBoEL!t1XeT~aIZB>couK6#5|a4sn^oyUkt1&7uedi)9-JEM_rvuJ zIDL3>439`zPVQETq0M#ucE_pSbhZr}B0acWGfG6_n6dc8uJXr6+Ff(?VoSz8n0@iT`H|HkDy zexWqzKzja=#_kPUWL>p6B`m7&l*BK+{b?2FzSDXxBfdvwCZ?w9rCnxT-&{v)&dt$G zF@Egs?!LURXs9F5vfJM%m~ZUkZQhM`Sb*#7)h!E_)+&A=sH~(! z*G#t%?4nscxKv`{XM~@Vwc&F5vXxp9GB<4`sF`wd=^8GTZ2R6K7X{o;uydcJ%k)6u z;P7zOeLm%)DlwCaqn9sVzIFSyvd@k)_X7g>shT-XsmDDxJ>GJ_{yw`_rnQ0p(B^|z z&dN@IyEZx2uh;SMfuPp?hYzFeda6x|9|R_fMVE|irLN)-JBzQ(LTue9%hv5Mo}He+ zF1s(n#l>ZzzH!~UuoWAHj62^4b=M|$o^d93bav)pX2B}o`}cRr$mlL!ycoMND`L#P zw@%fxhM}sa_5d4#jhkB-i{ICntv((cqZ~z>8tv80bItjA-MUo$!qwPp>Kb`&bFL%3 z_3B4FR;%LdJ=lFtsZYaEW^URh)vQ)YS6A1*wX-aw25UOf0sygA-=X0kS6(_Mx|nBCmWSPxmkiY8n)r9lQu z!Fc7dMu4AR&QDlR?bVG{sjf3l$BrItsF18?=)_%kb>m%4jr#ld?`h?szf~vdB^s6n z_-eTgd_I<-k@RbvdZJp0j0+DnPAx&HwZKc@`Hj51yoQE`w!S_KiS61=A4jm=RSj#? zOARaE-rGL%HMuMnwr!u>Utu0w(Ry(AjC>;G7@Oa01~3?(I{`#wHl zWIbR1xwa0+L2&U`a!`uizpu`lo3Wgjm@vUDI(SDX^=8q1r@;pxH^s^U?Mw0LmSzJ5$9m}+cnwCk#f;O}3Hi+$01;oSbB z_U`W3j?%z5Jcmz$;`}rS6h3oG*H5i$L>q{;qAvgqbqu3$3I=O zzI^$^ShJfjb@}Sm4|uZOURc@KROXB|#o?~%b0h==P9apQ)~z8Avi?NI@d)Ita|zXQ zHF|-ncGl14*s#XIB1fEaaJ|KE-W;P+sRbJL_G!6OecAP2#yjd#O`|OHW~zm%=4Sfc z$O_)4)1TbGzuvs*`MC-vZ(p)e8Ka_yTiQwG&6_vJqb(3wut996$1jIPW@KnY$pQ!;U0riOx3;nl3J=dF@pI?SNx4j> zybn6ClY0D@mCKhaojrTEFS}nx+_Fhqzwi$8_r)vPdV1mz*^GOt6DHey6pC^zu=3)x zs$FNWA#-zcm#IVn^u3lG3MI* zb!z|R&wo_Mp9^cA9Zeq`8mg3Nz8dzDCH7@#XxIeaaoxH0uP6n@!j{q(x&P@K|zAI-)u{+-HktpwT~Wc&JMszntpom z+u*lvcd;!l>g%hjsHnJbkFUNwJMDOG_lD~p9#}-(x@?EUpMU<@)5DWmQQ*bNWau2w z=Zq`lcGK>yORdzh%;BYy&4P7q@9$6iZPC(L^ZGOrMAZ^BBgmbz={h4AI2YI1l7CJmcC;^N|FheGCd?bva=rKRPT zmzS!oZOSb6N}O(vQ-NbgSx6$eh7GrmAyBK(XtCJRJ1i|N!#b|5kvVH?YwKTYMxI(~ z;ib4PPMtY7q}m#iK-=3RuyDo$pWo2K9fvwhM!(i-W?DxebHwD$^jc0%50%+rmpM8) z$Q<9&82#F|t_ z8gMz;qnzL+FM)0a6;p-&zKF5rLn@u=?dkmjjF3k(X zO}9-`w8V6XB;@ad+zx9-YQpe0%ns z?a!ObEx5BKuAxDj1mymlvEZcLqsW<2zaBX<((g8>B_bjsW&iadR{s9|`{4-3RC<$R z=ayr~j&+R>v@Tz>M*ZT&C%F?HoI#Q{1t0I2;8r;#t-}q=f^|A>=QqY6NIK1o)L4>c#;>Yf~1fjFMuVyZV4 zw7Y+71Fm24(c9{b;;NCnz^3Rtjx{v{0|Qc5zJ-qVH|uq1I5}nU;CYu@9K=J9iHSMx zg?-&99bH3Xc?lz4q7!=7YngI`x7SRess+VQE?!kEf@n zmwPjfGh}Q!Ja?Ra82bA4YtE)llfAWb>mHRF3W(b0^q3aK*MHH*0Hj@)yN^iR#_|sX^C*1Cx z*KD75*iB(c$#d-N?Dh(7LVSE5Iz!wT5vTk(>NjE$Ii2U^l#U<2AyT|~{rYfZPXi;P z`chq^OP8Wg`3W5_ofzsY)|M3&t?WDKJl2URU?5&vpHm9P`qF23F?>dHmZ#^gc)IlDazn6zSbFWc3!QFE@7vpGAG$ zmorAHEfYle0%z1*DK?$*+xMqiHtPxr3E7LAn3zaexBaQOj*Tq>r=zz%y)3#GA<){! zrk8eSv*Z&Lu3m#pG29{<)t=Si-OR}4FDGT8QRaAd^S+uJ+xwEvTR=5?DC)>txQeJ2qZKZKny zMQu!HSiBKG|AK-@e2V@gER1dK+O3G*cuWFFz{-)QKcJ{{RN7DC+@}XsnVi4rT^%?E z*g$K@xQyyW4OtSUs&V?#fhWgqw|)H@k31Xu^yyhrBoz4ycu6yOO-cjA5naWD`DGBD z1$DBj_Uk6Sl z?N?NRS*D|RW|41)2aAOTBqcNRoqru~{DJaG=%q|eMt#+Q3&ZmMT=!_>=V&%k`PZkJ zBaZV53kxG2p$ZpdtM*1f*x~c;$2|IdU*8DiK!Jk?wNaIq19gUd{HW31(IM5)JUcTb zZdkIZbp+1?$TSiKhs#)_1KS~NH^a9S-f;<@x`Ps zz`X`CZi%`%Rndyu9i5#K#5DQ%_zDIVEL!sR(%vrAX?Yi9$pZzv)7mf_ceZja|H|#T z$x5J5;4XDmVV0noDtB$^nrc*Z5!DI3V@eHAFI*e#Ym5VK&K|JoeBXu7!Q%Y3ZQC{= zm{_9_7c~Hl$`jl!ZRLdnV`J&gE}vg7OT`WcAj$ylZMr&iru30ya>=;f?BGUHpb;sLC=0Cf{4SsW%%Qm{2MoJ+_K;55bBYtuF5Dj z4i0`Q2|##Yaon!shjGw+cyIx+D08Gx86prh?d>!;hvVz8?7{U`i&xLA8WO;Btmo!V z{QdWZ*xKx^BpYe$4v?~=`PY8Er9HJcewW6nimDjpRF^6H1id`nn>TOPCFy(6eS8Gs zoAccANcg}h2TOD2)Pvo84)F&=+-5UT-JI*IZ0Rfw+^MIhC;jckq5x^_r`gGxJc5SUsuFG}F*1giQSWx%522hsh zojK4oct%VTnMr+wB~$|Bo*eGd->QtigM+x)$Fm_WqyGDVDmu6Y0XUn#V8Pg29awvN zbqhy3DW8yv=ydvUgH0KMos$5KJXBOPn$IrXRAXrK`E+%hx;HpDSe57I24MY#sjips zv=2y+pP&C(;MTE$0g=%iiL~q2ueS@`4BmhFNtRttBx+znxO_lW9)V@}g4%%2cwdK& z*@K!c`{Komq$YCt_Hyau>{LIt(5Q7HsoKdA3fyO|1gzr~!LvqYJhX9xbp^9_DlvCb z4m&IsU@rLTAXO*ZISlDTL3SpEBnDny-mYTdJQbwUk^1IYHC&8+=Vb?n8f5&~&v6TJ zzgFuGmalBLwzCuGGOm7nWNB}6o*wq)hr#yZ5n$I7^iY{>G4QPyhd6*TlD_%-QO4x&)&1Fk7{5Pj8)r8@8sT zq$Eh1jM1!=l&VeHh}ZJqCBPvyrf`$pc1OR1!=MCw*p7RTcN5OIVS2Y7u0F&qal|e9K~LB zaberEDcVoh>D|M}2{dV^;j7$?oyZw15};Mb`zNX7Dul?8s|t3Tb@bwv)kWA0IN-o|6cp6U2)citM?yj(n|G8v zk=QjY90reM-E^^ZsP*;+meT6e5&&}$Rzl=LwxfDQ?di*-?>Hg8Qz2LdHF1$k^Yp}! z9mp%x6DLvT1srsIhS+5}mE7`wstjZwAJ$tNY~WUB$t(t$BhP!O_nqu$jI zeL2V^)i}URoM9`iH%)?dYU}DcXV58a*L|NbxR$vS$5Aar zKO0gzZfF=uSQ6y6(2|*kns3?5Pyqh!!nmA%A;k34`>Mz@oH?_TJ*v3|s`$_XPj=eFvuqfN(yND_(N(2~W$j8g|@!&h+-W%Tbwf#40gURZc$ z=;1{MhK5BTpXBAIwU*HIkmdG@ijtoL&zvA1aF%U6&`umF4iB^%;m0>1K(oG=vSjDZ zosBkS(kxwu=v-Fnzi6wAs;jF5{QVx$60NMQqjQc*gbbCf&>*Zpn&N<&X0ue5L*f^@ z$uChT{{<=NKNpMs|MLpwl?+|*@2Gcxr-dCI9r?fyYo(cq+#w_#iu1lG95rB1#<`Nc_P!Mep7v#Yuv%Te5odt~G1cASdwc+oy&d z&KzqYkDlTX0k}=-71Y0WpI$5=#h0}6XccNryWua2VUd>2x!}-6U6F~+9PVUf>?Z{i zsRmL_{e*O%N23gn4iA4F%`S75#RA{G8~Vo|KSYFwzrC2WsqmDX97(<;`PVRXAhOgT z#fmu&wsp__Og^6Cz($lnru{k@85u&|9rxmhxVQZT|IVE!IQNIFb+p=3}#dh`!uAXEoCsmLMpCr_RX1BdXVrowak;J#pyZAS?^ z7`~Hq5SilEZ6}cdz#qjaMtCK^`h%JSwsrFH3S28Pf8nwOfUhYebn|lX1RQTe!Fw0C(+eZN`mR zb{wSQ?`w$p^UojkW72t!9Y0Rw1XQS%Q6a9epypJr$Oo40@adYHwVh~pJD_BfBWcrN z4DQyTKvY!J4p@A7GtnYd#1-gztZ|B~ zmMmQ=uzUCEW;c-he7km?#Qp_>NDIt)|Mu-!KOr5@fT7MZKS^~2SrM%*pha?=$Lh9} z0lWh>`?L4z1<1H$0Y_hY{=T@Fr10sPnaoZSQ^kwAI>RJ{7xw+q*B*a}aSGytUQ3eZ zvYMLYS~L4o32t2||D-_tkP3;4)aiHo_HDqHSil20dPPNrZQI+mByiKiqoa2LA49!! zar*MugNKR?sA^%sSoQ8PSh_D=cJr|+Z8u1mMnyO0R>0}Ci1+Sk^`vvX(K zDmaSPoK_@0O3NF^Nos%x_sx@?F~dV!46V| z=HoQd9pNjW9t&FfI3S+sdQMK$gQ{0irV&l%@^k!r5=6eZMZ2I&Bk;zfSrk;QK zd%zT~8($9BrIN}MpZyL9h>1N)D7?(DmCoJEK5n!Hl8W>_I{j zrNNhWP+P*HK?BK^3H`+usgT52RN4C$4)-MHF(jI<909?~1p9?-7LFax(tJ-qMIzel z`1lnRjHifpfQ+M}q;#=rCAW+=m@3q#XHn82HQ^k2wgS9#H|OOcouru72)!vRG(~Jh zF>GDx!cMNlJ^S}}px$ZO%ec9@OG9M!2vGd>*I(Cz35CEx&5}B`1(<=TE&%mKrKP?o z@}Wsf6E6YXwy!}Fp@KHlQF;rEQIdY)`n78xL#2dS9CH&EYAc#r^+vfz}dF7vLw-chfHRIi- zxu`$>k@_$77d-*I!Y>-^?;is?xA`e4jzjq!CsP@{8{GMQL+&Zo3pSJshb*?wL4IK{ z7__G6nLIIwfOVjp;5fM0U=$P_%s`|Jli9xg)kA~TP*)Wd?-8-tFi~{xUP2aybs*jv zFarJkO&o&%@1(hBnh7ADV}AercVYlYHLZ~+kRTjl6Ryya9zNPzPt<2XEhb1UUg~#t zU)OEMj;e$;K@I~iDkD8yeuPdkFTaFD&5xgOa0xwz6zTa($is)bfa&n6+q$=&OXSDn#V&e$(UO&CVPN3l1FyVx?OH+SDj3Y>b0MOU^dSgdv&8P1q?xWH%i@0ESVaB^U{k8o2#u;FFMo${?AKRpq<#DJ zl2FH~`+g*^JEWr?W934eLV0szb39aTm*IO07cBy6js+Sg6r=M$yb_-VZd8sAw-RxT z5z+$@Q|`=}gz6k5>xWFP^2De)JbgwbdHmMwzRqPK7F`uVB<{kzQ+2_zb$ zh9kTPxjZBT&{9eDprjL45vq}LwnkDt^2-{x@oQQ3z3|tF5xJt^3iKDTOKhJ=lO~Q=u0(@OvbLVgOHZ+A)I{EiOY0zSM9EmE!mphO-35%MK>;3|mtYp_ zKa8hlwLJRSutA{l1$c0H}9?KpW|1`tYnm2?4>MP(nN=pdb;Q z>QYR6!sC+rq1^Z(^Mj_0Ag(yj-%#A0hDs$YyKGqd!ZqjgKIGNpCskD{--#I2rbB>8 z3l}aVX-sgzP(yQ&Y0IGk+n64-+) zCn_MG`5BaX`}+DJrvTz5LDuz}1a%BdN<+CpDj#sCm-YpEofo8H`5yoa!T!evJPU3E z_`JemR2uAoTAV6CbT^b@f7b-HI6(3NsqZd^SR}6cSuU)(yH@+rv;v(-hEXi%2+ppw zL!WleNVS$h0SXH-!;**ft84ZJo&m?H1d4-=o!!K9tfxl7TYB_iety0!tQR@cgT=5c zMgi{wAx5Gy$;`?ss;{r_y9#KIddZImsNHYsX)0u%>;QoH>{je^Lip1d5=QSg0}{oR zT*1gUo%3mN_bT2LYI_Er!j>spGiIDV$2Z5MD$L97H5Yfg3J`zING>)f6 zG||_yv&TTr--Vap6vu#kRce{Qj&7_N8GMk`EMF@8Ppk~PR}g^eWj#4zY@SY0;Tx06 zmvVt_Cgp}j}R+oA36?_XeM2fYCrt*OJo0|&HlPz(%H5-;1>_yh)O z`CkUTC@HB*tGB|#2&Gl^)+_&}r8du8sC2@^m;o~u(f6RPf}c7HU|LD&a!6LqN}^aV z;pg9emzf>cM;1?LU?1qw$*?*M&xj)QSDqLm>cEc$yRlaQVjw({uYasTMT1(HiNos| z@V(4obii3j=rJ0?Gg({IO3f9WtVlZTq}pxL0;&Rsb`pEK{p(e`XW1w>6=WvDo1A(Q z_SRo&e4BB(br>=dG_Xm({8z8MF2h^~qFM)1ov6|KDv}N$g!h2m3J}r1As-@>fKc}i znk-B8>he*L`2cC1*qh)&C0doe)@8^&3PikyKEw;J5~!0B3jxMbt}BPI-TqeHfCrPF z{1WEdbnAB1b+A97)!$OA8qE7XMO|0t*VY?Cd?H?HKy7u92J1I$urDrEu@xT5?$3<_ zK}Q@A-ZGQXxs%-jAc%JD->(7rq8zuOPK@&-u(Vd=qDOFVao^1n5U1B<2f+LFrl5d^ z-9QfqZD|5~CghNNiZ+k}u~p=;P(DC-H`Z`77O&WFKYKX);2)+~<0_7P@xFa|53F=h zn-aIzxn#qFm7|ICEd!}Hth};Iw`^tz*p-byITM515N!RS1g-Ra!9Q_va>ju-RC6gL zM~BjI7#R~sgMr%H1Pr3RH&;LGVldrmnU}I+r`68rs=7YnVFajy^**^#x)Q}96z8L< z;y4AU2ZKVDL~F23m+vwX?MLmrock;buDnP@9bkud9qEb+;Ydte+)aFB*gMQh7QUl6 z{3$26A_Ue`qB0YSfXMlD-cz=qTM1)$z@aa`k83`)SMrP7G@|M!?)og-E;^`@$k(rr zE?%;P2BS9Nn(W5rtBN4CuF}~06q7L3I5kxxVRDI-hI{-gYXIU3B0u7v5(lX$liR*( zcRgYy@fSJ~`35bBGo()7fj#pizGs^S*`x>Qr~*6LH2yN{eY$Tgj|xnOH4tf-nq^{~ zrkPL&`vl7f4k)Qwzm@eeJMHT&SwIQxvku3Ic@Jz^A}xz`7Cx<3>f($5R7`=z8Cem& z5Gwf)luKUD35cV>+rQCF*OT{g&x%!QnZ&u`IC$kLAu_qRgs77cPud`|Vu!@S700am zULc%{7|nF;@HeW;vNMGjLJdy!Dwu6Y5JP4;< z!X21iv}{;+(69{8o3NgQPGm0wxk&1n=X|XH*vG?mqX$(>YbQY*W@hm+lV7%z&qRV| z!Y2|AZXB$1;*KfOS3wDVA7gdy3N*({d;L90g1}}IFF=gtF~Pm^^YO)W?3-WYY^+j! zx@F=!(!YbcNndpnBO6G}$A$IpR)w)umHE7U0HH*@2c*Y45>yY`RnsszQ)-IEAcww> z!^_re-X98^xCvSzir_FPlbwqv#03&aB_2y$_6gQocBjadYQuc3hz(J3p#S-Hqs}V8 z4XphP>vyeo#;9a)?K68wSbYG?Zj`6Q?}_qYq(rprOWtfxUhKbJli&(HIB>8xSTcF0 z^)Y(e=g&`I2a-p{`)}ngDQnbfKZA_vf)a?ThX85Vbk7FmG_D16fxPb<7zp_!alZE@ zZ$~9sgBF}k?NF5puD-?Do+sCDBgpDDKxB(ekz_rSK3m@`79|XV+VkJjgVFg`Gw_@9 zBM;GsNrMPT=h&o3)-PuGsBB`uITZ=J3AG~rTr!y1Hyoxc^~(JK^}50ew@<$<9j;PV zhynYF%Oy4=sPY;Xy#k5PmCXNEcVpEN31Wyk8FF>-CLTkYS?xBX&S&jAd~VKP(70;c z9&PYk0%&%ix3#r}6M1?0k3U8y70h3XGA_jpltxO{6zd0{?x@CC2~u~;KPvzH<~9sE zBBYFlLuK8L(`P|f#3r+xQQd|Sm)||qS6{M)^`-P?SDzB-?h~&lMEOh)x5c+Nq2!)gpyG~z$D);Wyt0Uwvf*0$h zf$%X%(!e%0z@H=yG*;-Vlk~W1^Zv7Nq~mr-_<>U>wk??zUR}3Xa4>YVwTT$ve_C~j zCm%&`H?aUi4Q!Y+gz*obbTFM8(snfDI%LjGcMyvtwX&%JIP0|NQedDp4JgPJHj( zQ;bPS{dkM?R;J!a$goP!tWW<8D^SJUblDu~R_HE&d<_8Y>z%xT0?zbCZOAt~KzAqT zq>4w0P7Ku1s6hY625N#!U=9cKCI0}`y9FJ zJ*LEtAGH7Suigg-|Afy>fbELFUhIDJh73hwuY*(8%ya4D#bCJOHgDc+<_SylrsNX1 zMX^LmXnSe=Y6ssl!otNWcu#Um?4T<4zE5`O*UfH)?hPBekZ71oDWU;T2oH&$X*Rn)qu{E?iQxNQTtNMI=K0iVtkEjx(p z%|T50z7_?LAD}akwlTOKAHyk9x5e@wNz)3Zb1nh?VCJcG2zfIEI4B6p{Ev^eo` zI+fftKO<-C75VtdkB;Z8Q&R_i{&R=*?L#f;Zo}Www;Vn=x#;}@y|Thx@(iq~u%{QCVH1UmjNoJ^(uMicPoeqwzrs}=`|O!g#h9N-RAeL#d9v=gV=^&> zfA8oaGzgF6=G8EY(qLSg1X@EXk8_*tbR%9Jal-K`q4FgvK7|Oc{C>f#Q3;z{vXR?qxk#)^|RC!9W^fFEgm@P)hKwucSS*ILCeVJ$oH%kXe(!% zF6kQN3)1u8H2ejvpA?tbiO#GZMs^As6NxAT?&yq7RrkFsWhC`9+!)giOEr1fpK0Iq z?3)Ld(@8kw#L~_81=37SNe6AyVwR02qc_v`+&FNcI4U+$_%fMH;^`t5v)dbmyI~wP z?dD)Tmv6>G3EE@b%u}R)6cX0(moV?xCr^HDE%a7L9PEQZDw09k=qMELjzMR}kt0V^ zty=OU<5|_N`-XSzR>(l{sPc$Tje3{3NYEd;e$%Fp_eWSC@o7LiYxh-z^6u({K;mM6 zB-Det#EJ9{Cum8a88HC)UC&b=PR@a zZCQTTxutC1PCLL&YWV(9%(|WO#wgKQT1GcKg-lCy0YxUx`&HH$imrSWo%jmUB<``M zYj$qYN3+pdvS zx?oZzdj5L&bpw0n`g@<&nz4{FT3AXWtY!k6r;q1Nlv)z$-qA60cvOh>>lU0syzS^3 zRGS<3o;&%Jg;_(ngd*C3D^b_bP!{9FJ{*6$eKBMbMA&P#fH5rctTiRCeB z7UbqW>O4qH`2m;pHy1Ezywt(5aI)+aYa=s>(k4(VH00L0+5+#TQ}1#?52og-*d41D@rbN`sev8T&BIe zhS06(s|w6QOA)>_dGNWg8wF(j^=SL%9RZY@&Wa*CbU|@k4H=o6?VsDHd6@MpDPp(@ z@TF0`(1+j)Qim2w`U?ZY_sqZLa=ax#Opm$S2vZ$?so#zZvhcV2g96+O@`aLt}5h1}$*P=2b2#UHR%PybmC}64Bz4@b1<;j^nlB)0Q20 zy-uDqHxlmQ#_BDx&iy&*)aCOph#)V3;rEQrQc8wJ(}>=xA5%0=t->y&PnQAetwyd>r%QeL92hGDBb$f2qBuj6j9 ziO9oZaHQMyH|dbxJL12DHTn={2$ zxKxF;zbS%##R`PI#Mj98BL4i^@1rGde>a6_9Q{7|VB2@+cE^rgCw|cc-JHoR{o{8wL!MnYWP~mq z`5Cmpk+cbdnly*NT`T}aR9sxV;~~-tT6*|ErDER}!RiFvpjHoY6OUaRRBK(1b2@(Q zK2&A^4T)JQYeQ*@hjh%oAnjBj*H_(1s_#YLZ$f$^SW}dKOww0^Spui%;4B5rRGgF_7gKw&iLos zsI!l34`yMgP;t{ytKM5Ux}<%r`(siVaROuaDS{2Y#{+8d3(^9;2#0so?^s@h=>6Dv@+2^WxTdv+rOB@Jn!2$Hh1t`i29fzm|}#}7q4=ZG!sQ331@`$rsLy+Bi|NMV%` zLa1hj)rFu@5KKK_QnR~;7F!K1{M|V_`-jjEzHDrn**{POf3n%8$*B;srnsnxQQvvF zg|zqVoohtP#t0fB0f?Ef?MI_MOV2^})t*bqGjkz*uH%*h`V4WzdT>3(4MKu~s_;c( z4oFEv+ab|`EUT(UUW(WI#9-Owi(Yu`+4&@OV0hvw$nTn7+^0(dNO`;7I@LmNZa*Fn zxUpGTnjnJY<7!X|k``#m4q_0!V+RG?CRGZ(jo1Y?-NHgb)$PT8J@LkKcmJVSY%*IS zw&GReVv3iy-lqS@Em*66|hdA z>A;clqhAyZ*mKv($`F1?9c0qRhTR&Wa99wgnNCh5hEU)r2*6ZG(C0iWYbqoC90{1T z;*#zXM7mS-Nco^xz+)&(a&*#)3`ZIt-2X@uHAs5m5NS}4UUrlqA?8!0m~(~xA#W%MiXW7W7Q#9EFFKlXvxo{ZU zeDi7#pk$4KB>}|;n-2W0X3SS9;;!|A?D1FFrKbyEe5nG9KJ4yth zj%Y@3u9Eg75&Z(<4v^PJ4F`+nFCMS9dc1lJt2W~7JChRNH!+ZJAYB9?<-oQU%V1ap ztYhWO6wOjD2IiF5HoJ_OK$jjF6$YL}(ys!%Nd-Vb!07&Pf?M_6Ie*C=XiSG2K@C$> zNHh81pf%!1Ok`xi=5=VA!NG+4nsh8dk$`(<&y*>RMnt(dKn<9t$*+O)7k3{08hOBx z?B+vpmr!wA)qB*Nk*r7BFsRLmnuG<#CxM^$8J^*{J9+ZY2cTpjr32C)1^ZP^(iRzq zLnjmlI^%wY8C(&x86FXQ7Q(%l2J_*=hk3&i(iWP2ucNa)6}&qkz(FFwAqQPHB7!s- z69xjfNL$L|uM$7w`5MaYj@kyD1?A%pvdpEHl+#5>RQ)#b^zn43;+S^GK}RK=bCSAh z4b$t1c>@kwDjjXtwiAN}Q6g9`(5Bv!5Fw+%z^n=pthl zE#tksM!nYT{haTfm0|Mrsf;bk&Es0^HVM3^9vFicv8yXX@!Fp|#dyiFK;4E52%Su@T6~>yj6{dBX<7w2f?R{&6KJ_pr~1gkRZh2df(l zoR{uMLSPSS4Uu(g>e~PBvrIYMZ)g+RZ9{}qP=Kh9MbSrJ`no6o1LvBBnw#>R7#R!Q zIgxNoNy0%|Nv!WwWa(0U+UKW?6Uv(Aa0^ z$Y-PY#%}N4U>`a%__b4;lK#g(W z-UJWgGlI>7k45_biU+7znW43)$!1l?g8t#7rfn_#BR_n4{BYfex+~*XKVHOtV1$00)!&S4P3StDcZXf#@h2s$bVQUa?1G;<%VnyA?gr4{BspyF;wdXg12LLq z%$sv{l9H0(&r!#RkeLG@5~&s3v)`7!dHwp!$0U8Q`ewO4EO4H3g^Qix%`MA-jzMVH z6DLSFFChYn1VN@!;23vk0DGX}W)~V<;T=;;&D7D< z(6pfv_1Sv8`?TkBU1SWNBne-T$#sJ?fiPPcm-xCuE z9U9B$6a^$;j48^a(vg~&Xl5icYu-PMo7xD<7r@R=CL90kc-V6et=X84pb<5LNKRCJ zZCOQ1`tDyc(z8!dICRb_-oxMJ3t6djc;cH!`gXly>>@~sC2nIF&<13Xh4ELZ7hWwP zav1$mvgF*&_j^30%xkjjPrJ10NWTA%DIytyb@``U3WrC{^)LypA5!+Xj!b{pNU``s za&3ZFRG9}te~v{+Aw&ukKyk1sOSw*Gd@nT)dR)KmuaDSX!u*PPjH>GDaTysOO-8b! zqfbJz+gtbugz9^vAV?DLW5-j!+Gl^YcMh_j@7`-0YH_bFv~pN*s8xUU0;^L9XGK>g zuYNr160DP%n^!0at`snc5Tej3U|HCgoCoLy4~W<`8+fbHAskcyhL7Z<4$RNlgk81^ zhN;cTa9rd2hx%lrQmlLn6ST`%+JlfwmbDO!}E50b$~BrgDUl}34tp#@_5i92X#ptJ1WWM~3N+d`jwR?wCx z#nM1#8&;SbRv<%~fH&h&!_Yv`5y;GA($|m)f?VaV2Dt>t3yXjF;bEc1e=&C$quwla zenxK6UG1gerX>zEjkDOgs9m9Da2YQV^K*Re@IykmQlvg5-JPtbN14Z}c3JPd?`pon ziGRiG{1xrF#`r;wx<_6-RomV7v-@#_@yzUn6!j$Qd1&;%_=fWIuV?lD4Ay%vTxXg{ zy1PkZIXXH?uZ)8qs{}SaY%ygu^Zr2pF4X^d)9u1UAtG`q*w+fcmcG|(GRWzTRU9tb>E zL{Ue%EQXebWEa%Y90%;pa;84^+9NMTrNQK|&m@S_4JvQZ+zIgAB0CqB_SUiW#98#f zaRLcO38@>yJ`)kNav-Mzo7^Y%A_u>D&SooFMVMl`0MtIL5PlwtJU-G@c>*x4g^NZW zy?ZfUB1nZp7s{Ev13N|ge0u!|W_>U-r5AzY#dUu=XLj7B?5BBVS7^xEffbf0DYpM) zm%0~fK;i>kZIqiRimD!)%iJmiP!Zrx4hev%%#Vm^N-DYS?u>}_Vmq^{Y8NKgfFOer zI>w;{R%o${uYZh6L(3b{0AbCvxI^Adi42Xvcv}X*L$lTHp>3aZfi5wz_Y2A?Vxq_= zJ>aw<6JX$5Fs%rulHs6si_%IBZR&EJqKZ>{DU=i*xG}52PWYmE3VSZ=IX5mw28+lQ zft^VFGCVPEbF-vT%5`q4xd)IwQFI?zNC0ekb5N9n0|SZ}U6fq4f^tat?YnpITi!$1 zC`k(l;2>kT$Sg&`-nQP}-jp@uM#+1Z?pcZR%d97pWg$wuh!KP?Onmi;iIz6^mr@S- zY=U=fHMh*ys(ok%YJ>;_e~NCT8s#QumqnAo6)=a97Cb1Uq*5e8$Cvk&5IdiN1H|4hj48zc78o8358x8dk;hkn zhecEa91f7(#Mny|DcFj5sYur3OiAWqBa;?MSB!e>fn2GIZYCnG&S3DCT@Ad8wn79mH$Dy4vDY>HzoQ}m^fNYxDyQ%UK%EM!MS5vT{bKRWLu44 ziXXy52Ew>!PGV%Cl8@^W$_tx`jBH$lCg{yb(p3yQm?Avn;C>o)D$H5)T*3O_(7US< zH{PLje5l#6s9^qK2<_{$wlH1o@d}(aMN?7;($1$P#ED#MmL??&?KP)Q=79azvv@Z4TUI0TS zpI02`SLcQ$CkYI`l0@RZ2J5hS=v8sP&V)=dL_7=ud53YO#1n<*2<@m=4=!oMq`>BP z@TzFf;#4QBgW%=P`RAX1k|8q_aM8!XfKR&iY8s8uX-y_o^6+7vA~qJwO?jEb8qDiW zfr}pAc7Bjiq`X9mEfIl6UR9&#mH^#m7k-7zJVU;)=-x~wpTT(oAH@;YtY36(6N#Tr z`f^EsN2P-6;5uS%Abtxn<`|9Ol3Vm4>yddZApFr#@z`yq#*GPUdkhTXU9e0Lxe3`M zcV<+}&-(!A9~Wq{PT+A!!HmT@1>3LfWCL-)!_h#7r=z(coQ$4C@md9;q*SEYbt*wE zUgI&w$10PS2h6sLK;ux=RCdOPF~{+57l3^-rgIp zT8ypopck+wJ|DbwVbcwajkY|w7qcxbN4e3cboWxcW@=GH055yqBr~z37mVpE!(?`x zYVnn?_8`EVKYIK)2bj8!hwz3njWNX6@y?MOD@A4@TsM8qgKf8^zu%IFhv%U=mhJ|R zKKMkTE$J9$^lfBBoXp?zpFs_>7wsQLC66R4_pE$HhU#8OuJy#CNCgpxohP(j&Kmpr_0EBSuNgS^ zWSB3Us$`xfl}v%dF(Vu|l{GmVX%}zQ^>Qh9%9Y5u#K-+Zzb)k|M?hl2QtUJakv58~ z`J$z(AI9}TGB&O1|AA6I`0bD=g<|muV3*}CMk&;k<4pQTkmoxVy^Q~YN&v#Y4k8or zsH+W;>r-}( zU#1?xvr49HfwYI|Tmcel%~XG0B3hVF(jh7Q&8r4Pjph4L<~k~h+OBc1e>rZLSPU|N zm{)(Px6ET=pe7n^3}-#LWt#?*Fu^MVVaf!tnJ7NUd_jGQ55kdw{3-w43UM8Vkx zKpl;v^*~mS-j~t3n)qMghs**mKMa@sF3edaktJX{10|aZM$cdzEa{eB#dS~vrHHLm zIUNleO2!Qesc#BkP-x|TJ^z9w%r70qprXlTMO$4F!` z_&CYjHfTwiZ<^OrLdBo5OE_1;hhr57DsLCcBXl^_AkZ}E$cM$^^lx!!c-%bG=YTe8 zTNVoOB=yNRD`I^IAIHRO5;8Oh{K4c0Z`Z0%&S_e=&_qavAad+E?+#IO7@PVe9cJO4 zjv}BKN$m}*(WOoPe*t8=N`D!$4ri|o85n>vJQ@SK$S77qP+;%A#bA9<)L9m*upKQ@ zI07neu?eu`EzD>om&ee_#`ylPeIOn(_~zf4G%oN92=w%>Sh-RKjvk+yvubLQun02i xJ2CJazp>8|kkb8o?|=dMEquBEYxZDv&58Bniob15I7VJbJtlvY@$32D{}1*Bk8%J2