Recover alpha-2 country codes for Australia and Abkhazia

README.md

@@ -41,7 +41,7 @@ The service automatically synchronizes cell tower data from [OpenCellID](https:/
 | ------------- | ----------------------------------------------------------------------------- | ---------- |
 | `operator`    | `mcc` + `net`                                                                 | `Vodafone` |
 | `country`     | `mcc` + `net`, falling back to the country most of that MCC's networks are in | `Germany`  |
-| `countryCode` | same as `country`, ISO 3166-1 alpha-2                                         | `DE`       |
+| `countryCode` | same as `country`, two-letter ISO 3166-1 alpha-2                              | `DE`       |
 
 Unknown values are `null` — never an empty string and never a guess. The three degrade
 independently:
@@ -53,10 +53,15 @@ independently:
 
 Clients MUST fall back to showing the raw numeric identifiers for whichever fields are `null`.
 
-`country` and `countryCode` degrade independently: 107 rows carry a country with no code, because
-the upstream code is a multi-territory list (`AU/CX/CC/NF`) or a subdivision (`GE-AB`) rather than
-an alpha-2. Australia is the largest affected country. Do not treat a present `country` as a
-guarantee that `countryCode` is present.
+`country` and `countryCode` degrade independently: 35 rows carry a country with no code, because
+upstream lists a multi-territory grouping that has no alpha-2 of its own — `BQ/CW/SX` (Former
+Netherlands Antilles), `BL/GF/GP/MF/MQ` (French Antilles), `YT/RE` (French Departments and
+Territories in the Indian Ocean). Do not treat a present `country` as a guarantee that
+`countryCode` is present.
+
+One code is not officially assigned: MCC 221 (Kosovo) reports `XK`, the user-assigned code the EU,
+IMF, SWIFT and CLDR all use because ISO has assigned Kosovo none. It is passed through as upstream
+gives it — a code that resolves everywhere in practice beats a `null`.
 
 The table is compiled into the binary from `src/utils/mcc-mnc.csv`, which is **generated — do not
 hand-edit**. It derives from the MIT-licensed, Wikipedia-sourced
@@ -71,6 +76,11 @@ Where one MNC is registered across several territories (Airtel-Vodafone in Guern
 the UK; Docomo in Guam, the Northern Marianas and the USA) no MNC can disambiguate them, so the
 table reports the umbrella country rather than guessing a territory.
 
+The two fields may therefore name different granularities. MCC 505 reports `Australia` / `AU` even
+though upstream codes it `AU/CC/CX`, because the umbrella country has its own alpha-2. MCC 289
+reports `Abkhazia` / `GE`: upstream codes it `GE-AB`, a subdivision, so the code names the sovereign
+state while the country names the territory.
+
 ## Requirements
 
 - Rust 1.98.0 (pinned in `rust-toolchain.toml`; requires rustup)

scripts/update-mcc-mnc.py

@@ -20,6 +20,13 @@
 # Better-attested rows win a duplicate (mcc, mnc) key; everything else ranks equal.
 STATUS_RANK = {"operational": 0, "temporary operational": 1}
 
+# Countries whose upstream code is not alpha-2 but is still recoverable: Australia is a territory
+# list ("AU/CC/CX") naming the umbrella, which has its own code; Abkhazia is a subdivision
+# ("GE-AB") whose sovereign parent does. Keyed by name — upstream reorders territory lists more
+# readily than it renames countries. Every entry is verified by hand: no rule infers a code from
+# the string, so a new grouping drops to null and shows up in this script's dropped-codes report.
+COUNTRY_CODES = {"Australia": "AU", "Abkhazia": "GE"}
+
 
 def clean_country(name):
     """"Guam (United States of America)" -> "Guam"."""
@@ -36,24 +43,27 @@ def normalize(record):
         mnc = int(record["mnc"])
     except (TypeError, ValueError):
         return None  # a few MNC values are ranges or notes, not codes
-    code = record.get("countryCode") or ""
+    source_code = record.get("countryCode") or ""
+    country = clean_country(record.get("countryName"))
+    code = source_code if len(source_code) == 2 else COUNTRY_CODES.get(country, "")
     return {
         "mcc": int(record["mcc"]),
         "mnc": mnc,
         "operator": record.get("brand") or record.get("operator") or "",
-        "country": clean_country(record.get("countryName")),
-        # Multi-territory codes ("BQ/CW/SX") and subdivisions ("GE-AB") are not alpha-2.
-        "country_code": code if len(code) == 2 else "",
+        "country": country,
+        "country_code": code,
         "rank": STATUS_RANK.get((record.get("status") or "").strip().lower(), 9),
+        "source_code": source_code,  # reported at the end of a run; DictWriter ignores it
     }
 
 
 def main():
     """Fetch the upstream list and write the lookup table Rust compiles in.
 
     Every ambiguity is resolved here rather than at runtime: duplicate keys collapse by
-    status then by the MCC's dominant country, non-alpha-2 country codes drop, and each MCC
-    gets an mnc-less fallback row. Output is sorted so the committed diff stays reviewable.
+    status then by the MCC's dominant country, non-alpha-2 country codes resolve or drop, and
+    each MCC gets an mnc-less fallback row. Output is sorted so the committed diff stays
+    reviewable.
     """
     parser = argparse.ArgumentParser()
     parser.add_argument("--source", default=SOURCE)
@@ -103,6 +113,14 @@ def main():
 
     print(f"{args.out}: {len(out)} rows ({len(main_country)} MCC fallbacks)")
 
+    # Silent drops are how the missing Australian codes went unnoticed. Report, do not raise:
+    # upstream legitimately carries groupings with no alpha-2, so a hard failure would be muted.
+    unresolved = collections.Counter(
+        row["source_code"] for row in rows if not row["country_code"] and row["source_code"]
+    )
+    for code, count in unresolved.most_common():
+        print(f"  dropped non-alpha-2 country code {code!r}: {count} rows")
+
 
 if __name__ == "__main__":
     main()

src/utils/carrier.rs

@@ -92,6 +92,22 @@ mod tests {
         assert_eq!(lookup(310, 260).operator.as_deref(), Some("T-Mobile"));
     }
 
+    #[test]
+    fn test_non_alpha2_source_codes_resolve() {
+        assert_eq!(lookup(505, 1).country_code.as_deref(), Some("AU")); // source "AU/CC/CX"
+        assert_eq!(lookup(289, 67).country_code.as_deref(), Some("GE")); // source "GE-AB"
+    }
+
+    #[test]
+    fn test_unresolvable_code_keeps_country_and_nulls_the_code() {
+        // Source code "BL/GF/GP/MF/MQ" is a grouping with no alpha-2 of its own.
+        let carrier = lookup(340, 1);
+
+        assert_eq!(carrier.operator.as_deref(), Some("Orange"));
+        assert_eq!(carrier.country.as_deref(), Some("French Antilles"));
+        assert_eq!(carrier.country_code, None);
+    }
+
     #[test]
     fn test_multi_country_mcc_uses_row_country() {
         let carrier = lookup(310, 120);

src/utils/mcc-mnc.csv

@@ -1114,9 +1114,9 @@ mcc,mnc,operator,country,country_code
 288,2,Nema,Faroe Islands,FO
 288,3,TOSA,Faroe Islands,FO
 288,10,Føroya Tele,Faroe Islands,FO
-289,,,Abkhazia,
-289,67,Aquafon,Abkhazia,
-289,88,A-Mobile,Abkhazia,
+289,,,Abkhazia,GE
+289,67,Aquafon,Abkhazia,GE
+289,88,A-Mobile,Abkhazia,GE
 290,,,Greenland,GL
 290,1,tusass,Greenland,GL
 290,2,Nanoq Media,Greenland,GL
@@ -2713,75 +2713,75 @@ mcc,mnc,operator,country,country_code
 502,155,Clixster,Malaysia,MY
 502,156,Altel,Malaysia,MY
 502,157,Telin,Malaysia,MY
-505,,,Australia,
-505,1,Telstra,Australia,
-505,2,Optus,Australia,
-505,3,Vodafone,Australia,
-505,4,Department of Defence,Australia,
-505,5,Ozitel,Australia,
-505,6,3,Australia,
-505,7,TPG Telecom,Australia,
-505,8,One.Tel,Australia,
-505,9,Airnet,Australia,
-505,10,Norfolk Island,Australia,
-505,11,Telstra,Australia,
-505,12,3,Australia,
-505,13,RailCorp,Australia,
-505,14,AAPT,Australia,
-505,15,3GIS,Australia,
-505,16,VicTrack,Australia,
-505,17,Optus,Australia,
-505,18,Pactel,Australia,
-505,19,Lycamobile,Australia,
-505,20,Ausgrid Corporation,Australia,
-505,21,Queensland Rail,Australia,
-505,22,iiNet,Australia,
-505,23,Vocus,Australia,
-505,24,Advanced Communications Technologies,Australia,
-505,25,Pilbara Iron,Australia,
-505,26,Sinch Australia,Australia,
-505,27,Ergon Energy Telecommunications,Australia,
-505,28,RCOM International,Australia,
-505,30,Compatel,Australia,
-505,31,BHP,Australia,
-505,32,Thales Australia,Australia,
-505,33,Sinch Australia,Australia,
-505,34,Santos,Australia,
-505,35,Bird.com,Australia,
-505,36,Optus,Australia,
-505,37,Yancoal,Australia,
-505,38,Truphone,Australia,
-505,39,Telstra,Australia,
-505,40,CITIC Pacific Mining,Australia,
-505,41,Aqura Technologies,Australia,
-505,42,GEMCO,Australia,
-505,43,Arrow Energy,Australia,
-505,44,Roy Hill,Australia,
-505,45,Clermont Coal Operations,Australia,
-505,46,AngloGold Ashanti Australia Ltd,Australia,
-505,47,Woodside Energy,Australia,
-505,48,Titan ICT,Australia,
-505,49,Field Solutions Group,Australia,
-505,50,Pivotel Group,Australia,
-505,51,Fortescue,Australia,
-505,52,OptiTel Australia,Australia,
-505,53,Shell Australia,Australia,
-505,54,Nokia,Australia,
-505,55,New South Wales Government Telecommunications Authority,Australia,
-505,56,Nokia,Australia,
-505,57,CiFi,Australia,
-505,58,Wi-Sky,Australia,
-505,59,Starlink,Australia,
-505,60,Starlink,Australia,
-505,61,CommTel NS,Australia,
-505,62,NBN,Australia,
-505,63,MarchNet,Australia,
-505,68,NBN,Australia,
-505,71,Telstra,Australia,
-505,72,Telstra,Australia,
-505,88,Pivotel Group,Australia,
-505,90,Alphawest,Australia,
-505,99,Telstra,Australia,
+505,,,Australia,AU
+505,1,Telstra,Australia,AU
+505,2,Optus,Australia,AU
+505,3,Vodafone,Australia,AU
+505,4,Department of Defence,Australia,AU
+505,5,Ozitel,Australia,AU
+505,6,3,Australia,AU
+505,7,TPG Telecom,Australia,AU
+505,8,One.Tel,Australia,AU
+505,9,Airnet,Australia,AU
+505,10,Norfolk Island,Australia,AU
+505,11,Telstra,Australia,AU
+505,12,3,Australia,AU
+505,13,RailCorp,Australia,AU
+505,14,AAPT,Australia,AU
+505,15,3GIS,Australia,AU
+505,16,VicTrack,Australia,AU
+505,17,Optus,Australia,AU
+505,18,Pactel,Australia,AU
+505,19,Lycamobile,Australia,AU
+505,20,Ausgrid Corporation,Australia,AU
+505,21,Queensland Rail,Australia,AU
+505,22,iiNet,Australia,AU
+505,23,Vocus,Australia,AU
+505,24,Advanced Communications Technologies,Australia,AU
+505,25,Pilbara Iron,Australia,AU
+505,26,Sinch Australia,Australia,AU
+505,27,Ergon Energy Telecommunications,Australia,AU
+505,28,RCOM International,Australia,AU
+505,30,Compatel,Australia,AU
+505,31,BHP,Australia,AU
+505,32,Thales Australia,Australia,AU
+505,33,Sinch Australia,Australia,AU
+505,34,Santos,Australia,AU
+505,35,Bird.com,Australia,AU
+505,36,Optus,Australia,AU
+505,37,Yancoal,Australia,AU
+505,38,Truphone,Australia,AU
+505,39,Telstra,Australia,AU
+505,40,CITIC Pacific Mining,Australia,AU
+505,41,Aqura Technologies,Australia,AU
+505,42,GEMCO,Australia,AU
+505,43,Arrow Energy,Australia,AU
+505,44,Roy Hill,Australia,AU
+505,45,Clermont Coal Operations,Australia,AU
+505,46,AngloGold Ashanti Australia Ltd,Australia,AU
+505,47,Woodside Energy,Australia,AU
+505,48,Titan ICT,Australia,AU
+505,49,Field Solutions Group,Australia,AU
+505,50,Pivotel Group,Australia,AU
+505,51,Fortescue,Australia,AU
+505,52,OptiTel Australia,Australia,AU
+505,53,Shell Australia,Australia,AU
+505,54,Nokia,Australia,AU
+505,55,New South Wales Government Telecommunications Authority,Australia,AU
+505,56,Nokia,Australia,AU
+505,57,CiFi,Australia,AU
+505,58,Wi-Sky,Australia,AU
+505,59,Starlink,Australia,AU
+505,60,Starlink,Australia,AU
+505,61,CommTel NS,Australia,AU
+505,62,NBN,Australia,AU
+505,63,MarchNet,Australia,AU
+505,68,NBN,Australia,AU
+505,71,Telstra,Australia,AU
+505,72,Telstra,Australia,AU
+505,88,Pivotel Group,Australia,AU
+505,90,Alphawest,Australia,AU
+505,99,Telstra,Australia,AU
 510,,,Indonesia,ID
 510,0,PSN,Indonesia,ID
 510,1,Indosat,Indonesia,ID