feat: capture cloud region and service, emit structured handoff records - #150
Merged
Conversation
Matching an address to a provider was only half an answer. "It is an AWS IP"
is close to no information: of roughly 10,500 published AWS prefixes, over
half carry the generic AMAZON tag, and the ones that matter operationally —
EC2 in a named region — were indistinguishable from CDN edge addresses.
All three providers publish region and service alongside each prefix, and all
of it was discarded at parse time. AWS gives region and service, GCP gives
scope and service, Azure gives region and systemService on the tag. The
fetchers now return that metadata, CSPIPAddresses carries it, and each matched
address is written as its own record:
domain|ip|provider|region|service|prefix
The CSP output is consumed by other tooling rather than read as prose, so it
is now structured fields rather than a sentence, one line per address rather
than a list per domain. Addresses under one provider can sit in different
regions, and a consumer acts per address.
The run summary groups matches by provider, region and service for the same
reason. A live run against AWS-fronted hosts reports CLOUDFRONT and
GLOBALACCELERATOR, which tells an operator these are CDN edges rather than
EC2 addresses in a region — a distinction the previous output could not make.
Fields are taken verbatim from the provider. Where none is published they read
"unknown" rather than being inferred, and no judgement is encoded about what
is worth pursuing; that is the operator's call.
Tests changed expectation where they asserted the old prose format, and new
tests cover metadata capture for each provider, the unknown fallback, Azure's
empty region on global tags, and the end-to-end handoff record.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Matching a resolved address to a provider was only half an answer. "It is an AWS IP" is close to no information: of roughly 10,500 published AWS prefixes, over half carry the generic
AMAZONtag, and the ones that matter operationally —EC2in a named region — were indistinguishable from CDN edge addresses.All three providers publish region and service alongside every prefix. All of it was being discarded at parse time.
ip_prefix,region,service,network_border_groupipv4Prefix/ipv6Prefix,service,scopeaddressPrefixes,region,systemServiceChange
Fetchers now return
(ipv4, ipv6, metadata), where metadata maps CIDR to(region, service).CSPIPAddressescarries it and exposesdescribe(cidr). The matcher retains which prefix an address matched, since that is the key to the metadata.Each matched address is written as its own record:
This file is consumed by other tooling rather than read as prose, so it is now structured fields instead of a sentence, and one line per address instead of a list per domain — addresses under one provider can sit in different regions, and a consumer acts per address.
The run summary groups by provider, region and service for the same reason.
Why it matters, from a live run
Previously every one of these read as "resolved to aws IPs". The output now shows they are CDN and Global Accelerator edges rather than EC2 addresses in a region — a distinction that decides whether an address is worth pursuing at all, and one the old format could not express.
Fields are taken verbatim from the provider. Where none is published they read
unknownrather than being inferred, and no judgement about what is claimable is encoded — that is the operator1s call.Compatibility
csp_matches_*.txtchanges format. It is a breaking change for anything parsing the old sentence form, documented in the README under Cloud IP attribution and inAGENTS.md. The other output files are untouched.Legacy Azure caches written before this change hold only the two range lists; they are still read, with metadata simply absent, rather than being discarded.
Tests
Tests that asserted the old prose format now assert the new contract rather than being relaxed. Added: metadata capture per provider, GCP
scopemapping to region, theunknownfallback when a provider publishes nothing, Azure global tags reportingglobalrather than unknown, the end-to-end handoff record, one-record-per-address across differing regions, and the summary breakdown ordering.251 passed, coverage 95%, ruff clean.