Trinarkular's probe list comes from the ISI's Internet Address History data set. We are currently getting this list over the IMPACT program, which makes it a bit of a pain to update the list periodically.
An alternative would be to create our own list by drawing on the data that's published by the Censys project. Each week they are running an ICMP ZMap scan over the entire IPv4 address space:
https://censys.io/data/0-icmp-echo_request-full_ipv4
The full scan is a CSV file of about 140 GB but they also publish diffs that would allow us to easily determine what changed throughout the week.
The issue with ZMap data is that it doesn't directly tell us the response rate of a given address. Instead, we'd have to determine it ourselves by taking into account past ZMap scans. In the long term we may want to:
- Explore if taking the ZMap data is a viable alternative
- See to what degree we can make the ZMap data agree with the Address History data
- Write some code that maps ZMap data to the format that Trinarkular currently expects
There's some more context in an email thread: <1522962663.2699.98.camel@malibu>
Trinarkular's probe list comes from the ISI's Internet Address History data set. We are currently getting this list over the IMPACT program, which makes it a bit of a pain to update the list periodically.
An alternative would be to create our own list by drawing on the data that's published by the Censys project. Each week they are running an ICMP ZMap scan over the entire IPv4 address space:
https://censys.io/data/0-icmp-echo_request-full_ipv4
The full scan is a CSV file of about 140 GB but they also publish diffs that would allow us to easily determine what changed throughout the week.
The issue with ZMap data is that it doesn't directly tell us the response rate of a given address. Instead, we'd have to determine it ourselves by taking into account past ZMap scans. In the long term we may want to:
There's some more context in an email thread:
<1522962663.2699.98.camel@malibu>