IUPD

IUPD Demo Project

Table of Contents

• How to Run the Code
• Repository Structure
• Modules Overview
• TODO List

How to Run the Code

To run the project, please follow the instructions at:

IODA Upstream Delay Container GitHub Repository

Repostory Structure

├── internet_prober
├── data_parser
├── data_uploader
├── source_data
└── docker

Modules Overview

Below is an overview of essential modules of this project:

internet_prober

• Description: This moduler probes target IP prefixes.

data_parser

• Description: This module processes the results generated by the probing tools and prepares them for analysis.
• Sub-Modules:
  • yrp2text: Convert raw Yarrp output to text format by grouping traceroute results by target IP address; we add additional field, by sorting hops based on this counter field, we can reconstruct the orginal order of packets in the raw output.
  • txt2json: Map IP addresses to ASN using prefix2as and map IP to prefix using longest prefix match (may be done at server); we group the results by ASN into JSON

data_uploader

• Description: This module handles the posting of processed data to our storage. It handles multiple cases where posting is not successful.

source_data

• Description: Contains datasets and input files required by the other modules.
• Datasets:
  • pfx2AS Dataset: Maps IP prefixes to Autonomous Systems (AS).
  • Target IP List: Provides the list of target IP addresses for the probing module (1M prefixes).

docker

• Description: Contains docker-related configuration files.
• Configuration files:
  • docker-compose.yaml: Contains defination for Docker Compose.
  • .env: Contains sensitive info - authentication.

TODO List:

1. write to stat table
2. remove cycle data after inserting into ES