Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
250 changes: 250 additions & 0 deletions docs/rlnctest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
# P2P Binary Deployment & Testing Guide

## Step 1 — Push Changes to `optimum-p2p`

Ensure all your changes have been pushed to the `optimum-p2p` repository before proceeding.

---

## Step 2 — Update the Dependency in `optimum-proxy`

In the `optimum-proxy` repository, open `go.mod` and update the `require` section to reference the newly pushed version of `optimum-p2p`:

```go
github.com/getoptimum/optimum-p2p <new-version>
```

Replace `<new-version>` with the version tag you just pushed (e.g. `v0.0.1-rc13`), then run:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe can add hash as example too (e.g. v0.0.1-rc13 or bd8c0b8)


```bash
go get github.com/getoptimum/optimum-p2p@<new-version>
go mod tidy
```

---

## Step 3 — Build the Binary

From the `optimum-proxy` repository root, build the P2P binary:

```bash
make build
```

---

## Step 4 — Deploy the Binary

Move the compiled `p2p-client` binary to the infrastructure repository:

```bash
mv ./p2p-client optimum-infra/optimump2p-native/data/p2pnode-rlnc-arch-test
```

Note: The build also produces `p2p-multi-publish` and `p2p-multi-subscribe` binaries, which are used for testing (see Step 8).

---

## Step 5 — Start Local Nodes

Run 50 local nodes:
Comment on lines +49 to +50

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

navigate to optimum-infra/optimump2p-native/


```bash
./run-local 50

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before run, make sure to update (LOCALHOST_IP: local IP address of the primary network interface) first
the cmd should be ./local-p2pnode-runs.sh run 50

```

---

## Step 6 — Start the RLNC Server

Build and start the RLNC server:

```bash
go run cmd/server/main.go
```

---

## Step 7 — Verify All Nodes Are Running

```bash
./run-local list

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./local-p2pnode-runs.sh list

```

Confirm that all 50 processes appear in the output before proceeding.

---

## Step 8 — Test Publish/Subscribe via gRPC

Use the publish/subscribe tools from the `optimum-dev-local-setup` repository. See below for full usage instructions.

---

# P2P Client Multi-Stream Tools

Tools for publishing and subscribing to P2P network topics via gRPC streams.

## Prerequisites

- Go compiler installed
- A file containing P2P node addresses in `IP:PORT` format, one per line (e.g. `ip.p2pnode.bnb.tsv`)

---

## Building the Binaries

Build both tools before use:

```bash
go build -o p2p-multi-publish p2p_client_multi_streams_publish.go
go build -o p2p-multi-subscribe p2p_client_multi_streams_subscribe.go
Comment on lines +99 to +101

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Find the actual source files for multi-publish and multi-subscribe

# Look for main.go files in cmd directories
fd -t f "main.go" grpc_p2p_client/cmd/

# Check if the referenced .go files exist
fd -t f -e go "p2p_client_multi_streams"

Repository: getoptimum/optimum-dev-setup-guide

Length of output: 198


Update build commands to reference the correct directory paths.

The documented build commands reference non-existent files (p2p_client_multi_streams_publish.go, p2p_client_multi_streams_subscribe.go). The actual source is located in the cmd subdirectories. Correct these to:

go build -o p2p-multi-publish ./grpc_p2p_client/cmd/multi-publish/
go build -o p2p-multi-subscribe ./grpc_p2p_client/cmd/multi-subscribe/

Ensure the documented commands are copy-pastable and match the repository structure.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/rlnctest.md` around lines 97 - 99, The build commands in the docs
reference non-existent source files (p2p_client_multi_streams_publish.go and
p2p_client_multi_streams_subscribe.go); update the documented commands to point
to the actual cmd directories under grpc_p2p_client (use the
./grpc_p2p_client/cmd/multi-publish/ and ./grpc_p2p_client/cmd/multi-subscribe/
paths) so the commands are copy-pastable and match the repository layout.

Comment on lines +99 to +101

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file doesn't exist, just use make build should be fine

```

This produces two executables:

| Binary | Role |
|---|---|
| `p2p-multi-publish` | Publishes messages to a topic across multiple nodes |
| `p2p-multi-subscribe` | Subscribes to a topic and listens for messages across multiple nodes |

---

## Publisher

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should move subscriber on the top before doing any publish


Connects to multiple P2P nodes and publishes messages to a specified topic.

### Usage

```bash
./p2p-multi-publish [options]
```

### Options

| Flag | Type | Default | Description |
|---|---|---|---|
| `-topic` | string | *(required)* | Topic name to publish to |
| `-ipfile` | string | — | Path to file containing node addresses |
| `-count` | int | 1 | Number of messages to publish per node |
| `-datasize` | int | 100 | Size of random message payload in bytes |
| `-start-index` | int | 0 | First index to use from the IP file |
| `-end-index` | int | 10000 | Last index to use from the IP file |
| `-output` | string | — | File to write outgoing message hashes |
| `-sleep` | duration | 50ms | Delay between consecutive publishes (e.g. `1s`, `500ms`) |
| `-poisson` | bool | false | Enable Poisson arrival distribution for publishing |

### Examples

**Basic** — publish 2 messages to the first 5 nodes:

```bash
./p2p-multi-publish \
-count 2 -datasize 100 \
-start-index 0 -end-index 5 \
-ipfile ip.p2pnode.bnb.tsv \
-output outgoing-hash.txt \
-sleep 500ms -topic test-topic
```

**Production** — publish 3 messages to 20 nodes:

```bash
./p2p-multi-publish \
-count 3 -datasize 12222 \
-start-index 0 -end-index 20 \
-ipfile ip.p2pnode.bnb.tsv \
-output outgoing-data-hash.txt \
-sleep 1s -topic mytopic
```

### Output Format

The output file is tab-separated (TSV):

```tsv
sender size sha256(msg)
34.127.26.15:33212 119 03fb89f1791f9bd70fa959a8acbae9dd...
```
Comment on lines +165 to +168

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Use real tab separators in the TSV examples.

These samples are described as TSV, but the rows are space-aligned. That makes the copied examples non-TSV and mismatches the publisher’s actual header format (sender\tsize\tsha256(msg) in grpc_p2p_client/cmd/multi-publish/main.go).

📝 Suggested fix
- sender                  size    sha256(msg)
- 34.127.26.15:33212      119     03fb89f1791f9bd70fa959a8acbae9dd...
+ sender\tsize\tsha256(msg)
+ 34.127.26.15:33212\t119\t03fb89f1791f9bd70fa959a8acbae9dd...

Also applies to: 219-222

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/rlnctest.md` around lines 165 - 168, The TSV examples in the
documentation are space-aligned instead of using real tab separators, which
makes them inconsistent with the actual TSV header format used by
grpc_p2p_client/cmd/multi-publish/main.go. Update the sample rows in the docs so
the values are separated by literal tab characters, and make the same correction
in the other referenced TSV block as well; keep the examples aligned with the
sender, size, and sha256(msg) fields.

Source: Path instructions


---

## Subscriber

Connects to multiple P2P nodes and listens for messages on a specified topic. Runs indefinitely until interrupted with `Ctrl+C`.

### Usage

```bash
./p2p-multi-subscribe [options]
```

### Options

| Flag | Type | Default | Description |
|---|---|---|---|
| `-topic` | string | *(required)* | Topic name to subscribe to |
| `-ipfile` | string | — | Path to file containing node addresses |
| `-start-index` | int | 0 | First index to use from the IP file |
| `-end-index` | int | 10000 | Last index to use from the IP file |
| `-output-data` | string | — | File to write received message hashes |
| `-output-trace` | string | — | File to write trace information |

### Examples

**Basic** — subscribe on first 5 nodes:

```bash
./p2p-multi-subscribe \
-end-index 5 \
-ipfile ip.p2pnode.bnb.tsv \
-output-data incoming-hash.txt \
-topic test-topic
```

**Production** — subscribe on 20 nodes:

```bash
./p2p-multi-subscribe \
-end-index 20 \
-ipfile ip.p2pnode.bnb.tsv \
-output-data incoming-data-hash.txt \
-topic mytopic
```

### Output Format

The output file is tab-separated (TSV):

```tsv
receiver sender size sha256(msg)
34.187.253.56:33212 [binary data] 12453 1b4bca4ab3703d64...
```

---

## End-to-End Test

**Terminal 1 — start the subscriber:**

```bash
./p2p-multi-subscribe \
Comment on lines +228 to +231

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should give them an example to achieve 2500

./p2p-multi-subscribe
-ipfile ip.p2pnode.bnb.tsv
-output-data incoming-hash.txt
-topic test-topic

-end-index 5 \
-ipfile ip.p2pnode.bnb.tsv \
-output-data test-incoming.txt \
-topic test-topic
```

**Terminal 2 — run the publisher:**

```bash
./p2p-multi-publish \
-count 2 -datasize 100 \
-end-index 5 \
-ipfile ip.p2pnode.bnb.tsv \
-output test-outgoing.txt \
-sleep 500ms -topic test-topic
```

Verify that message hashes in `test-outgoing.txt` appear in `test-incoming.txt`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a "validation result"?

RLNC Validation Results

The test achieved a total of 25,000 decoded messages, validating the full propagation of data across the network.

The Math:

  • 50 Nodes × 10 Unique Messages/Node = 500 Distinct Messages in the system.
  • 500 Messages × 50 Receiving Nodes = 25,000 Total Successful Deliveries.

The "RLNC Impact" Proof:

"The significance of RLNC is proven by the delta: without RLNC encoding/decoding, the total deliveries would drop from 25,000 to only 500 (representing only the locally generated messages that didn't require network transmission). This 50x increase in decoded volume confirms that our RLNC implementation is successfully driving the P2P message exchange.


Loading
Loading