Excess Network Usage

[TRM13]

Kubuntu 20.04, core is qortal-1.4.5-b9a0d48 (minting mode) and using vnstat 2.6 measuring network usage.

I am logging 300 MB per hour (200+ GB) per month. That seems quite excessive.
Is that normal?
Do we have any baseline to go by?

Thanks (MeadMan)

Hey, thanks for raising this. I imagine it's a symptom of the constant syncing / re-orgs that are going on right now, causing far more data to move between nodes than necessary. I'm currently working on a new approach to syncing which I hope will fix a lot of these issues.

Please can we revisit this in a couple of weeks when some new updates are out, and check that it has improved?

[TRM13]

Sure I'd love to. Whenever you want.
Thanks

[TRM13]

Just a quick update. I've also noticed daily transmit of 2-4 GB in one hour, twice daily for several days. The times are 08:00 and 18:00 to start. Anything in the core set to go off twice a day? I have nothing else running on this system, it is dedicated to Qortal and no system side stuff is happening at those times.

No reply required. Just putting it in here so if someone notices they don't open another ticket as I think it is related.

The huge bursts of transmit traffic seemed to have stopped. I re-sync'd yesterday and I'm still looking at 250-300 GB a month but no bursts (yet lol).

Hey @TRM13, just wondering if you have any recent data on network usage since 1.5.2? Would be interesting to know if it has reduced at all, now that the re-orgs are much smaller.

[TRM13]

I've switched to ntopng as it gives a finer resolution of traffic. I'll run a 12 hour & 24 hour log and get back to you tomorrow.

Thanks - much appreciated! I'm hoping it will be a lot less then the 300MB per hour we used to use...

[TRM13]

Ran ntopng to gather stats for 24 hours:
Sent: 4.4 GB
Recd: 3.2 GB
Which works out to 228 GB/Month.

I've verified with the data my ISP provides and I went from about 130 GB a month to 330 GB after I started my Qortal node. So it seems to be 200+ GB per month is about correct. This may just be inherent in the design. Low CPU but high bandwidth requirements.

Thanks for checking it. You're right that high bandwidth is inherent in the design (every block has a large number of candidates, so nodes must sync several "versions" of each block before finding the best one). In the long run I do expect bandwidth to decrease from where we are now, as we continue to tighten up the sync process, but clearly we're not there yet!

[IceBurst]

Block syncing has been greatly reduced by limiting the Minters to those in a group VS all people with a key.

[TRM13]

When did the block syncing get implemented? The bandwidth usage on my node is still 10+ gigabytes a day without QDN (a lot more with it). I have been monitoring this for a while as you can tell from the initial date.

[IceBurst]

Changing from an unlimited number of minters to a limited number based on the Mintership Group reduced the number of potential block candidates. This was implemented in late 2024. Can you post some information to include a graph over time if available. Thanks,

[TRM13]

I have used vnstat for over two years and it shows the same 10+ GiB a day for the whole time.

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       2025-06    172.24 GiB |  169.90 GiB |  342.14 GiB |    1.13 Mbit/s
       2025-07     87.87 GiB |   83.47 GiB |  171.34 GiB |    1.17 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated    187.39 GiB |  178.00 GiB |  365.39 GiB |

   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     2025-07-14     5.19 GiB |    5.28 GiB |   10.47 GiB |    1.04 Mbit/s
     2025-07-15     3.13 GiB |    3.23 GiB |    6.35 GiB |    1.18 Mbit/s
     ------------------------+-------------+-------------+---------------
     estimated      5.83 GiB |    6.02 GiB |   11.85 GiB |