Skip to content

[benchmark] Report from <Western Europe> #368

Description

@EhudKirsh

Sanitized JSON report

PS C:\Users\ehudk> aero benchmark --user Benchmarks --tui

=== GoogleDrive ===
Using profile: GoogleDrive (GOOGLEDRIVE via OAuth [googledrive])
running benchmark with payload 10.0 MB
trash purge: aeroftp-bench/1ee8d0eb-d770-4311-b910-33aeccb1352c hard-deleted

=== pCloudDrive OAuth ===
running benchmark with payload 10.0 MB

=== pCloudDrive WebDAV ===
warning: could not create benchmark base dir: Permission denied: aeroftp-bench
Error: benchmark cannot create scratch dir 'aeroftp-bench/b2a59793-f40c-47d0-9165-70de825019d8': Invalid path: Parent directory does not exist. Provider may not allow folder creation in the configured root.

=== Filen API ===
running benchmark with payload 10.0 MB

=== Filen WebDAV ===
Error: Connection failed: Authentication failed: Invalid credentials (check credentials in --profile or URL)

=== Filen S3 ===
Error: Connection failed: Authentication failed: S3 auth error: The signature
does not match (check credentials in --profile or URL)

=== MEGA API ===
running benchmark with payload 10.0 MB
trash purge: aeroftp-bench/936ecd75-23df-4334-9f18-f4653550968f hard-deleted

=== MEGAcmd API ===
MEGAcmd: starting the local server in the background if needed (first connect
can take a few seconds)...
running benchmark with payload 10.0 MB

=== MEGAcmd WebDAV ===
running benchmark with payload 10.0 MB

=== kDrive ===
Note: path 'aeroftp-bench' resolved to '/Private/aeroftp-bench' (profile base: /Private)
running benchmark with payload 10.0 MB
trash purge: /Private/aeroftp-bench/bb2d9108-5d26-4410-8c39-2a1cce00f9c5 hard-deleted

=== S3Drive ===
running benchmark with payload 10.0 MB

=== Koofr API ===
running benchmark with payload 10.0 MB

=== Koofr WebDAV ===
running benchmark with payload 10.0 MB

=== box ===
running benchmark with payload 10.0 MB
trash purge: aeroftp-bench/0b3631ac-338a-4371-99aa-49363edde7a9 hard-deleted

=== OpenDrive API ===
Error: Connection failed: Authentication failed: <html><head><title>OpenDrive
WebDAV server require authentication</title></head> (check credentials in --profile or URL)

=== OpenDrive WebDAV ===
running benchmark with payload 10.0 MB

=== Jottacloud ===
running benchmark with payload 10.0 MB

=== OneDrive ===
running benchmark with payload 10.0 MB

=== YandexDisk WebDAV ===
running benchmark with payload 10.0 MB

=== Dropbox ===
running benchmark with payload 10.0 MB

Profile: GoogleDrive
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=6044ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  3155.6ms  3155.6ms  26.58 / 26.58
  download   10.0 MB     1  1287.4ms  1287.4ms  65.16 / 65.16

Profile: pCloudDrive OAuth
Benchmark complete: level=Quick runs=0 bytes=0 B duration=6665ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs    p50    p95  Mbps
  ────────────────────────────────────────────
  upload     10.0 MB     0  0.0ms  0.0ms     -

Errors / warnings (2):
  - upload 10.0 MB bytes failed: Transfer failed: pCloud upload_write part 2 returned result 2068: Error writing to upload.
  - trash purge of aeroftp-bench/ca3add95-8200-435c-830c-3a2ee4caee2b failed (item still in trash, will be auto-deleted by provider retention): Authentication failed: Log in required.

Profile: Filen API
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=9142ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  4809.4ms  4809.4ms  17.44 / 17.44
  download   10.0 MB     1  3705.0ms  3705.0ms  22.64 / 22.64

Profile: MEGA API
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=6716ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  3527.9ms  3527.9ms  23.78 / 23.78
  download   10.0 MB     1  1622.5ms  1622.5ms  51.70 / 51.70

Profile: MEGAcmd API
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=2744ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95             Mbps
  ─────────────────────────────────────────────────────────────
  upload     10.0 MB     1  1653.4ms  1653.4ms    50.74 / 50.74
  download   10.0 MB     1   763.7ms   763.7ms  109.84 / 109.84

Profile: MEGAcmd WebDAV
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=1647ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95             Mbps
  ─────────────────────────────────────────────────────────────
  upload     10.0 MB     1  1247.5ms  1247.5ms    67.24 / 67.24
  download   10.0 MB     1   298.0ms   298.0ms  281.48 / 281.48

Profile: kDrive
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=3929ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  1781.1ms  1781.1ms  47.10 / 47.10
  download   10.0 MB     1  1338.2ms  1338.2ms  62.69 / 62.69

Profile: S3Drive
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=2070ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95             Mbps
  ─────────────────────────────────────────────────────────────
  upload     10.0 MB     1  1491.2ms  1491.2ms    56.25 / 56.25
  download   10.0 MB     1   365.3ms   365.3ms  229.65 / 229.65

Profile: Koofr API
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=4419ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  2931.5ms  2931.5ms  28.61 / 28.61
  download   10.0 MB     1  1360.4ms  1360.4ms  61.66 / 61.66

Profile: Koofr WebDAV
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=4094ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  2170.9ms  2170.9ms  38.64 / 38.64
  download   10.0 MB     1  1656.4ms  1656.4ms  50.64 / 50.64

Profile: box
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=7597ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  3560.3ms  3560.3ms  23.56 / 23.56
  download   10.0 MB     1  2270.0ms  2270.0ms  36.95 / 36.95

Profile: OpenDrive WebDAV
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=157493ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs         p50         p95         Mbps
  ─────────────────────────────────────────────────────────────
  upload     10.0 MB     1  104821.7ms  104821.7ms  0.80 / 0.80
  download   10.0 MB     1   52313.8ms   52313.8ms  1.60 / 1.60

Profile: Jottacloud
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=3592ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  1635.9ms  1635.9ms  51.28 / 51.28
  download   10.0 MB     1  1766.3ms  1766.3ms  47.49 / 47.49

Profile: OneDrive
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=4242ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95             Mbps
  ─────────────────────────────────────────────────────────────
  upload     10.0 MB     1  2495.2ms  2495.2ms    33.62 / 33.62
  download   10.0 MB     1   791.0ms   791.0ms  106.05 / 106.05

Profile: YandexDisk WebDAV
Benchmark complete: level=Quick runs=0 bytes=0 B duration=602732ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs    p50    p95  Mbps
  ────────────────────────────────────────────
  upload     10.0 MB     0  0.0ms  0.0ms     -

Errors / warnings (1):
  - upload 10.0 MB bytes failed: Network error: error sending request for url
(https://webdav.yandex.ru/aeroftp-bench/ce079ec7-70c0-4330-b29f-60f871fd082b/payload-10485760.bin)

Profile: Dropbox
Benchmark complete: level=Quick runs=2 bytes=20.0 MB duration=3597ms

Single-file throughput (Mbps p50/p95, higher is better)
  operation     size  runs       p50       p95           Mbps
  ───────────────────────────────────────────────────────────
  upload     10.0 MB     1  1792.9ms  1792.9ms  46.79 / 46.79
  download   10.0 MB     1   957.9ms   957.9ms  87.57 / 87.57

Errors / warnings (1):
  - trash purge of aeroftp-bench/6a699e80-9152-4c09-8c3e-100d9ce7950f failed (item still in trash, will be auto-deleted by provider retention): Permanent delete failed 400 Bad Request: Error in call to API function "files/permanently_delete": Your app (ID: 6832595) is not permitted to access this endpoint because it does not have the required scope 'files.permanent_delete'. The owner...

Comparison: single-file throughput (Mbps p50, higher is better)
  profile            upload Mbps  download Mbps
  ─────────────────────────────────────────────
  GoogleDrive              26.58          65.16
  pCloudDrive OAuth            -              -
  Filen API                17.44          22.64
  MEGA API                 23.78          51.70
  MEGAcmd API              50.74         109.84
  MEGAcmd WebDAV           67.24         281.48
  kDrive                   47.10          62.69
  S3Drive                  56.25         229.65
  Koofr API                28.61          61.66
  Koofr WebDAV             38.64          50.64
  box                      23.56          36.95
  OpenDrive WebDAV          0.80           1.60
  Jottacloud               51.28          47.49
  OneDrive                 33.62         106.05
  YandexDisk WebDAV            -              -
  Dropbox                  46.79          87.57

Skipped (failed to benchmark): pCloudDrive WebDAV, Filen WebDAV, Filen S3, OpenDrive API

Country / region (coarse)

Western Europe

Connection type

Cable / DSL (10-100 Mbps)

Anything else worth noting

I ran another benchmark. This time with aero benchmark --tui, so there's no JSON, so I just copied the terminal output.
Here's what I noticed:

My first run ran into a problem. Yandex Disk took forever, like an infinite loop. So I had to stop the benchmark.
The reason is that I connected with my VPN to a country that I think sanctions Russia. I'm not sure. I noticed that when my VPN connects to some countries, Yandex Disk isn't reachable. This gets unblocked the moment I switch to certain countries, but the issue is that it means that profiles that were tested after I change a VPN location were tested with a different ping, so that whole benchmark was discarded.

The benchmark results I attached are of another full attempt. My feedback here is that:

  1. AeroFTP should check that the IP address at the beginning of the benchmark is the same as at the end to make sure that it's fair, without any VPN IP changes.

  2. Just like with AeroVault, there should be progress bars, so as not to have to blindly wait without knowing roughly how long it should take.

  3. Maybe there should be some note specific to Yandex Disk somewhere so users realise why the connection is failing in general, and certainly in the CLI benchmark in particular.
    I managed to guess why, because I encountered it before, but other users may not realise it if AeroFTP doesn't leave an indication.

  4. FYI, I manually had to delete the empty "aeroftp-bench" folder from every cloud drive, so it seems that AeroFTP doesn't clean up after itself.

  5. The results table of the benchmark misses an important column: the protocol (i.e. OAuth, API, WebDAV, S3, etc).
    So, for this benchmark, I renamed the profiles appropriately after the first run for the second run, to include the protocol names in the profile names for profiles that support multiple protocols.

  6. We wondered whether the differences in connection speeds of MEGAcmd WebDAV > MEGAcmd API > MEGA API possibly also affect syncing performance, and it seems that it does, and significantly so. It is interesting to see how well a local bridge/server performs.
    You'd think that the extra layer and the legacy "chatty" protocol WebDAV would theoretically slow the profile down, but it seems that there are far more important factors to consider here in terms of performance, even if they aren't as clearly seen or as intuitive. So I'm glad that I pushed for support for WebDAV MEGAcmd in AeroFTP, as it seems currently like the best protocol for MEGA, other than S4. Again, MEGA is tier 2 in Rclone, and I'm not sure if this relates.
    The fact that AeroFTP connects to MEGAcmd API is unique, as Rclone doesn't.

  7. It's not "all lost" for cloud drives that performed poorly. They can still be used for long-term storage.
    Maybe scheduled with AeroSync for once every few hours or once per day/boot.
    For syncing notes that update constantly every few minutes, it's best to use the fast cloud drives.

  8. As happened in my last benchmark, the Filen local servers were up. I checked before and after the benchmarks.
    And yet, they still failed to perform at all at the benchmarks. Same with pCloudDrive WebDAV and OpenDrive API.
    Totally reachable, I checked before and after. What a shame, I wanted to test a premium protocol and another S3.

  9. The 'Connection type' dropdown list doesn't specify if the unit of Mbps refers to upload or download, so please specify.
    I tested in https://librespeed.org that with Ethernet, which I used during this benchmark, I have 640Mbps download, 100Mbps upload and a ping of 19ms.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions