Skip to content

README.md

Latest commit

 

History

History
207 lines (154 loc) · 6.9 KB

README.md

File metadata and controls

207 lines (154 loc) · 6.9 KB
Raw

eclone

An enhanced fork of rclone with advanced Google Drive Service Account (SA) rotation.

Based on gclone with additional features ported from fclone, including SA preloading, 25-hour blacklisting, and anti-thrashing protection.

Current version: v1.73.0-mod2.0.2 (rclone v1.73.0 base)

Features

All standard rclone features, plus:

Feature Origin Description
Dynamic SA rotation gclone Automatically switches SA files on rate-limit errors
Rolling SA gclone Proactive SA rotation before each operation for balanced usage
Folder ID support gclone Use {folder_id} syntax for direct access to shared drives
SA preloading fclone Pre-creates OAuth services at startup to eliminate 200-500ms switch latency
25h blacklist fclone Temporarily blacklists rate-limited SAs aligned with Google's daily quota reset
Anti-thrashing fclone Configurable minimum sleep between SA changes
Pacer reset on SA change fclone Fresh backoff avoids inheriting exponential sleep from exhausted SA
Broader rate-limit detection fclone Catches dailyLimitExceededUnreg and Daily Limit prefix errors
Service recycling fclone Old OAuth services returned to pool for reuse instead of discarded
Auto-assign SA fclone Automatically picks an SA if none configured but SA path exists
Auto-lower pacer fclone Reduces pacer min sleep when >10 SAs preloaded (more headroom)

Install

Quick install (Linux/macOS/BSD)

sudo -v ; curl -fsSL https://raw.githubusercontent.com/ebadenes/eclone/master/install.sh | sudo bash

This auto-detects your OS and architecture, downloads the latest release from GitHub, and installs the binary to /usr/bin (Linux/BSD) or /usr/local/bin (macOS).

Docker (no build needed)

FROM alpine:latest
RUN apk add --no-cache ca-certificates fuse3 tzdata curl unzip bash
RUN curl -fsSL https://raw.githubusercontent.com/ebadenes/eclone/master/install.sh | bash
ENTRYPOINT ["eclone"]

Self-update

sudo eclone eselfupdate

Build from source

With Docker (no Go installation needed)

# Clone the repository
git clone https://github.com/ebadenes/eclone.git
cd eclone

# Build using Go 1.24 Docker image
docker run --rm -v "$(pwd)":/app -w /app golang:1.24 \
  go build -buildvcs=false -ldflags="-s -w" -o /app/eclone .

# Verify
./eclone eversion

With local Go

# Requires Go 1.24+
go build -ldflags="-s -w" -o eclone .

# With mount support (requires FUSE + gcc)
go build -v -tags 'cmount' -ldflags="-s -w" -o eclone .

Manual install after build

sudo cp eclone /usr/local/bin/eclone

Configuration

1. Service Account File Path

Add service_account_file_path to your rclone config for dynamic SA rotation. SAs are switched automatically when rate-limit errors occur.

~/.config/rclone/rclone.conf example:

[gc]
type = drive
scope = drive
service_account_file = /path/to/accounts/1.json
service_account_file_path = /path/to/accounts/
root_folder_id = root

The accounts folder should contain multiple SA JSON files with appropriate Google Drive permissions.

2. Advanced SA Options

These options can be set in rclone.conf or via command-line flags:

Option Flag Default Description
service_account_file_path --drive-service-account-file-path (empty) Path to directory containing SA JSON files
random_pick_sa --drive-random-pick-sa false Random SA selection at startup instead of first file
rolling_sa --drive-rolling-sa false Proactive SA rotation before each operation
rolling_count --drive-rolling-count 1 Parallel operations sharing the same SA
service_account_min_sleep --drive-service-account-min-sleep 100ms Minimum time between SA changes (anti-thrashing)
services_preload --drive-services-preload 50 Number of SA services to preload at startup
services_max --drive-services-max 100 Maximum preloaded services kept in memory

3. Folder ID Support

eclone supports passing Google Drive folder/file IDs directly using curly braces:

# Copy between folders by ID
eclone copy gc:{folder_id1} gc:{folder_id2} --drive-server-side-across-configs

# Copy to a subfolder path within a destination ID
eclone copy gc:{folder_id1} gc:{folder_id2}/media/ --drive-server-side-across-configs

# Copy from a shared drive
eclone copy gc:{shared_drive_id} gc:{folder_id2} --drive-server-side-across-configs

4. Command-Line Examples

# Basic copy with SA rotation (auto on rate limit)
eclone copy gc:{id1} gc:{id2} --drive-service-account-file-path=/path/to/SAs/

# Rolling SA mode (proactive rotation + random initial pick)
eclone copy gc:{id1} gc:{id2} \
  --drive-rolling-sa \
  --drive-rolling-count=1 \
  --drive-random-pick-sa

# With preloading and custom throttle
eclone copy gc:{id1} gc:{id2} \
  --drive-service-account-file-path=/path/to/SAs/ \
  --drive-services-preload=30 \
  --drive-service-account-min-sleep=200ms

5. Self-Update

sudo eclone eselfupdate [--check] [--output path] [--version v] [--package zip|deb|rpm]

How SA Rotation Works

                   Startup
                      |
           Load SA files from path
                      |
              Preload N services
             (OAuth clients ready)
                      |
        +----- Begin operations -----+
        |                            |
   [rolling_sa=true]          [rolling_sa=false]
        |                            |
  Rotate SA proactively       Use current SA
  before each operation              |
        |                     On rate-limit error
        |                            |
        |                   shouldChangeSA()?
        |                    (throttle guard)
        |                            |
        |                   changeSvc():
        |                   - Blacklist old SA (25h)
        |                   - Get new SA from pool
        |                   - Recycle old service
        |                   - Reset pacer
        |                            |
        +------- Continue -----------+

Google Drive Quotas

Service Accounts allow bypassing some Google quotas:

CAN bypass with multiple SAs:

  • Copy/upload quota (750 GB/account/day)
  • Download quota (10 TB/account/day)

CANNOT bypass:

  • Shared Drive quota (~20 TB/drive/day)
  • File owner quota (~2 TB/day)

Credits

  • rclone - The cloud sync tool
  • gclone - SA rotation foundation and rolling SA
  • fclone - SA pool, preloading, and blacklist features

License

Same as rclone - MIT License.