Skip to content

README.md

Latest commit

聽

History

History
51 lines (38 loc) 路 1.38 KB

README.md

File metadata and controls

51 lines (38 loc) 路 1.38 KB
Raw

馃 HuggingFace Downloader

Fast parallel downloader for HuggingFace repositories using aria2c.

Usage

./hf.sh <repo_name> [OPTIONS]

Examples

# Download a model
./hf.sh Qwen/Qwen2.5-0.5B

# Download a dataset
./hf.sh HuggingFaceFW/fineweb-2 -ds

# Download with custom settings
./hf.sh Qwen/Qwen2.5-0.5B -x 16 -s 10 -d ./models

# Interactive file selection
./hf.sh Qwen/Qwen2.5-0.5B -i

# Exclude specific file patterns
./hf.sh Qwen/Qwen2.5-0.5B --exclude "*.sh,*.md,*.yaml"

Options

  • -ds - Download as dataset (default: disabled)
  • -i - Interactively select files to download (default: disabled)
  • -x <num> - Connections per server (default: 8)
  • -s <num> - Number of splits (default: 20)
  • -k <size> - Min split size (default: 1M)
  • -V <true|false> - Verify integrity (default: true)
  • -c <true|false> - Continue downloading (default: true)
  • -d <dir> - Download directory (default: repo_name)
  • --console-log-level <lvl> - Console log level (default: error)
  • --download-result <res> - Download result (default: full)
  • --summary-interval <sec> - Summary interval (default: 10)
  • --exclude <patterns> - Comma-separated glob patterns to exclude files (e.g., .sh,.md,*.yaml)
  • --token <token> - HuggingFace token for private repos
  • -h, --help - Show help

Requirements

  • aria2c
  • Python with huggingface_hub
  • fzf (for interactive mode)