[Feature] Bash script to concurrently download from a list

[nuzzzen]

Feature Proposal

Hi, I've written a simple bash script that allows you to download multiple files from a list, using concurrency.

Essentially, the script requires a list of n URLs and will generate n processes that call vimeo-dl.
I just tried downloading ~50 files and it's worked flawlessly. Eventually, it could be integrated directly in the Go code.

Bash script

#!/bin/bash

# Read the list of URLs from the file
urls=$(cat $1)

# Count the number of URLs
url_count=$(echo "$urls" | wc -l)

# User agent. You can change it as you wish
user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36"

# Function to download using vimeo-dl. Add flags if needed.
download_video() {
  local url=$1
  vimeo-dl \
    --combine \
    -i "$url" \
    --user-agent "$user_agent"
}

# Export the function so it can be used by parallel
export -f download_video

# Run the download_video function concurrently for each URL
echo "$urls" | xargs -n 1 -P "$url_count" -I {} bash -c 'download_video "$@"' _ {}

Usage

1. Generate a list.txt file containing master.json urls.
2. Execute the script and pass the list via args:
   ./script.sh list.txt
3. Your videos will be exported to your pwd.