BOB YouTube

Break Ordinary Boundaries for YouTube.

BOB YouTube is a Docker-first ingestion runtime that turns YouTube URLs into structured, inspectable, reusable data.

It uses yt-dlp as the extraction engine, then wraps it with a product-grade surface: CLI, Docker image, OpenAPI service, runtime profiles, durable jobs, metrics, transcript/RAG workflows, and release proof.

In one line: URL in, clean /data corpus out.

Visual Reading Path

The README visuals are intentionally split into three jobs:

Visual	Purpose
Hero banner	The product promise: BOB YouTube is the ingestion layer for YouTube workflows.
Capability map	The product surface: URL input, Docker, CLI, OpenAPI, `/data`, captions, transcripts, segments, media, jobs, metrics, RAG, sidecars.
Data contract	The integration contract: predictable `/data` folders that other tools and agents can consume.

The 5 Ws

Question	Answer
What is it?	A Docker-first YouTube ingestion runtime over yt-dlp, packaged as a CLI and local API.
Who is it for?	Developers, AI agents, RAG builders, researchers, content operators, internal tools, and bulk workflows.
Why does it exist?	Raw download commands are powerful but hard to repeat safely across products. BOB YouTube turns that power into a stable corpus and API contract.
When should I use it?	When you need transcripts, captions, media, metadata, segment JSON, job logs, metrics, or RAG-ready outputs from YouTube URLs.
Where does it run?	Locally, in Docker, in Docker Compose, or as a token-gated API service mounted to a `/data` volume.
How does it work?	It calls yt-dlp through controlled BOB commands, writes artifacts into predictable folders, records jobs/events/logs, and exposes the same workflow through CLI and OpenAPI.

Why It Is Different

BOB YouTube is not just a downloader wrapper and not a consumer GUI. It is a small operating layer for YouTube ingestion.

Normal approach	BOB YouTube approach
One-off terminal commands	Reusable `bob-youtube` commands
Local machine setup required	Docker-first runtime
Files scattered by command options	Stable `/data/VIDEO_ID/` corpus layout
Terminal output disappears	Job JSON, logs, events, request IDs
Transcript text only	VTT, clean text, timestamped segments, RAG bundles
Manual integration	OpenAPI endpoints and signed webhooks
Unclear proof	Release gate, manifest, SBOM, checksums, readiness JSON

What You Can Build With It

Build	BOB YouTube gives you
Research notebook	Captions, transcripts, metadata, thumbnails, segment JSON
RAG pipeline	`chunks.jsonl`, `corpus.txt`, `manifest.json`, segment timestamps
AI agent tool	Deterministic CLI/API surface, request IDs, machine events
Internal dashboard	Token-gated API, `/library`, `/jobs`, `/metrics`, OpenAPI
Bulk downloader	Profiles, archive file, retries, workers, durable job records
Content operations flow	Audio, video, subtitles, thumbnails, metadata, indexes
Monitoring integration	JSON metrics, Prometheus text metrics, failure categories
Private/authenticated workflow	Explicit read-only `cookies.txt` mount contract
Advanced yt-dlp workflow	`bob-youtube raw` with BOB-owned safety boundaries

Common Problems It Solves

People search for this space in many different ways. BOB YouTube is designed to cover the practical developer side of those needs:

YouTube transcript generator: turn a video URL into readable transcript text when captions are available.
YouTube transcript downloader: save VTT captions and clean .txt transcripts into a repeatable folder layout.
YouTube subtitle downloader: list and download available subtitle or caption tracks with explicit language keys.
YouTube to text: convert captions into text plus timestamped segment JSON for search, review, and automation.
Bulk YouTube transcript extraction: process URL lists with archives, profiles, retries, and durable job records.
YouTube transcript API: run a local token-gated OpenAPI service for products and agents.
yt-dlp Docker API: use yt-dlp power through Docker while keeping BOB-owned output, archive, cookie, config, and plugin boundaries.
YouTube RAG pipeline: export transcript segments into chunks.jsonl, corpus.txt, and manifest.json.
YouTube media ingestion: collect metadata, thumbnails, captions, audio, video, logs, events, and metrics into one /data contract.

Core Capabilities

Ingest

inspect a URL,
list available captions,
download metadata,
download thumbnails,
download audio,
download merged video,
download captions and subtitles,
batch URLs with archive support.

Transform

clean VTT captions into readable transcript text,
convert captions into timestamped segment JSON,
create corpus indexes,
export RAG-ready bundles,
validate trusted embedding-sidecar output,
run trusted STT-sidecar fallback when captions are missing.

Operate

run as CLI,
run as Docker CLI,
run as Docker Compose service,
run as local HTTP API,
persist job JSON, logs, and JSONL events,
recover queued/running job state after restart,
expose metrics and Prometheus output,
send signed job-finished webhooks.

Prove

smoke test local CLI, Docker CLI, and Docker API,
generate release manifests,
generate SBOM artifacts,
sign SHA-256 checksums,
emit machine-readable readiness JSON,
keep runtime outputs and proof downloads out of source control.

Quick Start

Docker Compose

cp .env.example .env
docker compose build
docker compose run --rm bob-youtube doctor

Download transcript and audio:

docker compose run --rm bob-youtube both 'https://www.youtube.com/watch?v=VIDEO_ID'

Run the API:

docker compose up bob-youtube-api

Open the API spec:

curl -fsS http://127.0.0.1:8080/openapi.json

Check health:

curl -fsS \
  -H 'Authorization: Bearer change-this-token' \
  http://127.0.0.1:8080/health

One Docker Command

docker run --rm \
  -v "$PWD/bob-youtube-data:/data" \
  bob-youtube:local both 'https://www.youtube.com/watch?v=VIDEO_ID'

Start API mode:

docker run --rm \
  -e BOB_YOUTUBE_API_TOKEN='change-this-token' \
  -p 8080:8080 \
  -v "$PWD/bob-youtube-data:/data" \
  bob-youtube:local api 8080

Local Command

bob-youtube doctor
bob-youtube both 'https://www.youtube.com/watch?v=VIDEO_ID'
bob-youtube index

Typical global symlink:

/opt/homebrew/bin/bob-youtube -> /absolute/path/to/bob-youtube/bin/bob-youtube

Output Contract

Docker writes to:

/data

Local runs write to:

outputs/

Per-video layout:

VIDEO_ID/
  VIDEO_ID.info.json
  VIDEO_ID.jpg
  VIDEO_ID.mp3
  VIDEO_ID.mkv
  VIDEO_ID.LANG.vtt
  VIDEO_ID.LANG.transcript.txt
  VIDEO_ID.LANG.segments.json

Corpus and API artifacts:

library-index.json
library-index.tsv
rag-export/chunks.jsonl
rag-export/corpus.txt
rag-export/manifest.json
.bob-youtube/jobs/JOB_ID.json
.bob-youtube/jobs/JOB_ID.log
.bob-youtube/jobs/JOB_ID.events.jsonl

This layout is the main integration contract. Another product can mount /data, call BOB YouTube, and consume the generated files without scraping terminal output.

CLI Surface

bob-youtube info URL
bob-youtube subs URL
bob-youtube metadata URL
bob-youtube thumbnail URL
bob-youtube transcript URL [SUB_LANGS]
bob-youtube transcript-or-stt URL [SUB_LANGS]
bob-youtube audio URL
bob-youtube video URL
bob-youtube both URL [SUB_LANGS]
bob-youtube batch urls.txt [SUB_LANGS]
bob-youtube clean PATH
bob-youtube segments PATH
bob-youtube index
bob-youtube rag [PATH] [OUT_DIR]
bob-youtube embed [RAG_DIR_OR_CHUNKS_JSONL] [OUT_FILE]
bob-youtube stt MEDIA_PATH [OUT_DIR]
bob-youtube raw YT_DLP_ARGS...
bob-youtube where VIDEO_ID
bob-youtube open VIDEO_ID
bob-youtube api [PORT]
bob-youtube profiles
bob-youtube version
bob-youtube root

API Surface

Public discovery:

GET /
GET /version
GET /openapi.json

Operational endpoints:

GET  /health
GET  /library
GET  /library?refresh=1
GET  /where/VIDEO_ID
GET  /jobs
GET  /jobs/JOB_ID
GET  /jobs/JOB_ID/log
GET  /jobs/JOB_ID/events
GET  /metrics
GET  /metrics.prom
POST /jobs

Operational endpoints require Authorization: Bearer TOKEN when BOB_YOUTUBE_API_TOKEN is configured.

Create an async job:

curl -fsS -X POST http://127.0.0.1:8080/jobs \
  -H 'Authorization: Bearer change-this-token' \
  -H 'Content-Type: application/json' \
  -d '{"command":"both","url":"https://www.youtube.com/watch?v=VIDEO_ID","async":true}'

Poll it:

curl -fsS \
  -H 'Authorization: Bearer change-this-token' \
  http://127.0.0.1:8080/jobs/JOB_ID

Formal contract: docs/openapi.json.

Captions and Language Truth

BOB YouTube does not pretend to detect spoken language itself. yt-dlp reads caption and manifest language metadata from the source platform.

Check captions first:

bob-youtube subs 'https://www.youtube.com/watch?v=VIDEO_ID'

Default transcript selection is auto:

hi-orig -> en -> hi -> first non-live-chat key

For bulk work, prefer exact keys such as en, hi, or hi-orig. Broad patterns can request many translated caption tracks and trigger rate limits.

RAG, STT, and Embeddings

Create timestamped transcript segments:

bob-youtube segments outputs/VIDEO_ID

Create a RAG bundle:

bob-youtube rag outputs/VIDEO_ID outputs/VIDEO_ID/rag-export

BOB YouTube does not bundle a speech-to-text model. It provides a trusted sidecar contract:

BOB_YOUTUBE_STT_COMMAND='python3 /path/to/real-stt-sidecar.py' \
  bob-youtube stt outputs/VIDEO_ID/VIDEO_ID.mp3 outputs/VIDEO_ID

BOB YouTube does not bundle an embedding model. It validates trusted sidecar output:

BOB_YOUTUBE_EMBED_COMMAND='python3 /path/to/embed-sidecar.py' \
BOB_YOUTUBE_EMBED_MODEL='my-embedding-model' \
  bob-youtube embed outputs/VIDEO_ID/rag-export

This keeps 0.1 honest: the contract exists, the smoke stubs prove the flow, but model quality belongs to the sidecar you choose.

Advanced yt-dlp Lane

bob-youtube raw exposes advanced yt-dlp options while keeping BOB-owned runtime boundaries:

bob-youtube raw --simulate --print id 'https://www.youtube.com/watch?v=VIDEO_ID'
bob-youtube raw --list-formats 'https://www.youtube.com/watch?v=VIDEO_ID'

It is not a shell. It rejects path/archive overrides, config overrides, cookie overrides, plugin loading, --exec, batch-file reads, and unsafe print-to-file paths.

Runtime Profiles

Profiles live in config/profiles:

default
consumer
research
datacenter
agent

Use one with:

BOB_YOUTUBE_PROFILE=agent bob-youtube api 8080

Private Content and Cookies

Authenticated use is opt-in and file-based. Mount a Netscape cookies.txt read-only:

docker run --rm \
  -e BOB_YOUTUBE_COOKIES_FILE=/run/secrets/bob-youtube/cookies.txt \
  -v "$PWD/secrets/cookies.txt:/run/secrets/bob-youtube/cookies.txt:ro" \
  -v "$PWD/outputs:/data" \
  bob-youtube:local info 'https://www.youtube.com/watch?v=VIDEO_ID'

Do not bake cookies into the image, commit cookies, place cookies under /data, or mount browser profile directories into Docker.

Security Boundary

BOB YouTube 0.1 is suitable for local/private Docker developer use:

token-gated operational API,
request body limit,
per-client rate limit,
profile, URL, video ID, and output path validation,
request ID correlation across API responses and job artifacts,
explicit read-only cookie file contract,
runtime state outside the image,
image excludes upstream .git, work/, generated outputs, and media.

For public hosting, put it behind a gateway or reverse proxy with TLS, access control, rate limits, logging, abuse controls, and secret rotation.

See SECURITY.md and docs/SECURITY.md.

Status and Non-Scope

Current version:

0.1.0

Current readiness verdict:

stable-0.1-local-private-ready

That means BOB YouTube is ready as a local/private Docker developer product. It does not claim:

hosted public SaaS readiness,
public GHCR image publication,
distributed Redis/SQS/Postgres queueing,
bundled speech-to-text model quality,
bundled embedding model quality,
external secret-manager integration,
fragment-level network telemetry inside yt-dlp.

Release Proof

Run the full gate:

scripts/release_gate.sh

Generate release metadata, SBOM, checksums, and readiness JSON:

scripts/release_manifest.sh
scripts/release_artifacts.sh
scripts/release_readiness.sh

Verify signed checksums:

openssl pkeyutl -verify -rawin -pubin \
  -inkey outputs/release/bob-youtube-0.1.0-signing-public.pem \
  -sigfile outputs/release/bob-youtube-0.1.0-SHA256SUMS.sig \
  -in outputs/release/bob-youtube-0.1.0-SHA256SUMS

Exact image proof is generated by the release scripts:

outputs/release/bob-youtube-0.1.0-image-inspect.json
outputs/release/bob-youtube-0.1.0-readiness.json

Public Repository Boundary

The source repository should contain source, docs, scripts, config, and curated product docs only. It must not publish:

work/
outputs/.bob-youtube/
outputs/release/
outputs/*/
outputs/library-index.json
outputs/library-index.tsv
*.mp3
*.mkv
*.wav
*.vtt
*.transcript.txt
*.segments.json
.env
cookies/
secrets/
cookies.txt

The curated outputs/ files that belong in source are:

outputs/WORKSPACE-MAP.md
outputs/BOB-YOUTUBE-FEATURE-MATRIX.md

Project Map

bin/bob-youtube                 CLI wrapper and product command surface
scripts/bob_youtube_api.py      HTTP API over the CLI contract
scripts/clean_vtt_transcripts.py VTT to readable transcript text
scripts/vtt_to_segments.py      VTT to timestamped segment JSON
scripts/index_outputs.py        JSON/TSV corpus index
scripts/rag_export.py           Retrieval-ready corpus export
config/profiles/                Runtime profiles
docs/                           Product, Docker, API, security, release docs
DIVYANSHU.md                    Long-form truth ledger and engineering record

Docs

Document	Purpose
docs/DOCKER.md	Docker usage and integration
docs/PRODUCT-0.1.md	Product scope and non-scope
docs/SECURITY.md	Security model
docs/OPERATIONS.md	Release and operations runbook
docs/RELEASE.md	Release readiness
docs/PUBLIC-RELEASE.md	Public repo boundary
docs/COMPARISON.md	Ecosystem comparison and USP
docs/BRAND.md	Product language and visual direction
docs/LEGAL.md	Non-affiliation and acceptable use

Brand and Legal

BOB YouTube is an independent open-source project by Divyanshu Singh Chouhan, founder of ABC Steps Technologies Pvt Ltd.

Public identity:

ABCsteps: https://abcsteps.com/about/
GitHub: https://github.com/div197
LinkedIn: https://www.linkedin.com/in/divyanshu-singh-chouhan-a19032ba

BOB YouTube is not affiliated with, endorsed by, or sponsored by YouTube, Google, Alphabet, or yt-dlp. YouTube and Google names are used only to describe source-platform behavior. yt-dlp is a separate upstream project used as the backend extraction engine.

Use BOB YouTube only for content and accounts you are authorized to access, and respect applicable laws, platform terms, creator rights, privacy, and data protection obligations.

License

MIT. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.github		.github
bin		bin
config/profiles		config/profiles
docs		docs
outputs		outputs
scripts		scripts
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.workspace-name		.workspace-name
AGENTS.md		AGENTS.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
DIVYANSHU.md		DIVYANSHU.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
VERSION		VERSION
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation

BOB YouTube

Visual Reading Path

The 5 Ws

Why It Is Different

What You Can Build With It

Common Problems It Solves

Core Capabilities

Ingest

Transform

Operate

Prove

Quick Start

Docker Compose

One Docker Command

Local Command

Output Contract

CLI Surface

API Surface

Captions and Language Truth

RAG, STT, and Embeddings

Advanced yt-dlp Lane

Runtime Profiles

Private Content and Cookies

Security Boundary

Status and Non-Scope

Release Proof

Public Repository Boundary

Project Map

Docs

Brand and Legal

License

About

Topics

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages