Modular TypeScript media extraction and download toolkit. Each site integration is split into a provider, parser, transformer, and pipeline so extraction rules, output mapping, and download planning stay easy to update independently.
npm install downfluxpnpm add downfluximport { BeegProvider } from 'downflux';
const provider = new BeegProvider('https://beeg.com/example-video-url');
const result = await provider.getVideo();
console.log(result);DownFlux uses ffmpeg when HLS or fragmented media needs to be finalized into a playable file. The package includes ffmpeg-static, but some package managers can block its postinstall script.
If you use pnpm in the consuming app, approve the bundled binary build:
pnpm approve-buildsSelect ffmpeg-static, then reinstall dependencies if needed.
You can also install ffmpeg yourself:
brew install ffmpegThen point DownFlux at that executable:
import { BeegProvider, OutputType } from 'downflux';
await new BeegProvider('https://beeg.com/example-video-url')
.setOutput(OutputType.DEVICE, { directoryPath: 'downloads' })
.setJobOptions({
transcodeOptions: {
ffmpegPath: '/opt/homebrew/bin/ffmpeg'
}
})
.getVideo();The generated Markdown API docs live in docs-md.
Useful entry points:
- BaseProvider - public provider API and fluent execution options.
- BaseParser - shared HTML extraction helpers.
- BaseTransformer - fetches pages and normalizes parser output.
- BasePipeline - turns extracted metadata into downloadable items.
- ExecutionCoordinator - job-level extraction and output flow.
- TaskCoordinator - concurrency, hooks, and background downloads.
- TransferCoordinator - streams one pipeline item into storage.
- HttpClient, StreamHttpClient, and HlsClient - HTTP and HLS engines.
- FileManager and FFmpegEngine - output sinks, filenames, JSON, and media finalization.
- Provider, OutputType, ExecutionType, VideoQuality.
Regenerate the Markdown docs with:
pnpm run docs:mdflowchart TD
User([Caller chooses Provider])
subgraph ProviderLayer["Provider Layer"]
ServiceProvider[ServiceProvider]
BaseProvider[BaseProvider<br/>• Validates URL<br/>• Builds execution request]
end
subgraph TransformLayer["Transformation Layer"]
TransformerRegistry[TransformerRegistry<br/>Resolves provider transformer]
Transformer[Transformer<br/>Coordinates extraction lifecycle]
HttpClient[HttpClient<br/>Fetches HTML / APIs]
ParserRegistry[ParserRegistry<br/>Loads matching parsers]
Parser[Parser<br/>• Extracts metadata<br/>• Detects media sources<br/>• Finds identifiers]
end
subgraph PipelineLayer["Pipeline Layer"]
PipelineRegistry[PipelineRegistry<br/>Loads provider pipeline]
Pipeline[Pipeline<br/>Builds execution items]
PipelineItems[Pipeline Items]
URLs[URLs<br/>Media / Playlist URLs]
MediaTypes[Media Types<br/>Video / Audio / Images]
Identifiers[Identifiers<br/>Content IDs / Metadata]
end
subgraph ExecutionLayer["Execution Layer"]
TaskCoordinator[TaskCoordinator<br/>Schedules concurrent tasks]
TransferCoordinator[TransferCoordinator<br/>Coordinates downloads]
end
subgraph StreamLayer["Stream & Output Layer"]
StreamHttpClient[StreamHttpClient<br/>Streams remote files<br/>• Uses Strategy for CDN fallback<br/>• Delegates HLS to HlsClient]
StrategyRegistry[StrategyRegistry<br/>Loads extraction strategies]
Strategy[Strategy<br/>• Defines extraction rules<br/>• Provider-specific behavior]
HlsClient[HlsClient<br/>Processes m3u8 playlists]
FileManager[FileManager<br/>Handles sinks & temp files]
FFmpegEngine[FFmpegEngine<br/>Muxing / Conversion / Finalization]
Output[Output<br/>JSON / Buffer / Device File]
end
User --> ServiceProvider
ServiceProvider --> BaseProvider
BaseProvider --> TransformerRegistry
TransformerRegistry --> Transformer
Transformer --> HttpClient
HttpClient --> ParserRegistry
ParserRegistry --> Parser
Parser --> Transformer
Transformer --> PipelineRegistry
PipelineRegistry --> Pipeline
Pipeline --> PipelineItems
PipelineItems --> URLs
PipelineItems --> MediaTypes
PipelineItems --> Identifiers
Pipeline --> TaskCoordinator
TaskCoordinator --> TransferCoordinator
TransferCoordinator --> StreamHttpClient
StreamHttpClient --> StrategyRegistry
StrategyRegistry --> Strategy
Strategy -.->|CDN Fallback<br/>Re-extraction| StreamHttpClient
StreamHttpClient --> HlsClient
StreamHttpClient --> FileManager
HlsClient --> FileManager
FileManager --> FFmpegEngine
FileManager --> Output
FFmpegEngine --> Output
classDef provider fill:#4f46e5,color:#fff,stroke:#312e81,stroke-width:2px;
classDef transform fill:#0f766e,color:#fff,stroke:#134e4a,stroke-width:2px;
classDef pipeline fill:#ca8a04,color:#fff,stroke:#854d0e,stroke-width:2px;
classDef execution fill:#dc2626,color:#fff,stroke:#7f1d1d,stroke-width:2px;
classDef network fill:#0369a1,color:#fff,stroke:#0c4a6e,stroke-width:2px;
classDef parser fill:#7c3aed,color:#fff,stroke:#581c87,stroke-width:2px;
classDef strategy fill:#8b5cf6,color:#fff,stroke:#6d28d9,stroke-width:2px;
classDef stream fill:#059669,color:#fff,stroke:#064e3b,stroke-width:2px;
classDef output fill:#ea580c,color:#fff,stroke:#7c2d12,stroke-width:2px;
class ServiceProvider,BaseProvider provider;
class TransformerRegistry,Transformer transform;
class PipelineRegistry,Pipeline,PipelineItems,URLs,MediaTypes,Identifiers pipeline;
class TaskCoordinator,TransferCoordinator execution;
class HttpClient network;
class ParserRegistry,Parser parser;
class StrategyRegistry,Strategy strategy;
class StreamHttpClient,HlsClient stream;
class FileManager,FFmpegEngine stream;
class Output output;
In short: a provider creates a typed request, coordinators run the extraction/download flow, registries load the correct provider-specific classes, engines handle network transport, pipelines decide what should be downloaded, and storage writes or returns the result.
| Site | Provider | Short description |
|---|---|---|
| AnalRz new | AnalRzProvider |
MP4 downloads; under development. |
| Beeg | BeegProvider |
MP4 downloads, HLS downloads; geo-sensitive, under development. |
| BlackPorn new | BlackPornProvider |
MP4 downloads; under development. |
| BoKepPorn new | BoKepPornProvider |
MP4 downloads, KVS metadata; under development. |
| ColliderPorn | ColliderPornProvider |
MP4 downloads, HLS downloads, embeds; geo-sensitive, under development. |
| CumLouder | CumLouderProvider |
MP4 discovery; under development. |
| DaFreePorn new | DaFreePornProvider |
MP4 downloads, KVS metadata; under development. |
| DaNude new | DaNudeProvider |
MP4 downloads, KVS metadata; under development. |
| EpicGfs | EpicGfsProvider |
MP4 downloads, KVS metadata; under development. |
| EPorner | EPornerProvider |
MP4 downloads, HLS downloads; external API, geo-sensitive, under development. |
| HqPorn | HqPornProvider |
MP4 downloads; under development. |
| Interracial new | InterracialProvider |
MP4 downloads, KVS metadata; under development. |
| ItsPorn new | ItsPornProvider |
MP4 downloads, KVS metadata; under development. |
| Lesbian8 | Lesbian8Provider |
MP4 downloads, KVS metadata; under development. |
| MegaTube | MegaTubeProvider |
MP4 downloads, KVS metadata; under development. |
| MomVids new | MomVidsProvider |
MP4 downloads, KVS metadata; under development. |
| MyLust | MyLustProvider |
MP4 downloads; under development. |
| OkPorn | OkPornProvider |
HLS downloads; under development. |
| PerfectGirls | PerfectGirlsProvider |
HLS downloads; under development. |
| Porn300 | Porn300Provider |
MP4 discovery; under development. |
| PornDoe | PornDoeProvider |
MP4 downloads; external API, under development. |
| PornHub | PornHubProvider |
MP4 downloads, HLS downloads, KVS metadata; under development. |
| PornId | PornIdProvider |
MP4 downloads, KVS metadata; under development. |
| PornOne | PornOneProvider |
MP4 downloads; Cloudflare challenge, under development. |
| PornSeven | PornSevenProvider |
MP4 discovery; under development. |
| PornsOk | PornsOkProvider |
MP4 downloads; under development. |
| PussySpace | PussySpaceProvider |
MP4 downloads; external API, under development. |
| SexVid | SexVidProvider |
MP4 downloads, KVS metadata; under development. |
| Shameless | ShamelessProvider |
MP4 downloads, KVS metadata; under development. |
| SuperPorn | SuperPornProvider |
MP4 downloads; under development. |
| SxyPorn | SxyPornProvider |
MP4 downloads; Cloudflare challenge, under development. |
| TheyAreHuge | TheyAreHugeProvider |
MP4 downloads, KVS metadata; login required, under development. |
| TnAFlix | TnAFlixProvider |
MP4 downloads; geo-sensitive, under development. |
| TubeVSex | TubeVSexProvider |
MP4 downloads; under development. |
| WallHaven | WallHavenProvider |
Provider-specific extraction; under development. |
| XCafe | XCafeProvider |
MP4 downloads; under development. |
| XDegu | XDeguProvider |
MP4 downloads, KVS metadata; under development. |
| XGroovy | XGroovyProvider |
MP4 downloads; under development. |
| XHamster | XHamsterProvider |
MP4 downloads, HLS downloads; under development. |
| XnXX | XnXXProvider |
MP4 downloads, HLS downloads; under development. |
| Xozilla new | XozillaProvider |
MP4 downloads, KVS metadata; under development. |
| XVideos | XVideosProvider |
MP4 discovery, HLS downloads, KVS metadata; under development. |
| ZbPorn new | ZbPornProvider |
MP4 downloads, KVS metadata; under development. |
| ZzzTube | ZzzTubeProvider |
MP4 downloads; under development. |
| Site | Provider | Short description |
|---|---|---|
| ArtStation new | ArtStationProvider |
Art gallery extraction; under development. |
| Behance new | BehanceProvider |
Portfolio platform extraction; requires API, under development. |
| Bluesky new | BlueskyProvider |
Social media extraction; requires API, under development. |
| Danbooru new | DanbooruProvider |
Anime image board extraction; under development. |
| DeviantArt new | DeviantArtProvider |
Art community extraction; requires API, under development. |
| Flickr new | FlickrProvider |
Photo sharing extraction; under development. |
| Gelbooru new | GelbooruProvider |
Anime image board extraction; under development. |
| Imgur new | ImgurProvider |
Image hosting extraction; under development. |
| Instagram new | InstagramProvider |
Social media extraction; login required, Cloudflare challenge, under development. |
| MangaDex new | MangaDexProvider |
Manga reader extraction; under development. |
| Mastodon new | MastodonProvider |
Social media extraction; geo-sensitive, under development. |
| Newgrounds new | NewgroundsProvider |
Art portal extraction; under development. |
| Pexels new | PexelsProvider |
Stock photo extraction; under development. |
| Pinterest new | PinterestProvider |
Image sharing extraction; requires API, Cloudflare challenge, under development. |
| Pixiv new | PixivProvider |
Art community extraction; requires API, login required, under development. |
| Reddit new | RedditProvider |
Social media extraction; under development. |
| TikTok new | TikTokProvider |
Video platform extraction; requires API, Cloudflare challenge, under development. |
| Tumblr new | TumblrProvider |
Social media extraction; under development. |
| Twitter new | TwitterProvider |
Social media extraction; requires API, login required, Cloudflare challenge, under development. |
| Unsplash new | UnsplashProvider |
Stock photo extraction; under development. |
| WikiArt new | WikiArtProvider |
Art museum extraction; under development. |
| Wikimedia new | WikimediaProvider |
Media library extraction; geo-sensitive, under development. |
More incoming...
pnpm install
pnpm run build
pnpm run docs:mdTo add a new provider you can run this following commands.
Use the --no-[registryCoordinatorType] to exclude that file.
Available flags: strategy | transformer | pipeline | method
pnpm run make:registry [registryName] OR
pnpm run make:registry --no-strategy TwitterAfter running the previous command, run this commands sequentially for barrel imports and auto entries
pnpm run make:index
pnpm run format
pnpm run buildThe generated docs are intentionally linked from this README so provider and API documentation remain browsable from the repository root.
pnpm run pack:dry-run
pnpm publishpnpm run pack:dry-run rebuilds the package and previews the files that npm will receive.