597 lines (305 loc) · 17.5 KB

downflux / TumblrExecArgs

Interface: TumblrExecArgs

Defined in: packages/providers/tumblr/TumblrContracts.ts:3

Configuration options for a DownFlux ExecutionCoordinator. Combines fetch, extraction, pipeline, and output settings.

Extends

ExecutionArgs

Properties

headers?

optional headers?: Record<string, string>

Defined in: packages/contracts/DownloadContracts.ts:127

Custom request headers

Inherited from

ExecutionArgs.headers

timeoutMs?

optional timeoutMs?: number

Defined in: packages/contracts/DownloadContracts.ts:130

Request timeout in milliseconds

Inherited from

ExecutionArgs.timeoutMs

retries?

optional retries?: number

Defined in: packages/contracts/DownloadContracts.ts:133

Failed request retry count

Inherited from

ExecutionArgs.retries

referer?

optional referer?: string

Defined in: packages/contracts/DownloadContracts.ts:136

Request referer URL

Inherited from

ExecutionArgs.referer

formData?

optional formData?: Record<string, string>

Defined in: packages/contracts/DownloadContracts.ts:139

Optional FormData for POST requests

Inherited from

ExecutionArgs.formData

provider

provider: Provider

Defined in: packages/contracts/ExecutionContracts.ts:21

Inherited from

ExecutionArgs.provider

method

method: string

Defined in: packages/contracts/ExecutionContracts.ts:22

Inherited from

ExecutionArgs.method

entryUrl

entryUrl: string

Defined in: packages/contracts/ExecutionContracts.ts:23

Inherited from

ExecutionArgs.entryUrl

targets

targets: string[]

Defined in: packages/contracts/ExecutionContracts.ts:24

Inherited from

ExecutionArgs.targets

executionShape

executionShape: ExecutionShape

Defined in: packages/contracts/ExecutionContracts.ts:29

Internal runtime metadata describing the structural shape of extracted output. single -> TResult; multiple -> TResult[]

Inherited from

ExecutionArgs.executionShape

executionType?

optional executionType?: ExecutionType

Defined in: packages/contracts/ExecutionContracts.ts:31

Job execution strategy

Inherited from

ExecutionArgs.executionType

extractionTarget

extractionTarget: ExtractionTarget

Defined in: packages/contracts/ExecutionContracts.ts:32

Inherited from

ExecutionArgs.extractionTarget

providerMetadata?

optional providerMetadata?: ProviderMetadata

Defined in: packages/contracts/ExecutionContracts.ts:35

Provider capabilities and restrictions

Inherited from

ExecutionArgs.providerMetadata

dirConfig?

optional dirConfig?: DirectoryOutputOptions

Defined in: packages/contracts/ExecutionContracts.ts:68

Directory output configuration

Inherited from

ExecutionArgs.dirConfig

allowedExtensions?

optional allowedExtensions?: AllowedExtension[]

Defined in: packages/contracts/ExecutionContracts.ts:71

Allowed file extensions

Inherited from

ExecutionArgs.allowedExtensions

allowedVideoQuality?

optional allowedVideoQuality?: VideoQuality

Defined in: packages/contracts/ExecutionContracts.ts:74

Allowed video quality

Inherited from

ExecutionArgs.allowedVideoQuality

preferredVideoFormat?

optional preferredVideoFormat?: VideoFormat

Defined in: packages/contracts/ExecutionContracts.ts:77

Preferred video format (e.g. hls, mp4)

Inherited from

ExecutionArgs.preferredVideoFormat

preferredVideoCodec?

optional preferredVideoCodec?: VideoCodec

Defined in: packages/contracts/ExecutionContracts.ts:80

Preferred video codec (e.g. h264, av1)

Inherited from

ExecutionArgs.preferredVideoCodec

tagFilterOptions?

optional tagFilterOptions?: TagFilterOptions

Defined in: packages/contracts/ExecutionContracts.ts:83

Tag filtering options

Inherited from

ExecutionArgs.tagFilterOptions

maxDownloads?

optional maxDownloads?: number

Defined in: packages/contracts/ExecutionContracts.ts:86

Maximum number of items to download

Inherited from

ExecutionArgs.maxDownloads

transformOutput?

optional transformOutput?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:89

Transform output to service-specific result type

Inherited from

ExecutionArgs.transformOutput

concurrency?

optional concurrency?: number

Defined in: packages/contracts/ExecutionContracts.ts:92

Download phase concurrency

Inherited from

ExecutionArgs.concurrency

noDownload?

optional noDownload?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:95

Iterate only-- this prop is only used for logging http-services

Inherited from

ExecutionArgs.noDownload

extractConcurrency?

optional extractConcurrency?: number

Defined in: packages/contracts/ExecutionContracts.ts:98

Extraction phase concurrency

Inherited from

ExecutionArgs.extractConcurrency

captureConsole?

optional captureConsole?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:113

Whether progress rendering may take over stdout/stderr.

Remarks

Off by default. Live progress redraws in place, so anything written directly to the terminal while a job renders is erased by the next frame. Enabling this patches both streams for the duration of the job so console.log survives.

It is opt-in because patching a global belongs to the application that owns the process, not to a library running inside it.

Default Value

false

Inherited from

ExecutionArgs.captureConsole

abortOnSignal?

optional abortOnSignal?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:130

Whether SIGINT/SIGTERM should cancel the job before the process exits.

Remarks

Defaults to true for OutputType.DEVICE and false for every other mode.

Device output writes partial .part files, so an interrupted run leaves debris unless something unwinds it. Progress rendering already registers a signal listener to restore the cursor, which suppresses Node's default exit and makes this library the owner of shutdown regardless - having claimed it, exiting without deleting the partial files it created is not defensible.

STREAM defaults to false because it is used inside servers that own their own shutdown sequence; set it explicitly to override either default.

Inherited from

ExecutionArgs.abortOnSignal

maxCdnFallbacks?

optional maxCdnFallbacks?: number

Defined in: packages/contracts/ExecutionContracts.ts:133

Maximum CDN fallback attempts allowed per download item.

Inherited from

ExecutionArgs.maxCdnFallbacks

maxReExtractions?

optional maxReExtractions?: number

Defined in: packages/contracts/ExecutionContracts.ts:136

Maximum expired-URL re-extraction attempts allowed per download item.

Inherited from

ExecutionArgs.maxReExtractions

transcodeOptions?

optional transcodeOptions?: TranscodeOptions

Defined in: packages/contracts/ExecutionContracts.ts:139

Transcoding options

Inherited from

ExecutionArgs.transcodeOptions

downloadRetries?

optional downloadRetries?: number

Defined in: packages/contracts/ExecutionContracts.ts:142

Download retry count

Inherited from

ExecutionArgs.downloadRetries

retryDelayMs?

optional retryDelayMs?: number

Defined in: packages/contracts/ExecutionContracts.ts:145

Delay between download retries in milliseconds

Inherited from

ExecutionArgs.retryDelayMs

pipelineHooks?

optional pipelineHooks?: PipelineHook[]

Defined in: packages/contracts/ExecutionContracts.ts:148

PipelineRegistry lifecycle hooks

Inherited from

ExecutionArgs.pipelineHooks

onProgress?

optional onProgress?: (event) => void

Defined in: packages/contracts/ExecutionContracts.ts:151

Progress event handler

Parameters

event

JobProgressEvent

Returns

void

Inherited from

ExecutionArgs.onProgress

logProgress?

optional logProgress?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:154

Enables console progress logging

Inherited from

ExecutionArgs.logProgress

outputType?

optional outputType?: OutputType

Defined in: packages/contracts/ExecutionContracts.ts:157

Output format for ExecutionCoordinator results

Inherited from

ExecutionArgs.outputType

signal?

optional signal?: AbortSignal

Defined in: packages/contracts/ExecutionContracts.ts:163

Abort signal for cancelling the ExecutionCoordinator

Inherited from

ExecutionArgs.signal

userAgent?

optional userAgent?: string

Defined in: packages/contracts/ExecutionContracts.ts:167

Inherited from

ExecutionArgs.userAgent

enableSniSpoofing?

optional enableSniSpoofing?: boolean

Defined in: packages/contracts/ExecutionContracts.ts:169

Inherited from

ExecutionArgs.enableSniSpoofing

proxy?

optional proxy?: ProxyOptions

Defined in: packages/contracts/ExecutionContracts.ts:171

Inherited from

ExecutionArgs.proxy

dispatcher?

optional dispatcher?: Dispatcher

Defined in: packages/contracts/ExecutionContracts.ts:173

Inherited from

ExecutionArgs.dispatcher