feat: dlt.Dataset interface over load packages

[zilto]

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	docs	bfc6dbd	Commit Preview URL Branch Preview URL	Aug 11 2026, 01:37 PM

[rudolfix]

@zilto Looks good - we need a destination with duckdb table scanner that will map files in load packages into view. What I'd do differently: I'd use filesystem destination for that
and reuse existing table scanner over there. We can adjust the layout of jobs in load package to match.

OK - I did an experiment and it looks everything works already. attached demo runs on stock dlt and uses filesystem to read PackageStorage after normalize. I patched Pipeline to add normalized_dataset() that will return it after normalization. we could even scan/load INSERT_VALUES files in duckdb - they are compatible.

we could add this to dq in dlthub. pls take a look:
load_package_as_filesystem_dataset.py