Feature/python api#91
Open
ctsilva wants to merge 17 commits into
Open
Conversation
Jupyter pages (e.g. :8888) load the runtime from a separate dev server (:8000), which previously failed in two ways: browsers block cross-origin Web Workers (DuckDB-WASM), and root-relative data/schema URLs resolved against the notebook origin and 404ed. - Create the DuckDB worker from a same-origin blob: URL that importScripts() the real worker script (DuckDB-WASM recommended pattern) - Resolve root-relative data URLs against the runtime server origin in the Python to_html, and fetch the schema from the runtime module's origin - Default runtime_url to the absolute dev-server URL - Add CORS dev server, Jupyter quick-start script, and a Playwright cross-origin regression check (page on :8001, runtime on :8000) - Consolidate Jupyter docs into python/examples/JUPYTER_TESTING.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Lock in the cross-origin worker fix with an automated regression test and remove the manual-server requirements from the suite: - Add tests/runtime/cross-origin-worker.test.ts: serves the page and the runtime from different origins (like Jupyter + cors_server.py) and asserts DuckDB initializes and the map renders without SecurityError - Extract tests/helpers/static-server.ts and use it in jupyter-test.test.ts, which previously failed unless a server was started by hand on :8765 - Parametrize test-cross-origin.html via ?runtimeOrigin= and drop the redundant standalone check script - Record verified Jupyter/VS Code manual testing in the implementation plan 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
spec.widget() now renders specs in Jupyter/JupyterLab/VS Code/Colab with a fully bundled runtime shipped inside the Python package, removing the local-dev-server requirement of _repr_html_(). To make the bundle work from the blob:/data: URLs anywidget uses: - Bundle the spec schema inline (@autark-schema alias) instead of fetching it from the runtime server - Fall back to jsDelivr CDN bundles for DuckDB worker/wasm assets when the module origin is not http(s)/file; the same-origin blob worker pattern already handles the cross-origin worker New autk-runtime widget entry + vite.widget.config.js produce a single-file ESM at python/autark/static/autark-widget.js (gitignored build artifact, built by npm run build). Python side adds autark.create_widget / Spec.widget() behind an autark[widget] extra, warning on root-relative data URLs that have no server to resolve them. Covered by tests/runtime/widget-bundle.test.ts, which imports the bundle from a blob: URL like anywidget and serves jsDelivr requests locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add zero-server widget demo cell (inline data) to the integration notebook and present spec.widget() as the primary Jupyter option in QUICKSTART/JUPYTER_QUICKSTART - Replace stale plain http.server instructions with cors_server.py, which the dev-server display path requires - Ignore Python build artifacts (build/, egg-info) produced by wheel builds 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- save_html() now embeds the widget runtime bundle by default, producing a standalone document that renders from file:// or any host with no dev server (DuckDB assets come from the jsDelivr CDN); pass runtime_url for the previous lightweight dev-server output. Adds ak.to_embedded_html(). - Add example specs 04 (geojson input), 05 (csv points + histogram), and 06 (multi-layer OSM); 04/05 are executed in browser tests. Spec 05 omits the histogram-to-map link because highlight links to points layers crash the runtime - recorded as runtime TODO 5. - Set up strict mypy for the Python package (dev extra, pyproject config) and fix the findings. - Cover the embedded export end to end: Playwright test generates the file via the real Python path and asserts rendering with zero dev-server requests; new unit tests for display helpers; jsDelivr routing extracted to a shared test helper. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Implements full runtime support for JSON and GeoTIFF data sources, completing the data loading pipeline for these formats. Schema updates: - Add JsonDataSource with url/values options and flatten flag - Add GeoTiffDataSource with url, band selection, and CRS override - Update DataSource union type to include both new sources Runtime implementation: - Add loadJsonSource() to handle URL and inline JSON data - Add loadGeoTiffSource() for GeoTIFF files with band selection - Update TypeScript types to match schema definitions - Integrate with existing AutkDb.loadJson() and loadGeoTiff() methods Python API updates: - Update JSON class to support object/array values and flatten flag - Update GeoTIFF class to include band parameter (1-based index) - Add example scripts for both data source types Documentation: - Update PYTHON_API.md with usage examples and runtime support status - Mark implementation tasks as completed in PYTHON_API_IMPLEMENTATION.md - Add new example specs demonstrating JSON and GeoTIFF usage The implementation enables new use cases for tabular data analysis (JSON) and raster visualization (GeoTIFF), expanding Autark's data handling capabilities. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ation Implements complete runtime support for JSON and GeoTIFF data sources, including geometry support for JSON and OSMnx integration examples. JSON/GeoTIFF Runtime Support: - Add JSON data source with geometry support (lat/lng and WKT) - Add GeoTIFF raster data source with band selection - Wire both into schema and runtime data loader - Update Python builders to match schema OSMnx Integration: - Add osmnx_autark_workflow.py example showing OSMnx to Autark pipeline - Document workflow for using OSMnx with Autark specs - Successfully tested with real street network data Updates: - Fix example spec validation issues - Update test notebook with working examples - Add geometry support to JSON data source - Fix GeoTIFF required fields in schema Testing: - JSON data loads and visualizes in tables/scatterplots - OSMnx street networks successfully convert to Autark specs - Jupyter notebook integration working
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: add declarative Python API and runtime
Overview
This PR introduces a declarative Python API for Autark, following the Altair/Vega-Lite design pattern. It enables users to create urban visual analytics applications using high-level Python code that generates JSON specifications executed by a TypeScript runtime.
Core Components
Python API (
python/autark/- ~2k lines of code)Runtime System (
autk-runtime/src/- ~2k lines of code)Schema (
schema/autark-spec-v0.1.json)Key Features
Example Usage