datannurpy

datannurpy is the Python builder for datannur. It scans files and databases, extracts metadata and statistics, then generates a ready-to-use catalog bundled with the datannur app.

Key features:

• Broad format support - CSV, Excel, ODS, Parquet, Delta Lake, Iceberg, SAS, SPSS, Stata
• Geospatial - GeoJSON, Shapefile, GeoPackage, GeoParquet, GeoTIFF, GML, KML, GPX, ESRI File Geodatabase
• Database introspection - PostgreSQL, MySQL, Oracle, SQL Server, SQLite, DuckDB
• Remote and cloud storage - public HTTP(S) URLs, SFTP, S3, Azure Blob, GCS via fsspec
• Metadata extraction - Schemas, statistics, frequencies, enumerations, auto-tagging
• Incremental scans - Only rescan what changed between runs
• YAML or Python API - Declarative configuration or programmatic control

Quick start

pip install datannurpy

# catalog.yml
app_path: ./my-catalog
open_browser: true

add:
  - folder: ./data
    include: ["*.csv", "*.xlsx", "*.parquet"]

  - database: sqlite:///mydb.sqlite

python -m datannurpy catalog.yml

This command scans the configured sources, generates the catalog files, and opens the datannur app.

Create your own catalog online

No local install needed: the datannur template lets you publish your own catalog on GitHub Pages in a few minutes — point at open data URLs or drop files, and every commit rebuilds the catalog automatically. See the example catalog built from the template.

Documentation

📖 Full documentation: docs.datannur.com/builder

🗂️ datannur app: github.com/datannur/datannur

🌐 Website: datannur.com

🚀 Demo: dev.datannur.com

Contributing

For development documentation and contributing guidelines, see CONTRIBUTING.md.

License

MIT - see LICENSE. All dependencies are MIT/Apache 2.0/BSD compatible.