Forty production-ready industry data models, each shipped in two flavours (ECM + MVM) — auto-generated by the Vibe Data Modeling AI agent found here under model-agent/ that turns a "user vibes" into a Unity-Catalog-ready model with schemas, foreign keys, metric views, ontology tags, and DBML diagrams. The installer can also populate every table with referentially-correct sample data on the way in.
40 industries · 80 models · 23,092 tables · 885,842 attributes · 156,641 foreign keys · 11,661 metric views
| Metric | ECM | MVM | Combined |
|---|---|---|---|
| Industries shipped | 40 | 40 | 40 / 40 |
| Models published | 40 | 40 | 80 |
| Domains | 722 | 524 | 1,246 |
| Sub-domains | 2,554 | 1,414 | 3,968 |
| Tables / data products | 16,592 | 6,500 | 23,092 |
| Attributes / columns | 615,764 | 270,078 | 885,842 |
| Foreign-key relationships | 98,709 | 57,932 | 156,641 |
| Metric views (BI-ready) | 7,307 | 4,354 | 11,661 |
| Distinct governance tags | 963 | 679 | 1,642 |
| Avg attributes / table | 37.1 | 41.6 | — |
| Avg FKs / table | 5.95 | 8.91 | — |
| Avg tables / domain | 23.0 | 12.4 | — |
MVM ≈ 39% of ECM by table count, but retains 44% of the attributes and 59% of the FK relationships — the MVM keeps the join-heavy entities and sheds low-traffic reference tables.
Install any model into Unity Catalog — catalog, schemas, tables, foreign keys, governance tags, and metric views — with the model-installer/data-model-installer.ipynb notebook. It is Databricks Serverless compatible (every operation is a plain spark.sql call).
- Import the installer. Download
model-installer/data-model-installer.ipynband import it into your Databricks workspace (or clone this repo into a Git folder so the wholemodel-installer/folder comes with it). - Pick an industry and click
Run All. Open themodelwidget, choose an industry, and run the notebook. Everything else has a sensible default — no other input required. - Watch it install live. With
session_idleft blank, the notebook launches itself as a Databricks job and prints the run URL so you can follow the install in real time. Progress is timestamped in dependency order:catalog → schemas → tables → foreign keys → tags → metric views. Any failed statements are retried automatically at the end. - Done. The model lands in the
<catalog_name>catalog (defaults to the industry name), ready to query.
An empty model is hard to demo, so the installer can fill it. Set generate_samples to Yes and pick sample_rows (5 / 10 / 20 / 50 / 100, default 10). Nothing else changes: samples are the last phase, after tables, foreign keys, tags, and metric views, and they are skipped automatically if the structural install left failures behind.
The generated data is safe to join and safe to demo on:
| Guarantee | How it is achieved |
|---|---|
| Primary keys are unique | Every table draws from its own key block; composite keys are unique as a tuple; each key value is minted in the type its column declares. |
| Every foreign key resolves | Parent keys exist before any child references them, and each child copies a real parent key (the whole tuple for composite keys). Cycles, self-references, and children keyed by their parent are ordered so no reference points at a key that does not exist yet. |
| Nothing lands half-broken | An integrity gate re-checks key uniqueness, foreign-key containment, and NOT NULL columns in memory before the first write. If it fails, no table is written. |
| Values look plausible | Column names and types drive the shape: codes come from a vocabulary, emails look like emails, decimals respect their declared precision and scale, and date pairs that name an order (created/updated, start/end) come out in that order. |
| Reruns are reproducible | A fixed seed means the same install produces the same rows. |
The structure is read back from information_schema after the install completes, so generation targets the tables, keys, and relationships Unity Catalog actually holds rather than what the model file declared. Views, metric views, and internal schemas (_metrics, _install, information_schema, default) are never populated.
An optional pass asks a Databricks Foundation Model endpoint for realistic value pools for free-text columns (names, descriptions, cities). It is time-boxed per table and never used for keys, so a slow, unavailable, or nonsensical answer costs that table some realism and nothing else.
Verified end to end on live installs, audited directly against information_schema rather than trusted from the generator:
| Install | Tables | Rows | Duplicate PKs | Foreign keys checked | Orphans |
|---|---|---|---|---|---|
restaurants MVM @ 10 rows |
87 | 870 | 0 | 506 (338 cross-domain) | 0 |
banking MVM @ 100 rows |
227 | 22,700 | 0 | 2,478 (2,002 cross-domain) | 0 |
The banking install is the harder case: 227 tables at the maximum row count, with 81% of its relationships crossing a schema boundary.
The installer shows nine widgets, in order:
| Widget | Default | Purpose |
|---|---|---|
model |
— | Industry to install (pre-loaded with all 40). Defaults to a placeholder so you choose an industry explicitly. |
model_size |
mvm |
mvm (demo-ready subset) or ecm (full coverage). The installer always resolves the latest version (v2/, v3/, …) automatically. |
catalog_name |
industry name | Base target Unity Catalog catalog. For the multi-catalog styles it also hosts the shared _metrics schema. |
cataloging_style |
One Catalog |
How schemas map to catalogs: One Catalog (everything in catalog_name), Catalog per Division (one catalog per operations/business/corporate division), or Catalog per Domain (one catalog per domain). |
catalog_prefix |
— | Optional prefix for the satellite catalogs created by the multi-catalog styles. Defaults to cat_ when a multi-catalog style is chosen and both prefix and suffix are left blank. |
catalog_suffix |
— | Optional suffix for the satellite catalogs created by the multi-catalog styles. |
local_install |
— | Optional workspace/Volume folder path. If set, the installer reads model files from there instead of fetching from this repo (useful for installing a pinned/older version offline). |
generate_samples |
No |
Yes populates every installed table with referentially-correct synthetic rows once the structure is in place. See Installing with sample data. |
sample_rows |
10 |
Rows per table when samples are on: 5, 10, 20, 50, 100. Applies to every table in the model. |
Advanced settings are not shown as widgets and use built-in defaults forwarded to the launched job automatically: 32 threads × 20-statement batches (the measured serverless optimum), metric views on, and source = this repo. session_id is job-injected — leave it blank on an interactive run and the notebook launches the install as a Databricks job that sets it automatically to run the install in-place. Sample generation adds four more: sample_seed (default 20260801), sample_llm (true), sample_llm_endpoints (comma-separated Foundation Model endpoints), and sample_threads (8).
The launched job tags itself (prefix dbx_vibe_agent_installer_) with the industry, size, version, and final install duration. The notebook's first cell documents every widget, phase, and sample-data guarantee in full.
The fastest way to explore one of these models visually is the model-viewer app, a Databricks App that renders any model.json as an interactive entity-relationship graph with three navigable views (full model, domain, single product).
- Download the installer notebook from this repo:
model-viewer/model_viewer_app_installer.ipynb. - Import the notebook into your Databricks workspace and run all cells. The installer provisions a Databricks App and prints the app URL when it finishes.
Open the app URL. You have two ways to load any model from this repo:
- Load from repo — paste
databricks-industry-solutions/lakehouse-industry-data-modelsand pick the industry + flavour from the dropdown. Note: GitHub sometimes rate-limits anonymous API calls — if you hit a 429 / "rate limit exceeded" message, fall back to the second option. - Load from JSON — navigate to the industry folder in this repo (e.g.
data-models/retail/v1/mvm/), downloadmodel.json, and click Load from JSON in the app to upload it directly.
Full-model overview — every entity in the model arranged on a single canvas, with every foreign-key relationship drawn between them. Domains are colour-coded (each rectangle is one domain) and products sit on the perimeter:
Domain drill-down — click any domain to zoom in. You see the domain's sub-domains as named groups and the products inside each, with the FK web restricted to within-domain links:
Single-product radial view — click any product (table) to centre it. The viewer fans out every other product it relates to via FK, grouped by domain, so you can see at a glance every join path leaving that table:
Every model in this repository was generated by the Vibe Modeling Agent, found here under model-agent/. Point it at a plain-English description of a business and it produces a governed, Unity-Catalog-ready model; the 80 models above are its output.
Learn more: Reimagining Data Modeling on the Lakehouse: Introducing Vibe Data Modeling and Jumpstart your Data Modeling with Databricks Industry Data Models.
- An LLM-powered modelling AI agent receives user vibes (e.g. "create 7 domains covering the arilines, ensure flight is one of them, cover airport operations too").
- It runs an 8-stage AI agent using a per-stage LLM ensemble + judge:
- Tier classification → 2. Domain generation → 3. Sub-domain expansion → 4. Product (table) generation → 5. Attribute (column) generation → 6. FK linking → 7. Semantic dedup + naming → 8. Metric view + ontology synthesis.
- Each stage is gated by structural validators (cycle detector, bidirectional-FK detector, dangling-FK detector, fidelity-precision gate) before the next stage starts.
- Output is written to a Unity Catalog volume + workspace folder, then published into this repo.
Each industry root folder ships two flavours of the same business domain:
<version>/ecm/— Expanded Coverage Model. Comprehensive, audit-grade model — the agent's source of truth. Covers every entity it can think of for the industry: operations, finance, regulatory, audit trail, reference data.<version>/mvm/— Minimum Viable Model. Production-ready, demo-friendly subset derived from the ECM. Roughly 40% of the ECM's table count, retaining the most-used entities and FK paths. Recommended starting point for new deployments.
Twenty-six industries are on v1; fourteen have a v2 produced by a later agent. The installer resolves the highest version present, so you get the newest model unless you point it at an older folder yourself.
Both flavours are byte-identical in shape — same files, same structure, same Unity-Catalog deployment story. You pick the size that fits your use-case.
<industry>/
└── v1/ # Model generation version (v2/, v3/, … land as siblings)
├── readme.md # Industry-level summary (counts, vibe, generation metadata)
├── ecm/
│ ├── readme.md # ECM-specific summary + per-domain breakdown
│ ├── model.json # Full agent model (single source of truth)
│ ├── schemas/ # Per-table DDL (CREATE TABLE / CREATE VIEW)
│ ├── metrics/ # Metric view SQL (one .sql per BI-ready metric view)
│ ├── ontology/ # Tag taxonomy + classification ontology JSON
│ ├── docs/ # Per-domain markdown docs (auto-generated)
│ ├── diagram/ # DBML + auto-rendered ER diagrams
│ └── vibes/ # next_vibes.txt — auto-generated improvement priorities for the next version
└── mvm/
└── (same structure as ecm/)
Each new model generation lands under a new vN/ sibling (v2/, v3/, …), so multiple versions of the same industry stay side-by-side and reviewable.
Sample data is not shipped as files. It is generated at install time from the physical catalog, so the rows match the keys and relationships Unity Catalog actually holds. A few older model folders still carry a samples/ directory from an earlier generation; it is not used by the installer.
Top-level helper:
data-models/models-info.csv— flat machine-readable manifest of every model, one row per industry. Theversioncolumn names the shipped version each structural count describes. The cost, duration, and token columns record the original generation run and were not re-measured for the fourteen industries that were later regenerated.
Every shipped model was validated against the agent's §9 model-level integrity contract. Findings are split out per flavour so you can see the MVMs are entirely structurally clean.
| Check | ECM (40 models) | MVM (40 models) |
|---|---|---|
| FK cycles (graph SCC) | 0 | 0 |
| Bidirectional FK pairs | 0 | 0 |
| Dangling FKs (target product missing) | 0 | 0 |
| Self-FKs on primary keys | 0 | 0 |
| Siloed tables (no FK in or out) | 15 (across 11 ECMs) | 0 |
| Cross-domain duplicate product names | 34 (across 18 ECMs) | 0 |
| Fidelity gates (Memory/JSON precision ≥ 0.85) | PASSED | PASSED |
| Per-version readme present | 40 / 40 | 40 / 40 |
All 40 MVMs ship with zero structural findings — clean across every check above. The 15 ECM silos and 34 ECM cross-domain name overlaps are the only outstanding items, all minor and called out in Known limitations. Cycles, bidirectional pairs, dangling FKs and self-FKs are zero across all 80 models.
Top-5 biggest ECMs by attribute count:
| Industry | Domains | Sub-domains | Tables | Attributes | FKs | Metric views |
|---|---|---|---|---|---|---|
| Healthcare | 22 | 80 | 542 | 22,180 | 4,002 | 285 |
| Oil & Gas | 19 | 66 | 568 | 22,088 | 3,533 | 107 |
| Sports & Entertainment | 19 | 70 | 473 | 21,075 | 4,474 | 180 |
| Transport & Shipping | 19 | 79 | 514 | 20,747 | 3,292 | 195 |
| Banking | 19 | 68 | 501 | 19,792 | 3,301 | 90 |
Top-5 biggest MVMs by attribute count:
| Industry | Domains | Sub-domains | Tables | Attributes | FKs | Metric views |
|---|---|---|---|---|---|---|
| Oil & Gas | 17 | 43 | 246 | 11,143 | 2,664 | 93 |
| Energy & Utilities | 15 | 42 | 236 | 10,384 | 2,107 | 84 |
| Banking | 17 | 46 | 227 | 9,883 | 2,478 | 80 |
| Life Insurance | 15 | 43 | 217 | 9,579 | 1,926 | 158 |
| Transport & Shipping | 14 | 44 | 210 | 9,524 | 1,918 | 122 |
Most relationship-rich (densest FK graph):
| Industry | Flavour | FKs | Tables | FKs / table |
|---|---|---|---|---|
| Real Estate | MVM | 2,410 | 177 | 13.62 |
| Pharmaceuticals | MVM | 2,423 | 213 | 11.38 |
| Agriculture | MVM | 1,995 | 177 | 11.27 |
| Banking | MVM | 2,478 | 227 | 10.92 |
| Oil & Gas | MVM | 2,664 | 246 | 10.83 |
| Telecommunication | MVM | 1,806 | 167 | 10.81 |
Most BI-ready (most metric views):
| Industry | Flavour | Metric views | Tables |
|---|---|---|---|
| Shipping Ports | ECM | 288 | 420 |
| Health Insurance | ECM | 287 | 411 |
| Travel & Hospitality | ECM | 285 | 353 |
| Healthcare | ECM | 285 | 542 |
| Automotive | ECM | 284 | 590 |
| Retail | ECM | 272 | 405 |
Deepest sub-domain hierarchy:
| Industry | Flavour | Domains | Sub-domains | Sub-domains / domain |
|---|---|---|---|---|
| Healthcare | ECM | 22 | 80 | 3.6 |
| Health Insurance | ECM | 19 | 80 | 4.2 |
| Transport & Shipping | ECM | 19 | 79 | 4.2 |
| Manufacturing | ECM | 20 | 78 | 3.9 |
| Shipping Ports | ECM | 20 | 77 | 3.9 |
| Media & Broadcasting | ECM | 17 | 75 | 4.4 |
Click an industry name to jump to its folder.
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Banking | v1 | 19 | 501 | 17 | 227 |
| Payments & Fintech | v1 | 18 | 546 | 15 | 223 |
| Health Insurance | v2 | 19 | 411 | 14 | 130 |
| Life Insurance | v1 | 19 | 468 | 15 | 217 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Healthcare | v2 | 22 | 542 | 12 | 121 |
| Pharmaceuticals | v1 | 19 | 441 | 15 | 213 |
| Genomics & Biotech | v1 | 19 | 403 | 15 | 182 |
| Clinical Trials | v1 | 19 | 379 | 13 | 193 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Airlines | v1 | 19 | 424 | 15 | 205 |
| Travel & Hospitality | v2 | 17 | 353 | 9 | 87 |
| Transport & Shipping | v1 | 19 | 514 | 14 | 210 |
| Shipping Ports | v2 | 20 | 420 | 12 | 117 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Oil & Gas | v1 | 19 | 568 | 17 | 246 |
| Energy & Utilities | v1 | 18 | 451 | 15 | 236 |
| Mining | v1 | 18 | 416 | 15 | 219 |
| Water Utilities | v2 | 15 | 377 | 10 | 103 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Education | v1 | 17 | 446 | 14 | 203 |
| NGO | v2 | 15 | 304 | 9 | 87 |
| Legal | v1 | 15 | 314 | 12 | 153 |
| Waste Management | v1 | 17 | 471 | 12 | 194 |
| Staffing & HR | v1 | 16 | 302 | 12 | 153 |
| Real Estate | v1 | 16 | 344 | 14 | 177 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Telecommunication | v1 | 20 | 451 | 15 | 167 |
| Media & Broadcasting | v2 | 17 | 425 | 12 | 132 |
| Sports & Entertainment | v1 | 19 | 473 | 14 | 200 |
| Gaming | v1 | 17 | 396 | 14 | 176 |
| Advertising | v1 | 13 | 262 | 10 | 95 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Retail | v2 | 20 | 405 | 13 | 125 |
| Grocery | v1 | 19 | 374 | 14 | 175 |
| Ecommerce | v1 | 18 | 369 | 14 | 148 |
| Consumer Goods | v2 | 18 | 405 | 10 | 116 |
| Apparel & Fashion | v1 | 19 | 400 | 12 | 163 |
| Food & Beverage | v1 | 19 | 376 | 14 | 157 |
| Restaurants | v2 | 14 | 293 | 9 | 87 |
| Industry | Version | ECM Domains | ECM Tables | MVM Domains | MVM Tables |
|---|---|---|---|---|---|
| Manufacturing | v2 | 20 | 414 | 13 | 114 |
| Chemical Manufacturing | v1 | 19 | 405 | 14 | 202 |
| Semiconductors | v2 | 19 | 386 | 12 | 113 |
| Automotive | v2 | 20 | 590 | 11 | 114 |
| Construction | v2 | 18 | 365 | 13 | 143 |
| Agriculture | v1 | 18 | 408 | 14 | 177 |
- Agent versions used: v4.5.5 (1 industry), v4.5.3 (1 industry), v4.4.7 (1 industry), v4.3.3 (2 industries), v4.3.2 (9 industries), v0.7.2 (7 industries), v0.7.1 (19 industries) — single-digit semver per
CLAUDE.md §3a. The 14 industries carrying av2/were regenerated by the 4.x agent; the rest are still on their original 0.7.x generation. - Every
model.jsoncarries a top-levelagent_versionfield so you can correlate model shape to the producing agent revision. - Every commit is one industry — search
git log --oneline | grep <industry>to find it. Diffs across versions of the same industry stay reviewable. data-models/models-info.csvis the flat machine-readable manifest of all per-model metrics.
- Metric views install cleanly. An earlier generation confused some column names in a handful of metric views, which caused occasional install errors. Every affected reference has been corrected against the physical schema — column renames, two source-schema corrections, and one nested-aggregate rewrite — with no views dropped. All 80 models now install with zero metric-view errors, validated end-to-end against live catalogs across all 40 ECMs and 40 MVMs.
- Sample data is synthetic and opt-in. The installer generates it only when
generate_samplesisYes. Keys and foreign keys are correct by construction and gated before the write, so joins and demos behave, but the values themselves are invented. Never treat them as ground truth for analytics; replace with real ingestion before going to production. - 18 ECMs carry 34 cross-domain duplicate product names (e.g.
partyowned by two domains inpayments_fintech/v1/ecm,legal_entityinbanking/v1/ecm). These are usually legitimate shared lookups, but a future agent version may consolidate each under a single owning domain. All 40 MVMs are clean of this. - 11 ECMs contain 15 siloed products between them (one or two each, e.g.
finance.ledgerineducation/v1/ecm,inventory.uominrestaurants/v2/ecm,audit.cae_charterinbanking/v1/ecm). These are legitimate top-level reference entities the agent chose not to link out from. All 40 MVMs are silo-free. - Industry coverage is broad, not deep. The ECMs aim for 70-80% of an enterprise's domain shape; the last 20-30% (organisation-specific extensions, third-party integrations) is a follow-up vibe-iteration the agent can take on.
These models are auto-generated and provided as-is for reference. Industry standards evolve; verify against your organisation's specific business rules and regulatory context before production use.
40 industries · 80 models · 23,092 tables · 885,842 attributes · zero FK cycles, zero dangling foreign keys across all 80.