Tagline: "From Cryptic Raw Part Rows to 252-Column Commerce-Ready Master Truth."
Challenge Track: AI-Powered Product Intelligence for Industrial Commerce (Unilog / UniHack Hackathon Challenge).
- ๐ฏ Problem Statement & Industrial Context
- ๐ง ReAct Cognitive Brain & Multi-Hop Architecture
- ๐งช Test Dataset & 252-Column Results Showcase (
test_and_result/) - ๐๏ธ 10-Agent Swarm Topology & Interactive Architecture Index
- ๐ Evidence-Aware 5-Pillar Confidence Engine
- โก Variable-Level Caching & Active Learning Loop
- ๐ฌ Cryptographic Data Bill of Materials (DBOM) & Lineage
- ๐ Parametric Search AST Compiler & Compatibility Engine
- ๐ป Complete Tech Stack
- ๐ Repository Structure
- ๐ Quickstart & Batch Evaluation
- ๐งช Automated Testing & Verification Suite
- โจ Multi-View Web Studio Workbenches
- ๐ Solution Documentation & Benchmark Artifacts
Industrial distributors and manufacturers manage millions of SKUs across technical catalogs, PDF datasheets, distributor feeds, and legacy ERP systems. Raw product data handed over by distributors is rarely e-commerce ready:
-
Cryptic & Abbreviated Strings: Short, unstructured descriptions like
"3/8 CPLG BRS 150#"or"4-1/2X.045X7/8 MTL CUT-OFF DISC". -
Missing & Dummy Entities: Crucial brand fields filled with placeholders like
"-- Unbranded --","-- No DIB Brand --", or vendor codes (APPDE,BOICA,JAMIN). -
Inconsistent Units & Formats: Non-standard units (
24invs24 in), decimals where tradespeople search fractions (0.5 invs1/2 in), and conflicting dimension order. -
Strict Compliance Governance: Industrial e-commerce buyers need exact 252-column structured delivery schemas, strict character limits (
INVOICE_DESC$\le 40$ chars ALL CAPS,MOBILE_DESC$60\text{--}80$ chars), legal brand casing with registered marks (ยฎ,โข), and 100% zero-hallucination sourcing.
OmniSpec AI is an autonomous, 10-agent LangGraph Swarm orchestrated by a central ReAct Cognitive Brain, backed by an in-memory DuckDB Knowledge Engine (27,000+ legal UniCat brands, 161,000 controlled LOVs, 63 fractional lookup tables) and Corrective RAG (CRAG) with live Web & OEM PDF Datasheet discovery. It expands a raw supplier row into a fully validated, 252-column commerce-ready delivery record with cell-level cryptographic provenance and zero hallucinations.
The ReAct Master Brain (backend/app/orchestrator/react_orchestrator.py) acts as the central cognitive orchestrator. It does not blindly query external search engines for every item; rather, it reasons iteratively over evidence:
flowchart TD
IN["๐ฅ INPUT PRODUCT<br/>(e.g., MPN='SHX78B75UC', Desc='Bosch 800 Series Dishwasher 42 dBA')"]
subgraph REASONING ["๐ง ReAct 10-Step Cognitive Loop"]
STEP1["1. Direct Knowledge Extraction<br/>(Bosch, SHX78B75UC, Dishwasher, 42 dBA)"]
STEP2["2. Missing Information Gap Detection<br/>(OEM identity, exact dimensions, LOV schema)"]
STEP3["3. In-Memory Hybrid KB Search<br/>(DuckDB Exact + RapidFuzz + BM25 + Vector)"]
DEC1{"4. Is Evidence<br/>Sufficient?"}
STEP4A["Local Extraction & Schema Binding"]
STEP4B["5. Targeted Multi-Hop Web/OEM RAG<br/>โข Hop 1: General Brand/Domain Search<br/>โข Hop 2: Authoritative OEM PDF Datasheet Crawler"]
STEP6["6. LLM Reasoning Over Grounded Evidence<br/>(Zero hallucination cross-source consensus)"]
DEC2{"7. Still Uncertain<br/>or Conflicting?"}
STEP8A["Accept & Fast-Path"]
STEP8B["8. HITL Human Review Queue<br/>(Specialist verifies & saves to DuckDB overrides)"]
STEP9["9. Deterministic Validation Audit<br/>(12 automated contract boundary rules)"]
STEP10["10. Synthesize 252-Column Delivery Record<br/>with Cell-Level DBOM Cryptographic Lineage"]
end
IN --> STEP1 --> STEP2 --> STEP3 --> DEC1
DEC1 -- "YES (conf >= 0.85)" --> STEP4A --> STEP9
DEC1 -- "NO (Unseen/Ambiguous)" --> STEP4B --> STEP6 --> DEC2
DEC2 -- "NO (Resolved)" --> STEP8A --> STEP9
DEC2 -- "YES (Uncertain)" --> STEP8B --> STEP9
STEP9 --> STEP10
The brain is equipped with 7 modular domain tools:
-
tool_kb_hybrid_retrieval: DuckDB exact + RapidFuzz + BM25 + vector search over 27K brands & overrides. -
tool_web_search_general: First-hop DuckDuckGo search for brand/category identity (consumer marketplaces blocked). -
tool_datasheet_pdf_search: Second-hop targeted crawler for OEM technical PDF datasheets (.pdf). -
tool_extract_specs_and_uoms: Deterministic numerical & dimension parser (LxWxH,V,A,W,dBA,PSI,RPM,GPM,AWG). -
tool_bind_lov_schema: Category LOV schema binder formatting up to 50 attribute triples (150 columns). -
tool_synthesize_unilog_copy: Strict character-bounded copy generator (INVOICE_DESC$\le 40$ chars ALL CAPS,MOBILE_DESC$60\text{--}80$ chars). -
tool_generate_digital_assets: Real image discovery via DuckDuckGo Image Search; fallback to standardized<CleanBrand>_<MPN>.jpgcanonical naming &.pdfdoc links. SetsActual Image (Yes/No)=Yeswhen real URL found.
OmniSpec AI comes pre-packaged with an automated test runner and verified 252-column delivery files located directly in test_and_result/:
| File | Type | Description |
|---|---|---|
test_and_result/test.csv |
Raw Input Feed | Raw input dataset containing ambiguous supplier rows with missing brands and cryptic descriptions. |
test_and_result/generate_output_for_test.py |
Execution Runner | Standalone Python pipeline runner that seeds the DuckDB Master KB, executes the 10-Agent Swarm, and generates 252-column delivery files. |
test_and_result/output.csv |
252-Col Delivery CSV | Final 252-Column delivery CSV strictly conforming to the Unilog standard (0% metadata leakage). |
test_and_result/output.xlsx |
Formatted Excel (.xlsx) | Formatted Excel delivery workbook with 50 attribute triples, 6 description tiers, and real OEM images. |
test_and_result/output.json |
Lineage JSON Export | Complete JSON export containing full attribute mappings, 5-Pillar confidence scores, and agent execution traces. |
.venv\Scripts\python.exe test_and_result\generate_output_for_test.py+-------------------------------------------------------------------------------------------------------------------------------+
| OMNISPEC AI 10-AGENT SWARM TOPOLOGY |
| |
| [Raw Catalog Row] |
| โ |
| โผ |
| โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ |
| โ AGENT 1 โ โโโโบ โ AGENT 2 โ โโโโบ โ AGENT 3 โ โโโโบ โ AGENT 4 โ โโโโบ โ AGENT 5 โ |
| โ Ingestion & โ โ Entity โ โ Taxonomy & โ โ Spec, Dim & โ โ OEM Sourcing โ |
| โ De-Noising โ โ Resolution โ โ Classificationโ โ UOM Parser โ โ & CRAG RAG โ |
| โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ |
| โ |
| โผ |
| โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ |
| โ Final Output โ โโโโ โ AGENT 10 โ โโโโ โ AGENT 9 โ โโโโ โ AGENT 8 โ โโโโ โ AGENT 7 โ โโโโ โ AGENT 6 โ
| โ (252 Columns)โ โQuality, Auditโ โReAct Attributeโ โDigital Asset โ โ Multi-Channelโ โ Constrained โ
| โ & Analytics โ โ & HITL โ โ Finalizer โ โ Synthesizer โ โ Copy Builder โ โ LOV Mapper โ
| โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ
+-------------------------------------------------------------------------------------------------------------------------------+
| # | Agent Name | Dedicated Architectural Specification | Primary Mission & Core Innovations | Key Deliverables |
|---|---|---|---|---|
| 1 | Ingestion & De-Noising | ๐ฅ Agent 1 Architecture Blueprint | Cleans raw input, strips placeholders (-- Unbranded --), tokenizes dimensions, DuckDB slang thesaurus (sawzall, romex). |
Cleaned MPN, de-noised Part_Desc, token bag, row hash. |
| 2 | Brand & Entity Resolution | ๐ท๏ธ Agent 2 Architecture Blueprint | Checks active reviewer overrides, resolves supplier names to canonical 27K UniCat brands with registered marks (ยฎ, โข), live DDGS web search. |
MANUFACTURER_NAME, BRAND_NAME, TRADE_NAME, ยฎ/โข. |
| 3 | Taxonomy & Classification | ๐ฒ Agent 3 Architecture Blueprint | Classifies SKU into 4-tier Classpath and assigns 8-digit leaf UNSPSC commodity code. |
Classpath, UNSPSC, Dept, Class, Fine, Product Name. |
| 4 | Spec, Dim & UOM Parser | ๐ Agent 4 Architecture Blueprint | Extracts physical dimensions, electrical specs, converts to 63 exact fractions (50.25 in 50-1/4 in), Master UOM single spacing. |
LENGTH, WIDTH, HEIGHT, WEIGHT, normalized UOMs, electrical specs. |
| 5 | OEM Sourcing & CRAG RAG | ๐ Agent 5 Architecture Blueprint | Discovers official OEM URLs, blocks marketplaces (0%), ingests PDF technical datasheets, Corrective RAG (CRAG). |
MFR URL, Ref URL 1..5, PDF datasheet link, certs. |
| 6 | Constrained LOV Mapper | ๐๏ธ Agent 6 Architecture Blueprint | Binds raw specs to 161,000-row UniCat LOV schema across 50 triples (150 columns). |
ATTRIBUTE_LABEL 1..50, ATTRIBUTE_VALUE 1..50, ATTRIBUTE_UOM 1..50. |
| 7 | Multi-Channel Copy Builder | โ๏ธ Agent 7 Architecture Blueprint | Generates 6 distinct formulaic descriptions: INVOICE_DESC (MOBILE_DESC ( |
INVOICE_DESC, MOBILE_DESC, SHORT_DESC, LONG_DESC1, ITEM_FEATURES_1..20. |
| 8 | Digital Asset Synthesizer | ๐ผ๏ธ Agent 8 Architecture Blueprint |
Discovers real product image URLs via DuckDuckGo Image Search (marketplace-filtered). Falls back to canonical <Brand>_<MPN>.jpg naming. Generates ReportLab 1-page engineering PDF submittals. |
Product Image (real URL), Alternate Image 1..4, Actual Image (Yes/No), Specification Sheet, SDS, RoHS. |
| 9 | ReAct Attribute Finalizer | ๐ง Agent 9 Architecture Blueprint | 5-Loop Autonomous ReAct SubGraph densely populating up to 50 verified domain triples across 8 property clusters. | Full 50-slot verified attribute triples, exact physical UOMs. |
| 10 | Quality Audit & HITL | ๐ก๏ธ Agent 10 Architecture Blueprint | Runs 12 integrity checks, computes 5-Pillar Evidence-Aware confidence, variable-level caching gate, routes to HITL. | Confidence scores ( |
Instead of arbitrary heuristic deductions, OmniSpec AI computes confidence using a transparent mathematical decomposition:
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ 5-PILLAR EVIDENCE-AWARE CONFIDENCE AUDIT โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโ
โผ โผ โผ โผ โผ
[1. Retrieval Quality][2. Evidence Auth] [3. Consistency] [4. Agreement] [5. Det. Validation]
Weight: 20% Weight: 20% Weight: 20% Weight: 20% Weight: 20%
โโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโ
โข Exact KB Match: โข Official PDF: โข >= 4 Attributes: โข Full Consensus: โข Invoice <= 40:
1.00 1.00 1.00 1.00 1.00
โข High TF-IDF: โข Verified OEM: โข 2-3 Attributes: โข Inferred: โข Mobile 60-80:
0.85 0.95 0.85 0.85 1.00
โข Fallback Search: โข Distributor: โข 1 Attribute: โข Single Source: โข Assets .jpg/.pdf:
0.50 0.80 0.70 0.60 1.00
โข Zero Match: โข Unbranded: โข 0 Specs Grounded:โข Conflicting: โข Overflow:
0.40 0.35 0.45 0.40 -0.35 each
-
Cached / Human-Approved Knowledge:
- If an SKU or variable was verified by a human specialist (in
kb_active_overrides) or exact Master KB match$\rightarrow$ Confidence = 100.0%,needs_hitl_review = False(Zero human review required).
- If an SKU or variable was verified by a human specialist (in
-
Uncached / Novel Knowledge:
- If an SKU is newly ingested, inferred, or web-discovered
$\rightarrow$ needs_hitl_review = True(Routes to human specialist for initial verification). - The specialist reviews and saves the override in DuckDB
$\rightarrow$ Every subsequent request for this SKU immediately resolves from cache with 100% confidence!
- If an SKU is newly ingested, inferred, or web-discovered
sequenceDiagram
autonumber
actor Supplier as Supplier / Feed
participant Agent as 10-Agent LangGraph Swarm
participant Audit as Agent 10 (Audit Gate)
actor Human as Human Data Specialist
participant KB as DuckDB Overrides Cache
Supplier->>Agent: Submit Unseen SKU (e.g., 'NOVEL-PUMP-99')
Agent->>Audit: Process SKU (Uncached, Confidence: 61.0%)
Audit->>Human: Route to HITL Review Modal (needs_hitl_review=True)
Human->>KB: Verify Brand ('Gorman-Ruppยฎ') & Save Override
Note over KB: Override persisted in DuckDB
Supplier->>Agent: Re-submit 'NOVEL-PUMP-99'
Agent->>KB: Exact Overrides Cache Match
KB-->>Agent: Return Verified Canonical Data
Agent->>Audit: Quality Audit Evaluation
Audit-->>Supplier: Return Enriched 252-Col Record (Confidence: 100.0%, needs_hitl_review=False)
Every single attribute populated in the 252-column record is tagged with an immutable provenance footprint:
flowchart LR
CELL["Delivery Record Cell<br/>(e.g., 'Sound Level': '42 dBA')"]
subgraph DBOM ["๐ Cryptographic Cell Lineage (DataBOM)"]
direction TB
LOC["Locator / Extraction Rule<br/>(e.g., 'REGEX: ACOUSTIC_DBA_PATTERN')"]
SOURCE["Source Type<br/>('oem_pdf_rag' / 'unicat_exact' / 'active_override')"]
CONF["Variable Confidence<br/>(1.00 / 0.95)"]
CACHE["Cache Status<br/>(is_cached: True / False)"]
AGENT["Contributing Agent<br/>('Agent 4: Spec, Dim & UOM Extractor')"]
HASH["SHA-256 Lineage Hash<br/>('e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855')"]
LOC --- SOURCE --- CONF --- CACHE --- AGENT --- HASH
end
CELL --> DBOM
OmniSpec AI features an ultra-fast Natural Language Parametric Search AST Compiler (models/parametric_search/compiler.py) and a Pairwise Product Compatibility Matrix Engine (backend/app/services/compatibility_engine.py):
flowchart TD
QUERY["User NL Query: 'Quiet Stainless Steel Dishwasher under 45 dBA with 5 wash cycles'"]
subgraph AST_COMPILER ["โก AST Constraint Compiler"]
direction TB
PARSE["Lexical AST Parser<br/>โข Sound Level: <= 45 dBA<br/>โข Finish: Stainless Steel<br/>โข Category: Dishwashers<br/>โข Number of Cycles: >= 5"]
SQL["DuckDB SQL AST Generation<br/>SELECT * FROM catalog WHERE sound_level <= 45 AND finish = 'Stainless Steel'..."]
EXPLAIN["Disqualification & Trade-off Explainer<br/>'Why did SKU X qualify/disqualify?'"]
PARSE --> SQL --> EXPLAIN
end
QUERY --> PARSE
SQL --> RESULTS["Sub-millisecond Search Results (< 2 ms)"]
| Layer | Technologies |
|---|---|
| Orchestration & Brain | LangGraph, LangChain, OpenAI GPT-4o-mini (Function Calling / Structured Outputs) |
| Knowledge Base & Search | DuckDB (Embedded OLAP), RapidFuzz (C++ Levenshtein), BM25, Scikit-Learn TF-IDF |
| Web Sourcing & CRAG | ddgs (DuckDuckGo Search), httpx, beautifulsoup4, ReportLab PDF Generator |
| Backend API | FastAPI, Pydantic V2, Uvicorn, Python 3.13 |
| Frontend Web Studio | React 18, Vite, TailwindCSS, Lucide Icons, Canvas Confetti |
| Quality & Governance | Pytest, 12-Rule Automated Integrity Engine, DBOM Lineage Engine |
UniHack/
โโโ backend/
โ โโโ app/
โ โโโ agents/ # 10 LangGraph Micro-Agents & DAG Workflow
โ โ โโโ agent_1_ingestion.py # Tokenization & noise cleansing
โ โ โโโ agent_2_entity_resolution.py# Brand resolution & live web discovery
โ โ โโโ agent_3_taxonomy.py # 4-tier category & UNSPSC classifier
โ โ โโโ agent_4_spec_uom.py # 63 fraction & Master UOM parser
โ โ โโโ agent_5_oem_sourcing.py # OEM PDF crawler & CRAG
โ โ โโโ agent_6_lov_mapper.py # 150-col EAV schema binder
โ โ โโโ agent_7_copy_builder.py # Character-bounded copy generator
โ โ โโโ agent_8_digital_assets.py # Real image URL discovery & asset namer
โ โ โโโ agent_9_quality_audit.py # 5-pillar audit & HITL routing
โ โ โโโ graph.py # LangGraph 10-Agent Swarm DAG
โ โโโ api/routes.py # REST API Endpoints
โ โโโ db/duckdb_client.py # DuckDB 27K KB & Overrides Client
โ โโโ orchestrator/ # ReAct Multi-Hop Cognitive Orchestrator & Finalizer
โ โโโ schemas/ # Pydantic State & Delivery Schemas
โ โโโ services/ # Intelligence Services (DBOM, DPI, AST, Search)
โโโ Solution/ # ๐ Master Blueprints & Agent Deep-Dives
โ โโโ AGENTS.md # Master Multi-Agent Architecture Specification
โ โโโ MASTER_ARCHITECTURE_AND_MVP_PLAN.md # Complete Architecture Roadmap & Plan
โ โโโ agents/ # 10 Dedicated Agent Architecture Documents
โโโ test_and_result/ # ๐งช Test Dataset, Runner & 252-Column Outputs
โ โโโ test.csv # Raw input catalog test feed
โ โโโ generate_output_for_test.py # Automated 10-Agent Swarm pipeline runner
โ โโโ output.csv # 252-Column delivery CSV standard output
โ โโโ output.xlsx # Formatted 252-Column Excel delivery output
โ โโโ output.json # Complete output JSON with traces & confidence
โ โโโ README.md # Test folder instructions
โโโ frontend/ # React 18 + Vite + TailwindCSS Web Studio
โโโ docs/dataset/ # UniCat Reference CSVs & LOV Dictionaries
โโโ models/parametric_search/ # AST Constraint Compiler & Benchmark Tests
โโโ tests/ # Pytest Test Suite (63 Tests, 100% Passing)
โโโ summary.md # System Evaluation & Uniqueness Report
โโโ README.md # Master Platform Documentation
# Clone repository
git clone https://github.com/SharadJhanwar/OmniSpec.git
cd OmniSpec
# Activate Python Virtual Environment
.venv\Scripts\activate # On Windows
# Install Dependencies
pip install -r requirements.txtCreate a .env file in the root directory:
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_MODEL_NAME=gpt-4o-mini
PORT=8000
HOST=0.0.0.0To run the complete 10-Agent Swarm against test_and_result/test.csv and generate the final 252-column CSV, Excel, and JSON outputs:
.venv\Scripts\python.exe test_and_result\generate_output_for_test.pyThis will automatically:
- Seed the in-memory DuckDB Knowledge Base (27K UniCat brands, 161K LOVs, 63 fractions).
- Process every row through all 10 DAG nodes (Tokenization
$\rightarrow$ Entity Resolution$\rightarrow$ Taxonomy$\rightarrow$ Specs$\rightarrow$ OEM Sourcing$\rightarrow$ LOV Mapping$\rightarrow$ 6 Copy Tiers$\rightarrow$ Real Image Discovery$\rightarrow$ ReAct Finalization$\rightarrow$ Quality Audit). - Discover real product image URLs via DuckDuckGo Image Search.
- Export the clean, sanitized 252-column delivery files to:
-
test_and_result/output.csv(100% 252-column conformant) -
test_and_result/output.xlsx(Formatted Excel delivery workbook) -
test_and_result/output.json(Full confidence scores, audit flags & agent traces)
-
# Start FastAPI Backend
uvicorn backend.app.main:app --reload --port 8000
# Start React Frontend (in another terminal)
cd frontend
npm install
npm run devOpen your browser at http://localhost:5173.
OmniSpec AI is backed by an automated test suite verifying every component:
python -m pytest tests/ -v- HITL Active Learning Lifecycle (
tests/features/test_hitl_active_learning_loop.py): Verifies unseen SKU routing to HITL, specialist override saving into DuckDB, and instant 100% resolution on re-processing. - Unseen Catalog Generalization (
tests/features/test_unseen_catalog_generalization.py): 20 unseen industrial SKUs evaluated without hardcoded rules. - Bosch Zero-Hallucination Test: Verifies
SHX78B75UCgrounds 42 dBA without hallucinating Whirlpool features. - DBOM & DPI Provenance Audits (
tests/integration/test_all_api_endpoints.py). - AST Parametric Search Stress Queries (
tests/integration/test_ast_stress_cases.py).
- Executive Summary & Benchmark Report:
summary.md - Master Multi-Agent Architecture Specification:
Solution/AGENTS.md - Detailed Master MVP Architecture Plan:
Solution/MASTER_ARCHITECTURE_AND_MVP_PLAN.md - Individual Agent Deep Dives:
Solution/agents/
This project is licensed under the MIT License. See the LICENSE file for details.
Copyright (c) 2026 Sharad Jhanwar