Skip to content

Latest commit

ย 

History

72 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

โšก OmniSpec AI โ€” Industrial Product Intelligence Platform

ReAct Brain Architecture 252 Columns Speed DuckDB DBOM

Tagline: "From Cryptic Raw Part Rows to 252-Column Commerce-Ready Master Truth."
Challenge Track: AI-Powered Product Intelligence for Industrial Commerce (Unilog / UniHack Hackathon Challenge).


๐Ÿ“‘ Quick Navigation & Documentation Index


๐ŸŽฏ Problem Statement & Industrial Context

The Core Industrial Commerce Challenge

Industrial distributors and manufacturers manage millions of SKUs across technical catalogs, PDF datasheets, distributor feeds, and legacy ERP systems. Raw product data handed over by distributors is rarely e-commerce ready:

  • Cryptic & Abbreviated Strings: Short, unstructured descriptions like "3/8 CPLG BRS 150#" or "4-1/2X.045X7/8 MTL CUT-OFF DISC".
  • Missing & Dummy Entities: Crucial brand fields filled with placeholders like "-- Unbranded --", "-- No DIB Brand --", or vendor codes (APPDE, BOICA, JAMIN).
  • Inconsistent Units & Formats: Non-standard units (24in vs 24 in), decimals where tradespeople search fractions (0.5 in vs 1/2 in), and conflicting dimension order.
  • Strict Compliance Governance: Industrial e-commerce buyers need exact 252-column structured delivery schemas, strict character limits (INVOICE_DESC $\le 40$ chars ALL CAPS, MOBILE_DESC $60\text{--}80$ chars), legal brand casing with registered marks (ยฎ, โ„ข), and 100% zero-hallucination sourcing.

OmniSpec AI's Solution

OmniSpec AI is an autonomous, 10-agent LangGraph Swarm orchestrated by a central ReAct Cognitive Brain, backed by an in-memory DuckDB Knowledge Engine (27,000+ legal UniCat brands, 161,000 controlled LOVs, 63 fractional lookup tables) and Corrective RAG (CRAG) with live Web & OEM PDF Datasheet discovery. It expands a raw supplier row into a fully validated, 252-column commerce-ready delivery record with cell-level cryptographic provenance and zero hallucinations.


๐Ÿง  ReAct Cognitive Brain & Multi-Hop Architecture

The ReAct Master Brain (backend/app/orchestrator/react_orchestrator.py) acts as the central cognitive orchestrator. It does not blindly query external search engines for every item; rather, it reasons iteratively over evidence:

flowchart TD
    IN["๐Ÿ“ฅ INPUT PRODUCT<br/>(e.g., MPN='SHX78B75UC', Desc='Bosch 800 Series Dishwasher 42 dBA')"]
    
    subgraph REASONING ["๐Ÿง  ReAct 10-Step Cognitive Loop"]
        STEP1["1. Direct Knowledge Extraction<br/>(Bosch, SHX78B75UC, Dishwasher, 42 dBA)"]
        STEP2["2. Missing Information Gap Detection<br/>(OEM identity, exact dimensions, LOV schema)"]
        STEP3["3. In-Memory Hybrid KB Search<br/>(DuckDB Exact + RapidFuzz + BM25 + Vector)"]
        DEC1{"4. Is Evidence<br/>Sufficient?"}
        
        STEP4A["Local Extraction & Schema Binding"]
        STEP4B["5. Targeted Multi-Hop Web/OEM RAG<br/>โ€ข Hop 1: General Brand/Domain Search<br/>โ€ข Hop 2: Authoritative OEM PDF Datasheet Crawler"]
        
        STEP6["6. LLM Reasoning Over Grounded Evidence<br/>(Zero hallucination cross-source consensus)"]
        DEC2{"7. Still Uncertain<br/>or Conflicting?"}
        
        STEP8A["Accept & Fast-Path"]
        STEP8B["8. HITL Human Review Queue<br/>(Specialist verifies & saves to DuckDB overrides)"]
        
        STEP9["9. Deterministic Validation Audit<br/>(12 automated contract boundary rules)"]
        STEP10["10. Synthesize 252-Column Delivery Record<br/>with Cell-Level DBOM Cryptographic Lineage"]
    end
    
    IN --> STEP1 --> STEP2 --> STEP3 --> DEC1
    DEC1 -- "YES (conf >= 0.85)" --> STEP4A --> STEP9
    DEC1 -- "NO (Unseen/Ambiguous)" --> STEP4B --> STEP6 --> DEC2
    DEC2 -- "NO (Resolved)" --> STEP8A --> STEP9
    DEC2 -- "YES (Uncertain)" --> STEP8B --> STEP9
    STEP9 --> STEP10
Loading

Dedicated ReAct Tool Registry

The brain is equipped with 7 modular domain tools:

  1. tool_kb_hybrid_retrieval: DuckDB exact + RapidFuzz + BM25 + vector search over 27K brands & overrides.
  2. tool_web_search_general: First-hop DuckDuckGo search for brand/category identity (consumer marketplaces blocked).
  3. tool_datasheet_pdf_search: Second-hop targeted crawler for OEM technical PDF datasheets (.pdf).
  4. tool_extract_specs_and_uoms: Deterministic numerical & dimension parser (LxWxH, V, A, W, dBA, PSI, RPM, GPM, AWG).
  5. tool_bind_lov_schema: Category LOV schema binder formatting up to 50 attribute triples (150 columns).
  6. tool_synthesize_unilog_copy: Strict character-bounded copy generator (INVOICE_DESC $\le 40$ chars ALL CAPS, MOBILE_DESC $60\text{--}80$ chars).
  7. tool_generate_digital_assets: Real image discovery via DuckDuckGo Image Search; fallback to standardized <CleanBrand>_<MPN>.jpg canonical naming & .pdf doc links. Sets Actual Image (Yes/No) = Yes when real URL found.

๐Ÿงช Test Dataset & 252-Column Results Showcase (test_and_result/)

OmniSpec AI comes pre-packaged with an automated test runner and verified 252-column delivery files located directly in test_and_result/:

File Type Description
test_and_result/test.csv Raw Input Feed Raw input dataset containing ambiguous supplier rows with missing brands and cryptic descriptions.
test_and_result/generate_output_for_test.py Execution Runner Standalone Python pipeline runner that seeds the DuckDB Master KB, executes the 10-Agent Swarm, and generates 252-column delivery files.
test_and_result/output.csv 252-Col Delivery CSV Final 252-Column delivery CSV strictly conforming to the Unilog standard (0% metadata leakage).
test_and_result/output.xlsx Formatted Excel (.xlsx) Formatted Excel delivery workbook with 50 attribute triples, 6 description tiers, and real OEM images.
test_and_result/output.json Lineage JSON Export Complete JSON export containing full attribute mappings, 5-Pillar confidence scores, and agent execution traces.

๐Ÿš€ 1-Command Pipeline Run:

.venv\Scripts\python.exe test_and_result\generate_output_for_test.py

๐Ÿ›๏ธ 10-Agent Swarm Topology & Interactive Architecture Index

+-------------------------------------------------------------------------------------------------------------------------------+
|                                            OMNISPEC AI 10-AGENT SWARM TOPOLOGY                                                |
|                                                                                                                               |
|  [Raw Catalog Row]                                                                                                            |
|         โ”‚                                                                                                                     |
|         โ–ผ                                                                                                                     |
|  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                     |
|  โ”‚   AGENT 1    โ”‚ โ”€โ”€โ”€โ–บ โ”‚   AGENT 2    โ”‚ โ”€โ”€โ”€โ–บ โ”‚   AGENT 3    โ”‚ โ”€โ”€โ”€โ–บ โ”‚   AGENT 4    โ”‚ โ”€โ”€โ”€โ–บ โ”‚   AGENT 5    โ”‚                     |
|  โ”‚  Ingestion & โ”‚      โ”‚    Entity    โ”‚      โ”‚  Taxonomy &  โ”‚      โ”‚ Spec, Dim &  โ”‚      โ”‚ OEM Sourcing โ”‚                     |
|  โ”‚ De-Noising   โ”‚      โ”‚  Resolution  โ”‚      โ”‚ Classificationโ”‚     โ”‚  UOM Parser  โ”‚      โ”‚  & CRAG RAG  โ”‚                     |
|  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                     |
|                                                                                                  โ”‚                            |
|                                                                                                  โ–ผ                            |
|  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                     |
|  โ”‚ Final Output โ”‚ โ—„โ”€โ”€โ”€ โ”‚   AGENT 10   โ”‚ โ—„โ”€โ”€โ”€ โ”‚   AGENT 9    โ”‚ โ—„โ”€โ”€โ”€ โ”‚   AGENT 8    โ”‚ โ—„โ”€โ”€โ”€ โ”‚   AGENT 7    โ”‚ โ—„โ”€โ”€โ”€ โ”‚   AGENT 6    โ”‚
|  โ”‚ (252 Columns)โ”‚      โ”‚Quality, Auditโ”‚      โ”‚ReAct Attributeโ”‚     โ”‚Digital Asset โ”‚      โ”‚ Multi-Channelโ”‚      โ”‚ Constrained  โ”‚
|  โ”‚ & Analytics  โ”‚      โ”‚    & HITL    โ”‚      โ”‚  Finalizer   โ”‚      โ”‚ Synthesizer  โ”‚      โ”‚ Copy Builder โ”‚      โ”‚  LOV Mapper  โ”‚
|  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+-------------------------------------------------------------------------------------------------------------------------------+

๐Ÿ“‘ Complete Agent Architecture Deep-Dive Blueprint Links

# Agent Name Dedicated Architectural Specification Primary Mission & Core Innovations Key Deliverables
1 Ingestion & De-Noising ๐Ÿ“ฅ Agent 1 Architecture Blueprint Cleans raw input, strips placeholders (-- Unbranded --), tokenizes dimensions, DuckDB slang thesaurus (sawzall, romex). Cleaned MPN, de-noised Part_Desc, token bag, row hash.
2 Brand & Entity Resolution ๐Ÿท๏ธ Agent 2 Architecture Blueprint Checks active reviewer overrides, resolves supplier names to canonical 27K UniCat brands with registered marks (ยฎ, โ„ข), live DDGS web search. MANUFACTURER_NAME, BRAND_NAME, TRADE_NAME, ยฎ/โ„ข.
3 Taxonomy & Classification ๐ŸŒฒ Agent 3 Architecture Blueprint Classifies SKU into 4-tier Classpath and assigns 8-digit leaf UNSPSC commodity code. Classpath, UNSPSC, Dept, Class, Fine, Product Name.
4 Spec, Dim & UOM Parser ๐Ÿ“ Agent 4 Architecture Blueprint Extracts physical dimensions, electrical specs, converts to 63 exact fractions (50.25 in $\rightarrow$ 50-1/4 in), Master UOM single spacing. LENGTH, WIDTH, HEIGHT, WEIGHT, normalized UOMs, electrical specs.
5 OEM Sourcing & CRAG RAG ๐ŸŒ Agent 5 Architecture Blueprint Discovers official OEM URLs, blocks marketplaces (0%), ingests PDF technical datasheets, Corrective RAG (CRAG). MFR URL, Ref URL 1..5, PDF datasheet link, certs.
6 Constrained LOV Mapper ๐Ÿ—„๏ธ Agent 6 Architecture Blueprint Binds raw specs to 161,000-row UniCat LOV schema across 50 triples (150 columns). ATTRIBUTE_LABEL 1..50, ATTRIBUTE_VALUE 1..50, ATTRIBUTE_UOM 1..50.
7 Multi-Channel Copy Builder โœ๏ธ Agent 7 Architecture Blueprint Generates 6 distinct formulaic descriptions: INVOICE_DESC ($\le 40$ ALL CAPS), MOBILE_DESC ($60\text{--}80$ chars), 20 bullet features. INVOICE_DESC, MOBILE_DESC, SHORT_DESC, LONG_DESC1, ITEM_FEATURES_1..20.
8 Digital Asset Synthesizer ๐Ÿ–ผ๏ธ Agent 8 Architecture Blueprint Discovers real product image URLs via DuckDuckGo Image Search (marketplace-filtered). Falls back to canonical <Brand>_<MPN>.jpg naming. Generates ReportLab 1-page engineering PDF submittals. Product Image (real URL), Alternate Image 1..4, Actual Image (Yes/No), Specification Sheet, SDS, RoHS.
9 ReAct Attribute Finalizer ๐Ÿง  Agent 9 Architecture Blueprint 5-Loop Autonomous ReAct SubGraph densely populating up to 50 verified domain triples across 8 property clusters. Full 50-slot verified attribute triples, exact physical UOMs.
10 Quality Audit & HITL ๐Ÿ›ก๏ธ Agent 10 Architecture Blueprint Runs 12 integrity checks, computes 5-Pillar Evidence-Aware confidence, variable-level caching gate, routes to HITL. Confidence scores ($0\text{--}100%$), audit flags, HITL review queue payload.

๐Ÿ“Š Evidence-Aware 5-Pillar Confidence Engine

Instead of arbitrary heuristic deductions, OmniSpec AI computes confidence using a transparent mathematical decomposition:

$$\text{Confidence Score} = 0.20 \cdot Q_{\text{retrieval}} + 0.20 \cdot A_{\text{authority}} + 0.20 \cdot C_{\text{consistency}} + 0.20 \cdot S_{\text{agreement}} + 0.20 \cdot V_{\text{validation}} - \text{Pen}_{\text{contradictions}} - \text{Pen}_{\text{missing}}$$

               โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
               โ”‚         5-PILLAR EVIDENCE-AWARE CONFIDENCE AUDIT       โ”‚
               โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                          โ”‚
    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
    โ–ผ                  โ–ผ                  โ–ผ                  โ–ผ                  โ–ผ
[1. Retrieval Quality][2. Evidence Auth] [3. Consistency]   [4. Agreement]     [5. Det. Validation]
   Weight: 20%          Weight: 20%        Weight: 20%        Weight: 20%        Weight: 20%
 โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€   โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
 โ€ข Exact KB Match:    โ€ข Official PDF:    โ€ข >= 4 Attributes: โ€ข Full Consensus:  โ€ข Invoice <= 40:
   1.00                 1.00               1.00               1.00               1.00
 โ€ข High TF-IDF:       โ€ข Verified OEM:    โ€ข 2-3 Attributes:  โ€ข Inferred:        โ€ข Mobile 60-80:
   0.85                 0.95               0.85               0.85               1.00
 โ€ข Fallback Search:   โ€ข Distributor:     โ€ข 1 Attribute:     โ€ข Single Source:   โ€ข Assets .jpg/.pdf:
   0.50                 0.80               0.70               0.60               1.00
 โ€ข Zero Match:        โ€ข Unbranded:       โ€ข 0 Specs Grounded:โ€ข Conflicting:     โ€ข Overflow:
   0.40                 0.35               0.45               0.40               -0.35 each

โšก Variable-Level Caching & Active Learning Loop

1. Cached vs Uncached Routing Principles

  • Cached / Human-Approved Knowledge:
    • If an SKU or variable was verified by a human specialist (in kb_active_overrides) or exact Master KB match $\rightarrow$ Confidence = 100.0%, needs_hitl_review = False (Zero human review required).
  • Uncached / Novel Knowledge:
    • If an SKU is newly ingested, inferred, or web-discovered $\rightarrow$ needs_hitl_review = True (Routes to human specialist for initial verification).
    • The specialist reviews and saves the override in DuckDB $\rightarrow$ Every subsequent request for this SKU immediately resolves from cache with 100% confidence!

2. The Closed-Loop Active Learning Lifecycle

sequenceDiagram
    autonumber
    actor Supplier as Supplier / Feed
    participant Agent as 10-Agent LangGraph Swarm
    participant Audit as Agent 10 (Audit Gate)
    actor Human as Human Data Specialist
    participant KB as DuckDB Overrides Cache

    Supplier->>Agent: Submit Unseen SKU (e.g., 'NOVEL-PUMP-99')
    Agent->>Audit: Process SKU (Uncached, Confidence: 61.0%)
    Audit->>Human: Route to HITL Review Modal (needs_hitl_review=True)
    Human->>KB: Verify Brand ('Gorman-Ruppยฎ') & Save Override
    Note over KB: Override persisted in DuckDB
    Supplier->>Agent: Re-submit 'NOVEL-PUMP-99'
    Agent->>KB: Exact Overrides Cache Match
    KB-->>Agent: Return Verified Canonical Data
    Agent->>Audit: Quality Audit Evaluation
    Audit-->>Supplier: Return Enriched 252-Col Record (Confidence: 100.0%, needs_hitl_review=False)
Loading

๐Ÿ”ฌ Cryptographic Data Bill of Materials (DBOM) & Lineage

Every single attribute populated in the 252-column record is tagged with an immutable provenance footprint:

flowchart LR
    CELL["Delivery Record Cell<br/>(e.g., 'Sound Level': '42 dBA')"]
    
    subgraph DBOM ["๐Ÿ“œ Cryptographic Cell Lineage (DataBOM)"]
        direction TB
        LOC["Locator / Extraction Rule<br/>(e.g., 'REGEX: ACOUSTIC_DBA_PATTERN')"]
        SOURCE["Source Type<br/>('oem_pdf_rag' / 'unicat_exact' / 'active_override')"]
        CONF["Variable Confidence<br/>(1.00 / 0.95)"]
        CACHE["Cache Status<br/>(is_cached: True / False)"]
        AGENT["Contributing Agent<br/>('Agent 4: Spec, Dim & UOM Extractor')"]
        HASH["SHA-256 Lineage Hash<br/>('e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855')"]
        
        LOC --- SOURCE --- CONF --- CACHE --- AGENT --- HASH
    end
    
    CELL --> DBOM
Loading

๐Ÿ” Parametric Search AST Compiler & Compatibility Engine

OmniSpec AI features an ultra-fast Natural Language Parametric Search AST Compiler (models/parametric_search/compiler.py) and a Pairwise Product Compatibility Matrix Engine (backend/app/services/compatibility_engine.py):

flowchart TD
    QUERY["User NL Query: 'Quiet Stainless Steel Dishwasher under 45 dBA with 5 wash cycles'"]
    
    subgraph AST_COMPILER ["โšก AST Constraint Compiler"]
        direction TB
        PARSE["Lexical AST Parser<br/>โ€ข Sound Level: <= 45 dBA<br/>โ€ข Finish: Stainless Steel<br/>โ€ข Category: Dishwashers<br/>โ€ข Number of Cycles: >= 5"]
        SQL["DuckDB SQL AST Generation<br/>SELECT * FROM catalog WHERE sound_level <= 45 AND finish = 'Stainless Steel'..."]
        EXPLAIN["Disqualification & Trade-off Explainer<br/>'Why did SKU X qualify/disqualify?'"]
        
        PARSE --> SQL --> EXPLAIN
    end
    
    QUERY --> PARSE
    SQL --> RESULTS["Sub-millisecond Search Results (< 2 ms)"]
Loading

๐Ÿ’ป Complete Tech Stack

Layer Technologies
Orchestration & Brain LangGraph, LangChain, OpenAI GPT-4o-mini (Function Calling / Structured Outputs)
Knowledge Base & Search DuckDB (Embedded OLAP), RapidFuzz (C++ Levenshtein), BM25, Scikit-Learn TF-IDF
Web Sourcing & CRAG ddgs (DuckDuckGo Search), httpx, beautifulsoup4, ReportLab PDF Generator
Backend API FastAPI, Pydantic V2, Uvicorn, Python 3.13
Frontend Web Studio React 18, Vite, TailwindCSS, Lucide Icons, Canvas Confetti
Quality & Governance Pytest, 12-Rule Automated Integrity Engine, DBOM Lineage Engine

๐Ÿ“ Repository Structure

UniHack/
โ”œโ”€โ”€ backend/
โ”‚   โ””โ”€โ”€ app/
โ”‚       โ”œโ”€โ”€ agents/                         # 10 LangGraph Micro-Agents & DAG Workflow
โ”‚       โ”‚   โ”œโ”€โ”€ agent_1_ingestion.py        # Tokenization & noise cleansing
โ”‚       โ”‚   โ”œโ”€โ”€ agent_2_entity_resolution.py# Brand resolution & live web discovery
โ”‚       โ”‚   โ”œโ”€โ”€ agent_3_taxonomy.py         # 4-tier category & UNSPSC classifier
โ”‚       โ”‚   โ”œโ”€โ”€ agent_4_spec_uom.py         # 63 fraction & Master UOM parser
โ”‚       โ”‚   โ”œโ”€โ”€ agent_5_oem_sourcing.py     # OEM PDF crawler & CRAG
โ”‚       โ”‚   โ”œโ”€โ”€ agent_6_lov_mapper.py       # 150-col EAV schema binder
โ”‚       โ”‚   โ”œโ”€โ”€ agent_7_copy_builder.py     # Character-bounded copy generator
โ”‚       โ”‚   โ”œโ”€โ”€ agent_8_digital_assets.py   # Real image URL discovery & asset namer
โ”‚       โ”‚   โ”œโ”€โ”€ agent_9_quality_audit.py    # 5-pillar audit & HITL routing
โ”‚       โ”‚   โ””โ”€โ”€ graph.py                    # LangGraph 10-Agent Swarm DAG
โ”‚       โ”œโ”€โ”€ api/routes.py                   # REST API Endpoints
โ”‚       โ”œโ”€โ”€ db/duckdb_client.py             # DuckDB 27K KB & Overrides Client
โ”‚       โ”œโ”€โ”€ orchestrator/                   # ReAct Multi-Hop Cognitive Orchestrator & Finalizer
โ”‚       โ”œโ”€โ”€ schemas/                        # Pydantic State & Delivery Schemas
โ”‚       โ””โ”€โ”€ services/                       # Intelligence Services (DBOM, DPI, AST, Search)
โ”œโ”€โ”€ Solution/                               # ๐Ÿ“‘ Master Blueprints & Agent Deep-Dives
โ”‚   โ”œโ”€โ”€ AGENTS.md                           # Master Multi-Agent Architecture Specification
โ”‚   โ”œโ”€โ”€ MASTER_ARCHITECTURE_AND_MVP_PLAN.md # Complete Architecture Roadmap & Plan
โ”‚   โ””โ”€โ”€ agents/                             # 10 Dedicated Agent Architecture Documents
โ”œโ”€โ”€ test_and_result/                        # ๐Ÿงช Test Dataset, Runner & 252-Column Outputs
โ”‚   โ”œโ”€โ”€ test.csv                            # Raw input catalog test feed
โ”‚   โ”œโ”€โ”€ generate_output_for_test.py         # Automated 10-Agent Swarm pipeline runner
โ”‚   โ”œโ”€โ”€ output.csv                          # 252-Column delivery CSV standard output
โ”‚   โ”œโ”€โ”€ output.xlsx                         # Formatted 252-Column Excel delivery output
โ”‚   โ”œโ”€โ”€ output.json                         # Complete output JSON with traces & confidence
โ”‚   โ””โ”€โ”€ README.md                           # Test folder instructions
โ”œโ”€โ”€ frontend/                               # React 18 + Vite + TailwindCSS Web Studio
โ”œโ”€โ”€ docs/dataset/                           # UniCat Reference CSVs & LOV Dictionaries
โ”œโ”€โ”€ models/parametric_search/               # AST Constraint Compiler & Benchmark Tests
โ”œโ”€โ”€ tests/                                  # Pytest Test Suite (63 Tests, 100% Passing)
โ”œโ”€โ”€ summary.md                              # System Evaluation & Uniqueness Report
โ””โ”€โ”€ README.md                               # Master Platform Documentation

๐Ÿš€ Quickstart & Batch Evaluation

1. Prerequisites & Installation

# Clone repository
git clone https://github.com/SharadJhanwar/OmniSpec.git
cd OmniSpec

# Activate Python Virtual Environment
.venv\Scripts\activate  # On Windows

# Install Dependencies
pip install -r requirements.txt

2. Configure Environment Variables

Create a .env file in the root directory:

OPENAI_API_KEY=your_openai_api_key_here
OPENAI_MODEL_NAME=gpt-4o-mini
PORT=8000
HOST=0.0.0.0

3. Generate 252-Column Output for Test Dataset (test.csv)

To run the complete 10-Agent Swarm against test_and_result/test.csv and generate the final 252-column CSV, Excel, and JSON outputs:

.venv\Scripts\python.exe test_and_result\generate_output_for_test.py

This will automatically:

  1. Seed the in-memory DuckDB Knowledge Base (27K UniCat brands, 161K LOVs, 63 fractions).
  2. Process every row through all 10 DAG nodes (Tokenization $\rightarrow$ Entity Resolution $\rightarrow$ Taxonomy $\rightarrow$ Specs $\rightarrow$ OEM Sourcing $\rightarrow$ LOV Mapping $\rightarrow$ 6 Copy Tiers $\rightarrow$ Real Image Discovery $\rightarrow$ ReAct Finalization $\rightarrow$ Quality Audit).
  3. Discover real product image URLs via DuckDuckGo Image Search.
  4. Export the clean, sanitized 252-column delivery files to:
    • test_and_result/output.csv (100% 252-column conformant)
    • test_and_result/output.xlsx (Formatted Excel delivery workbook)
    • test_and_result/output.json (Full confidence scores, audit flags & agent traces)

4. Launch the Web Studio

# Start FastAPI Backend
uvicorn backend.app.main:app --reload --port 8000

# Start React Frontend (in another terminal)
cd frontend
npm install
npm run dev

Open your browser at http://localhost:5173.


๐Ÿงช Automated Testing & Verification Suite

OmniSpec AI is backed by an automated test suite verifying every component:

python -m pytest tests/ -v

Test Coverage Highlights (63 / 63 Tests Passed โ€” 100%):

  • HITL Active Learning Lifecycle (tests/features/test_hitl_active_learning_loop.py): Verifies unseen SKU routing to HITL, specialist override saving into DuckDB, and instant 100% resolution on re-processing.
  • Unseen Catalog Generalization (tests/features/test_unseen_catalog_generalization.py): 20 unseen industrial SKUs evaluated without hardcoded rules.
  • Bosch Zero-Hallucination Test: Verifies SHX78B75UC grounds 42 dBA without hallucinating Whirlpool features.
  • DBOM & DPI Provenance Audits (tests/integration/test_all_api_endpoints.py).
  • AST Parametric Search Stress Queries (tests/integration/test_ast_stress_cases.py).

๐Ÿ“š Solution Documentation & Benchmark Artifacts


๐Ÿ“„ License

This project is licensed under the MIT License. See the LICENSE file for details.

Copyright (c) 2026 Sharad Jhanwar

About

Autonomous 10-Agent LangGraph Swarm transforming messy industrial feeds into 252-column master catalog truth with in-memory DuckDB (27K brands, 161K LOVs), SHA-256 DBOM cryptographic provenance, AST parametric search, and closed-loop active learning.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages