Blueprints, rebuilt. A drawing-first estimation workspace for Indian construction & EPC teams. Track: Autonomous Orchestration with Managed Agents
Reconstruct turns dense engineering blueprints into a costed, source-grounded Bill of Quantities (BOQ). Upload a tender PDF and it renders every sheet, builds a persisted per-sheet element index, answers element-anchored questions, rebuilds the structure as a measurable 3D model, runs a five-agent estimation pipeline, and renders the final BOQ as a downloadable PDF report — every number traceable back to the sheet it came from.
The flow: Upload PDF → navigate the canvas → select an element & ask → explore the Truth-3D model → run the 5-agent pipeline → download a costed BOQ PDF.
For EPC contractors in India, estimating tenders is slow and error-prone:
- Time: a manual BOQ takeoff from A2/A3 drawings takes ~15 days per tender.
- Accuracy: manual takeoff runs ~85% accurate; the missing 15% becomes a cost overrun once a bid is won.
- Throughput: the takeoff bottleneck caps most firms at ~40 tenders a year.
- Knowledge lock-in: reading complex reinforcement schedules lives in a few senior engineers' heads.
Reconstruct compresses that cycle and keeps every figure defensible.
A modular Vite + React frontend and a FastAPI backend over SQLite, PyMuPDF, and the OpenAI SDK.
┌────────────────┐ Upload PDF ┌────────────────────────────────┐
│ React (Vite) │ ─────────────► │ FastAPI Backend (Python 3.12) │
│ PDF Canvas │ │ │
│ Element pick │ Ask + context │ Ingestion: PyMuPDF render │
│ Agent status │ ─────────────► │ Pipeline: 5-agent DAG │
└────────────────┘ /estimate │ Persistence: SQLite │
▲ │ Deliverable: BOQ PDF (PyMuPDF) │
│ SSE agent stream └────────────────────────────────┘
└─────────────────────────────────────────┘
| Layer | Choice |
|---|---|
| Frontend | Vite 8, React 19, TypeScript, Tailwind v4 (CSS-first), Three.js |
| Backend | FastAPI, Uvicorn, Python 3.12 (managed with uv) |
| AI | OpenAI SDK — default model gpt-4.1-mini (overridable via OPENAI_MODEL) |
| Rendering | PyMuPDF (fitz) for page render, hi-res crops, and PDF report output |
| Persistence | SQLite (construct.db) |
A single model call fails at visual-geometric analysis, careful arithmetic, rate estimation, and structured output all at once. Reconstruct runs a sequential 5-agent DAG, streamed live to the UI over Server-Sent Events (SSE):
| # | Agent | Role | Output |
|---|---|---|---|
| A1 | Drawing Reader | Reads each drawing element (vision) — dimensions, rebar schedules, concrete grade. | Extraction (raw drawing data) |
| A2 | Quantity Surveyor | Worked-arithmetic reasoning for concrete volumes, steel weights, formwork areas. | TakeoffResult (quantities + auditable calc_note) |
| A3 | Rate Analyst | Estimates current INR (₹) unit rates from the model's knowledge of Indian material costs. | RatedBOQ (base ₹ rates) |
| A4 | Bid Risk Analyzer | Weighs escalation terms, location, and volatility; adds a safety buffer per rate. | RiskedBOQ (buffered rates) |
| A5 | PDF Report | Persists the run; a clean BOQ PDF is rendered on demand. | Downloadable PDF |
Honesty by design. A2's arithmetic and A3's rates come from the model's own reasoning and
knowledge — kept auditable via per-line calc_note and source strings — not a code-execution
sandbox or live web search. Indicative ₹ fallbacks keep the demo alive if a call is rate-limited.
If a dimension isn't figured on the drawing, it's flagged as missing rather than invented.
At ingest, every sheet is broken into a structured element index (footings, walls, columns,
rebar schedules, notes) — each with a bbox, kind, description, and (for schedules)
transcribed rows. Select an element and it highlights on the canvas; an Ask AI chat opens,
scoped to that element but grounded in the whole sheet.
POST /api/elements/{id}/ask(app/qna.py) → one OpenAI call assembled from persisted data (element record + sheet understanding + a hi-res crop of the element's bbox). Both turns are saved tomessages;GET /api/elements/{id}/messagesrestores the thread on reload.
A drawing set describes one project across several sheets by role — a member's plan is on one
sheet, its section on the GAD, its grade in the notes, its reinforcement in a schedule elsewhere.
After per-sheet ingest, the Structure Mapper (app/structure.py) classifies each sheet by
role, reads the quantifiable sheets, and fuses everything into a single bill of elements — one
entry per physical assembly, each traced to its source and flagged ready | missing_dims | out_of_scope. The estimate then runs at the project level, costing only ready assemblies,
so a layout/locator sheet contributes zero confabulated line items.
The element index is also rendered as an interactive 3D model of the whole structure
(app/geometry.py, app/scene.py → GET /api/documents/{id}/geometry, Three.js viewer):
- Trust contract: every solid's size comes only from a figured dimension read off a sheet;
the arrangement is inferred from the general-arrangement drawing and labelled as such. Anything
not dimensioned renders flagged
assumed— never confabulated. - Typology-aware: a detected overhead water tank gets a high-fidelity parametric model (bracing rings, animated water fill to FSL, helical stair); other structures are assembled by a layout pass with a deterministic exploded-parts fallback so the screen is never blank.
- Grade-coloured members, per-system layer toggles, a live section cut, a cinematic reveal orbit, a 1.7 m human figure for scale, and click-any-part → provenance (grade, dimensions, confidence, and a jump straight to the source sheet with the element highlighted).
The BOQ lives in the app and is exported as a downloadable PDF — no accounts, no service
credentials. GET /api/boq_runs/{id}/pdf (app/boq_pdf.py) renders a completed run with
PyMuPDF's Story/HTML layout (paginates automatically for long BOQs). No extra PDF dependency.
ReconstructAI/
├── backend/ # FastAPI backend
│ ├── app/
│ │ ├── main.py # FastAPI endpoints & routing
│ │ ├── db.py # SQLite schema, connections, migrations
│ │ ├── ingest.py # PDF render + per-sheet understanding index (OpenAI client)
│ │ ├── qna.py # Element-anchored Q&A (grounded in persisted context)
│ │ ├── structure.py # Structure Mapper: fuse sheets into the project element index
│ │ ├── geometry.py # Truth-3D: parametric solids (tank path)
│ │ ├── scene.py # Truth-3D: typology-agnostic scene builder
│ │ ├── pipeline.py # 5-agent orchestration (SSE events)
│ │ ├── boq_pdf.py # A5 PDF report (PyMuPDF Story/HTML)
│ │ ├── models.py # Pydantic schemas (agent contracts + API shapes)
│ │ └── config.py # Env + application configuration
│ ├── storage/ # Uploaded PDFs and rendered page PNGs
│ ├── construct.db # SQLite database
│ └── pyproject.toml # uv config & Python dependencies
│
├── frontend/ # Vite + React SPA (see frontend/README.md, docs/fe.md)
│ ├── src/
│ │ ├── App.tsx # Routes: "/" landing, "/app" workspace
│ │ ├── api.ts # API types + fetch/SSE helpers
│ │ ├── pages/ # Workspace (the orchestrator)
│ │ ├── components/ # CanvasViewer, ModelViewer, EstimatePanel, ElementChat, …
│ │ └── landing/ # Marketing page sections
│ └── package.json
│
└── docs/ # Design & specification material
| Method & path | Purpose |
|---|---|
GET /api/health |
LLM-enabled flag + liveness |
GET /api/project |
The single local project + documents/sheets |
POST /api/documents |
Upload a PDF (renders + persists, then processes) |
GET /api/documents/{id} |
Poll a document's status/sheets |
POST /api/elements/{id}/ask · GET /api/elements/{id}/messages |
Element-anchored Q&A |
POST /api/documents/{id}/build_structure · GET …/structure |
Structure Mapper |
GET /api/documents/{id}/geometry |
Truth-3D geometry |
POST /api/documents/{id}/estimate |
Project-level 5-agent run (SSE stream) |
GET /api/documents/{id}/latest_boq · GET /api/boq_runs/{id} |
Restore a saved run |
GET /api/boq_runs/{id}/pdf |
Download the BOQ as a PDF |
Interactive OpenAPI docs at http://localhost:8000/docs.
- Python 3.12+ (managed with
uv) - Node.js 18+ & npm
- An OpenAI API key (for understanding + estimation)
cd backend
cp .env.example .env # then fill in the key belowOPENAI_API_KEY="sk-..."
# optional:
# OPENAI_MODEL=gpt-4.1-mini # any capable OpenAI model id
# RENDER_DPI=150 # canvas-quality page PNG
# CROP_DPI=300 # hi-res crop for schedule readinguv sync
uv run uvicorn app.main:app --reload # http://localhost:8000 (docs at /docs)Without a key, the spine still works — every sheet renders, persists, and reads back. The
understanding, Structure Mapper, and estimation layers activate once OPENAI_API_KEY is set.
cd frontend
npm install
npm run dev # http://localhost:5173Vite proxies /api and /storage to the backend at :8000, so the app uses relative URLs. Open
http://localhost:5173 to upload drawings, ask questions, explore the 3D model, and run estimates.
- Real blueprints: tested on Maharashtra PWD RCC drawings, GAD bridge drawings, and overhead water-tank schedules.
- Grounding: dimensions come only from figured values on the drawing; missing ones are flagged, never invented.
- Truth-3D: the set is rebuilt as an interactive, measurable model — every solid sized from figured dimensions, colour-coded by grade, with click-through provenance to the source sheet.
- Persistence: SQLite keeps uploaded blueprints, understanding, Q&A threads, and BOQ runs across restarts.
- Deliverable: a completed run downloads as a clean, paginated BOQ PDF.