🌾 sawit-field-crm

An end-to-end Streamlit field-CRM operational application built for palm-oil (kelapa sawit) field enumerators, purchasing teams, and agronomists.

It replaces slow, disconnected spreadsheet entries with a guided 3-step transactional workflow that writes directly into a multi-table Lark Base (Feishu Bitable) CRM relational schema.

---

🎯 The Problem Solved

Field enumerators collecting supplier, fresh fruit bunch (FFB), and plantation engagement data frequently deal with:

1. Unstructured Data & Typo Leaks: Free-text phone numbers create massive duplicate contact records in the warehouse.
2. Disconnected Records: Field staff log meetings without linking them to existing commercial Opportunities or account profiles.
3. Complex Form Abandonment: Standard spreadsheets or heavy multi-page forms lead to delayed submissions and missing mandatory operational signals.

sawit-field-crm solves this with an interactive, mobile-responsive wizard featuring:

• Automatic Phone Normalization: Validates and standardizes Indonesian telephone patterns (+62..., 08..., 8...) into canonical E.164/local formats to enforce exact-match deduplication.
• Linked Transaction Flow: Resolves or creates a Contact $\rightarrow$ optionally attaches an active commercial Opportunity $\rightarrow$ writes an Engagement Report with foreign-key links in a single atomic-like submission.
• Strict Schema Shaper: Conforms domain records directly to Lark Base REST API expectations (epoch-ms timestamps, GIS coordinate strings, foreign record IDs, multi-select array wrappers).

---

🏗️ Architecture & Linked Flow

┌─────────────────┐       ┌────────────────────────┐       ┌──────────────────────┐
│  Step 1: Phone  │ ────> │ Step 2: Contact Lookup │ ────> │ Step 3: Opportunity  │
│  Normalization  │       │ (Exact Match / Create) │       │ (Attach or Skip)     │
└─────────────────┘       └────────────────────────┘       └──────────────────────┘
                                                                       │
                                                                       ▼
                                                           ┌──────────────────────┐
                                                           │  Step 4: Engagement  │
                                                           │  (Hub / Save Record) │
                                                           └──────────────────────┘
                                                                       │
                                                                       ▼
                                                           ┌──────────────────────┐
                                                           │ Lark Base Bitable API│
                                                           │ (Linked Records)     │
                                                           └──────────────────────┘

---

🧪 Testing & Data Quality

The core business logic, schema shaping, and phone sanitization are decoupled from the UI and covered by a comprehensive 53-test pytest suite:

pytest tests/ -v

• tests/test_normalize.py: Validates canonical local Indonesian mobile normalization and rejects invalid phone formats.
• tests/test_schema.py: Verifies exact column mappings, epoch timestamp converters, geographic coordinate formats (lng,lat), and multi-record foreign key links for Bitable REST payloads.
• tests/test_flow.py: Asserts deterministic multi-step state machine transitions across form steps.

---

🚀 Quickstart

1. Clone & Setup Environment

git clone https://github.com/itw-code/sawit-field-crm.git
cd sawit-field-crm

python -m venv .venv
# On Windows:
.venv\Scripts\Activate.ps1
# On Linux/macOS:
source .venv/bin/activate

pip install -r requirements.txt

2. Configure Environment Variables

Copy .env.example to .env and fill in your Lark app credentials and Bitable Table IDs:

cp .env.example .env

3. Run Locally

streamlit run app/streamlit_app.py

---

📂 Repository Structure

├── app/
│   ├── auth.py            # Lark OAuth identity & access token management
│   ├── config.py          # Runtime configuration & environment loading
│   ├── flow.py            # 3-step transactional state machine
│   ├── lark_client.py     # Resilient Lark Bitable REST API client
│   ├── normalize.py       # Indonesian phone number canonicalization
│   ├── schema.py          # Relational Bitable schema mapping & cell shapers
│   └── streamlit_app.py   # Multi-step mobile-responsive Streamlit interface
├── docs/
│   ├── adr/               # Architectural Decision Records (ADRs)
│   └── PIVOT.md           # Engineering evaluation & architectural pivot history
├── tests/
│   ├── test_auth.py
│   ├── test_flow.py
│   ├── test_normalize.py  # Comprehensive phone parsing test cases
│   └── test_schema.py     # Bitable cell payload validation tests
└── requirements.txt

---

📄 License

MIT © Ihsan Wanda