LExiA

Logistics EXception CommunIcation Agent

A self-evaluating agentic workflow for resolving problems in multi-party logistics communication. LEXiA reads and analyzes communication threads between multiple parties (previously identified as needing resolution), drafts and evaluates communications to enable a resolution, and revises them towards a quality threshold.

Built with LangGraph, Claude, Pydantic, and Streamlit.

Demo: Watch on Loom. Access to deployed version available on request.

What it does

LEXiA takes a multi-party logistics exception thread, for example, a messy email chain where a driver, a warehouse, a carrier, and a customer may all be contradicting each other, and produces a structured analysis with drafted communications ready for human review or autonomous dispatch.

The architecture is an evaluator-optimizer loop: a first-pass set of drafts is scored against a six-dimension rubric by a second model call acting as evaluator, weaknesses are identified, and the analysis is revised. This repeats until quality threshold is met or maximum iterations are reached, consistent with the Self-Refine pattern (Madaan et al., 2023).

Directory Structure

└── lexia
    ├── app.py                  # streamlit ui (iteration viewer)
    ├── data
    │   └── sample_threads.json # 12 synthetic cases
    ├── .env                    # API keys (excluded from repo)
    ├── LICENSE                 # MIT License
    ├── README.md
    ├── requirements.txt        # required installs
    └── src
        ├── config.py           # provider, model and client info
        ├── __init__.py
        ├── prompts.py          # all LLM prompts
        ├── schemas.py          # Pydantic models
        └── workflow.py         # LangGraph evaluator-optimizer graph

Installation

Requirements: Python 3.10+

Clone the repo

git clone https://github.com/sjtobin/lexia.git
cd lexia

Create and activate a virtual environment

python -m venv .venv
source .venv/bin/activate        # Linux/Mac
# .venv\Scripts\activate         # Windows

Install dependencies

pip install -r requirements.txt

Configuration

1. API Keys Create a .env file at the project root. You will need at least one key to access an LLM API.

$ cat .env
ANTHROPIC_API_KEY=your_anthropic_api_key_here
# see https://pickaxe.co/post/how-to-get-your-claude-api-key-a-step-by-step-guide
GOOGLE_API_KEY=your_gemini_api_key_here
# see https://aistudio.google.com/app/api-keys
GROQ_API_KEY=your_groq_api_key_here
# see https://console.groq.com/keys

2. Model Provider Open src/config.py and edit the provider according to your preference. Any of Anthropic, Gemini or Groq is possible and at least one of them is required:

PROVIDER = Provider.ANTHROPIC
PROVIDER = Provider.GEMINI
PROVIDER = Provider.GROQ

You can also specify a particular model of these providers if you wish, e.g.:

MODELS = {
    # Provider.ANTHROPIC: "claude-sonnet-4-6",   # higher quality
    Provider.ANTHROPIC: "claude-haiku-4-5-20251001",  # faster, cheaper
    ...
}

3. To run the project locally...

# Load environment variables
export $(cat .env | xargs)

# Start the app
.venv/bin/python -m streamlit run app.py 

Then navigate to http://localhost:8501 in a browser if the page does not automatically launch.

Sample Cases

12 synthetic cases (communication threads) are provided in data/sample_threads.json. These range from a small typo (Case 0) to repeated late delivery (Case 5) to a non-existing delivery location (Case 9) or to temperature excursion in pharmaceutical supplies (case 4). Severity is agent-assigned, ranging from LOW through MEDIUM and HIGH to CRITICAL (financial, contractual, and operational risk, compressed timeline).

LangGraph Agent Workflow

The evaluator-optimizer loop is implemented in LangGraph according the following schematic. Each node in the graph constitutes an agent (with its own prompt), whose output is passed on to the next agent for further processing.

EXTRACT --> DRAFT --> EVALUATE -(pass)-> FINALIZE
                       ^    |
                       |    v(fail)
                       REVISE

Evaluation Rubric

The evaluator agent scores response drafts according to six criteria:

1. Field completeness: Are all fields of the case analysis object (src/schemas.CaseAnalysis) filled if the information is available? Is the information accurate?

2. Conflict detection: Are all conflicts between parties identified? Are the conflict descriptions as complete as information provided in the thread allows?

3. Missing information quality: How specific and actionable is the list of missing information? Are both the needed item and the reason why it is needed clearly identified?

4. Party specificity: Are the drafted messages appropriately and specifically formulated, considering the addressee's role, the established facts relevant to them and the action requested of them?

5. Operational plausibility: How specific, realistic and actionable is the recommended action? How consistent is it with the deduced case severity?

6. Risk handling: is the decision on whether to require human review appropriate given the stakes? Is the provided reason for human review specific?

Additionally, the evaluator agent is prompted both to be systematic in applying scores (to prevent excessive variability) and to acknowledge improvement implemented in response to provided feedback (to prevent critical feedback leading to lower scores, in spite of genuine improvements to the drafts).

Limitations

• Output quality depends on the selected model. Claude Sonnet 4.6 produces noticeably better analysis of complex cases than other models. However, Haiku 4.5 provides good outputs at lower API rates.
• The evaluator is non-deterministic, which means that scores can vary on identical input. The prompt includes text aimed to minimize this variability.
• The sample cases presented here are synthetic. Real logistics threads may contain additional domain-specific terminology or communication patterns that are not presented here.

References

Anthropic. (2024). Building effective agents. https://www.anthropic.com/research/building-effective-agents

LangChain. (2024). LangGraph documentation. https://langchain-ai.github.io/langgraph/

Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., & Clark, P. (2023). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36. https://arxiv.org/abs/2303.17651

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li D., Xing, E. P., Zhang, H., Gonzalez, J.E. & Stoica, I. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems, 36, 46595-46623. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html