VectorTokenizer

Track 1: General-Purpose AI Agent | AMD Developer Hackathon: ACT II

VectorTokenizer is a headless, hybrid-routing AI gateway designed to minimize frontier API token spend while preserving strict SLA-grade accuracy. It replaces brittle, traditional regex-based task routing with a lightning-fast, context-aware semantic vector classifier.

Routine tasks are handled by a zero-cost, in-container model, while premium cloud tokens are spent only when mathematical or programmatic complexity strictly demands them.

---

🎯 The Thesis: Semantic Routing over Brittle Rules

Most baseline hackathon submissions attempt to lower costs by routing tasks using hardcoded keyword matching or Regex rules. This approach is highly volatile; it shatters on unseen prompt variations, causing tasks to misroute to incapable models and crash below the 80% accuracy gate.

VectorTokenizer solves this at the entry point:

1. Semantic Classification: We pre-load the all-MiniLM-L6-v2 embedding model directly into RAM. Incoming prompts are vectorized and mapped to 8 core capability domains via Cosine Similarity in under 5 milliseconds.
2. Context-Aware Decisions: Because we route by mathematical intent rather than specific string rules, the system is robust against hidden evaluation prompt variants.
3. The Escalation Ladder: The cheapest inference tier capable of handling the mapped domain accurately always wins the task.

---

🏗️ Architecture Pipeline

• Tier 1 (Classification): Zero-latency vector similarity mapping.
• Tier 2 (Zero-Cost Local): A highly compressed, 4-bit quantized Phi-3-mini (2.3GB GGUF) executes directly inside the container via llama-cpp-python. Routine workloads (Factual Lookups, Sentiment Analysis, Summarization, NER) cost 0 cloud tokens.
• Tier 3 (Cloud Escalation): Complex domains (Code Generation, Debugging, Logic) dynamically escalate to the Fireworks API proxy.

---

🚀 Quickstart: Running the Container

The fully compiled, production-ready image is hosted on the GitHub Container Registry (GHCR). All model weights are baked directly into the image layers during build-time to guarantee zero cold starts and a sub-5-second boot sequence.

1. Pull the Image

docker pull ghcr.io/abdulraheem05/vectortokenizer:latest

2. Prepare the I/O Directories

Ensure you have a local directory structure for the headless agent to read from and write to:

mkdir -p input output
# Place your evaluation tasks inside input/tasks.json

3. Execute the Run

Run the container, replacing your_fireworks_api_key with your actual token. The agent operates entirely headlessly and exits with Code 0 upon completion.

docker run --rm \
  -e FIREWORKS_API_KEY="your_fireworks_api_key" \
  -e FIREWORKS_BASE_URL="[https://api.fireworks.ai/inference/v1](https://api.fireworks.ai/inference/v1)" \
  -e ALLOWED_MODELS="accounts/fireworks/models/llama-v3p1-8b-instruct,accounts/fireworks/models/llama-v3p1-70b-instruct" \
  -v "${PWD}/input:/input" \
  -v "${PWD}/output:/output" \
  ghcr.io/abdulraheem05/vectortokenizer:latest

---

📂 Codebase Structure

The architecture is built for strict MLOps separation of concerns:

VectorTokenizer/
├── Dockerfile              # Multi-stage build baking in weights to eliminate cold starts
├── requirements.txt        # Pinned dependencies for deterministic builds
├── input/                  
│   └── tasks.json          # Input mount point for the automated harness
├── output/                 
│   └── results.json        # Output mount point for the routing results
└── src/
    ├── main.py             # Headless entry point, handles I/O parsing and graceful exits
    ├── agent.py            # Orchestrator routing to Fireworks API with network retry loops
    ├── router.py           # The Semantic Cosine-Similarity classification engine
    ├── local_model.py      # llama-cpp-python execution logic for the local Phi-3 container
    └── reference_data.py   # Vector domain mappings and capability categorizations

---

🛡️ MLOps & Engineering Guardrails

To ensure resilience against the automated judging harness, VectorTokenizer implements the following defensive programming rules:

• Dynamic Loading: Parses the ALLOWED_MODELS array at runtime; automatically filters out non-text trap models (e.g., image generators).
• Deterministic Guardrails: Temperature strictly locked at 0.0 with strict output max token caps per domain.
• Network Resilience: Built-in multi-attempt retry loops with rigid 28-second request timeouts to prevent proxy hangups.
• Never Empty: A fallback watch-dog guarantees that every single task ships a formatted JSON answer, even if the cloud API fully drops.