IntelliTab

Private, low-latency AI code completion for VS Code—running entirely on Apple Silicon. IntelliTab connects a TypeScript extension directly to a persistent Python MLX process, then combines native fill-in-the-middle prompting, adaptive context, streaming, cancellation, KV caching, speculative decoding, and dual-model routing into one local completion engine.

Why IntelliTab · Architecture · Completion lifecycle · Quick start · Configuration

Why IntelliTab

Cloud autocomplete adds a network round trip to the hottest path in the editor. IntelliTab removes that dependency completely:

VS Code → native IPC → persistent MLX process → Apple Metal

No REST server. No Ollama. No OpenAI-compatible gateway. No source code sent to a hosted model.

• True FIM completion — native Qwen Coder prefix/suffix/middle tokens for code inserted at the cursor.
• Intent-aware generation — comments, signatures, and empty bodies route to a richer multi-line policy.
• Structured context — imports and the enclosing class or function outrank an arbitrary wall of nearby lines.
• Progressive ghost text — useful tokens appear as they stream and can be accepted with Tab.
• Cancel-on-type — new keystrokes invalidate stale work before it can overwrite the editing flow.
• Persistent inference — models and tokenizers stay warm instead of restarting per completion.
• Latency-aware routing — a fast model handles short mid-line work while a quality model handles harder paths.
• Local privacy boundary — editor context stays on the developer's Mac.

The screenshot above is an authentic Development Host run. Its Output panel shows the MLX backend ready, dual-model routing active, and a sample local first-token measurement. Latency varies by model, prompt, hardware, and cache state.

Architecture

flowchart LR
    E["VS Code editor"] --> X["TypeScript extension"]
    X --> C["Adaptive context extractor"]
    C --> P["FIM / intent policy"]
    P --> IPC["Length-prefixed JSON over stdin/stdout"]
    IPC --> S["Persistent Python server"]
    S --> R{"Dual-model router"}
    R -->|"short mid-line FIM"| F["Fast 3B model"]
    R -->|"intent / multi-line"| Q["Quality model"]
    Q --> D["Optional draft model"]
    F --> M["MLX on Apple Metal"]
    Q --> M
    D --> M
    M --> K["Prefix KV cache"]
    K -->|"stream tokens"| IPC
    IPC --> G["Inline ghost text"]

Loading

Why native IPC?

The extension owns the Python child process and communicates using a compact framed protocol:

[4-byte big-endian length][UTF-8 JSON payload]

That gives IntelliTab explicit request IDs, streaming tokens, cancellation, readiness signaling, and error handling without opening a port or maintaining an HTTP stack.

Completion lifecycle

sequenceDiagram
    participant U as Developer
    participant V as VS Code extension
    participant P as Python MLX server
    participant M as Local model

    U->>V: Types in the editor
    V->>V: Debounce + extract imports/scope/window
    V->>P: complete(id, mode, before, after)
    P->>M: Route FIM or intent prompt
    M-->>P: Stream generated tokens
    P-->>V: stream(id, token)
    V-->>U: Paint progressive ghost text
    alt Developer keeps typing
        V->>P: cancel(id)
        P->>M: Cooperative cancellation
    else Developer presses Tab
        V-->>U: Accept completion
    end

Loading

Two completion policies

Path	Trigger	Context strategy	Decode policy
FIM	Mid-expression or mid-line editing	Imports + enclosing scope + tight cursor window	Fast route, up to 32 tokens, normally stops at newline
Intent	Comment, bare signature, or empty body	Imports + scope + richer surrounding window	Quality route, multi-line generation with structural early stops

The policy is decided before inference. This keeps common completions small and responsive without forcing complex comment-to-code tasks through the same token budget.

Latency engineering: phases A–E

Phase	Engineering decision
A · FIM + adaptive context	Native fill-in-the-middle prompts and high-value context selection
B · Dual policy	Separate mid-line completion from intent and multi-line generation
C · Prefix KV cache	Reuse common prompt prefixes across adjacent keystrokes
D · Speculative decoding	Let a smaller draft model propose tokens for the quality model to verify
E · Dual-model routing	Send simple FIM work to a fast model and reserve the larger model for difficult paths

What this repository demonstrates

Area	Implementation
IDE engineering	VS Code activation, inline completion provider, commands, settings, output telemetry, and lifecycle cleanup
Local ML systems	MLX inference, quantized Qwen Coder models, Apple Metal, speculative decoding, and cache reuse
Inference orchestration	Deterministic routing, streaming, cancellation, early stopping, and model fallback discovery
Context engineering	Language-aware intent detection, imports, enclosing scope, cursor windows, and prompt-size caps
Systems design	Persistent child process, framed IPC, request correlation, backpressure-safe writes, and fault isolation
Quality control	Completion cleanup, echo removal, structural filters, duplicate-prefix handling, and focused unit benchmarks
Privacy	No hosted inference path and no network service between the editor and model process

Repository map

ide-extension/
├── src/
│   ├── extension.ts            # Activation, configuration, backend lifecycle
│   ├── completion-provider.ts  # Ghost text, policies, streaming, quality filters
│   ├── context-extractor.ts    # Imports, scope, cursor context, intent detection
│   ├── backend-ipc.ts          # Child process, framed messages, cancellation
│   └── debounce.ts             # Keystroke coalescing
├── python-server/
│   ├── server.py               # Concurrent IPC loop and policy dispatch
│   ├── model.py                # MLX engines, routing, KV cache, speculative decode
│   ├── protocol.py             # Length-prefixed JSON protocol
│   └── requirements.txt
├── tests/bench_completion.py   # Focused behavior tests and local benchmarks
└── package.json                # VS Code manifest and settings contract

Run it locally

Requirements

• Apple Silicon Mac with MLX support
• Python 3.10+
• Node.js and npm
• VS Code 1.90+
• Approximately 4 GB of free memory for the default 3B model

1. Install dependencies

git clone https://github.com/captain-jack-sparrow909/IntelliTab.git
cd IntelliTab
npm install
python3 -m pip install -r python-server/requirements.txt

2. Download the default local model

python3 -c "from huggingface_hub import snapshot_download; from pathlib import Path; snapshot_download('mlx-community/Qwen2.5-Coder-3B-4bit', local_dir=str(Path.home()/'.mlx-models'/'Qwen2.5-Coder-3B-4bit'))"

3. Build and launch

npm run compile

Open the repository in VS Code and press F5. In the Extension Development Host, open a code file and start typing; ghost text can be accepted with Tab.

For the best debugging signal, open Output → MLX Code Completion. The channel reports backend readiness, selected routes, cache state, prompt size, tokens, first-token time, and total generation time.

Configuration

Search VS Code Settings for MLX Code Completion.

Setting	Default	Purpose
mlxCompletion.modelPath	auto	Quality-model directory; empty enables local discovery
mlxCompletion.debounceMs	50	Delay used to coalesce rapid keystrokes
mlxCompletion.maxTokens	32	Mid-line FIM token budget
mlxCompletion.temperature	0.0	Deterministic generation by default
mlxCompletion.contextLinesBefore	60	Upper bound before the cursor
mlxCompletion.contextLinesAfter	15	Upper bound after the cursor
mlxCompletion.speculative	true	Enables optional draft-model verification
mlxCompletion.numDraftTokens	3	Draft tokens proposed per verification step
mlxCompletion.dualModel	true	Routes simple and difficult paths separately
mlxCompletion.fastModelPath	auto	Optional fast mid-line model override

Verify and benchmark

npm run compile
npm run test:unit
npm run test:bench

test:unit checks completion cleanup, insertion composition, statement boundaries, and intent classification. test:bench exercises the installed MLX models and should be interpreted on the target machine—model size, cache state, and Apple Silicon generation all affect the result.

Current scope

• macOS on Apple Silicon
• VS Code inline completion
• Qwen2.5-Coder MLX checkpoints
• Single local developer workflow

Planned work includes JetBrains support, richer accept/reject affordances, and partial completion acceptance.

---

Built by Jabir Khan as an exploration of what IDE intelligence feels like when the model, transport, context policy, and latency budget are designed as one local system.