LeadWhisper is an agentic-AI centric CRM showcase for iPhone. The app is built around an internal Swift-native agent that can drive the CRM experience end to end: capture updates, look up local records, answer pipeline questions, draft CRM mutations, and hand every proposed change back to the user for review.
The classic Today, Contacts, Opportunities, and Settings surfaces are still available, but they are not the main idea. The intended interaction model is agent-first: after a call, meeting, or quick thought, you can speak or type what happened and let the internal agent turn that context into reviewed CRM work.
The project started as a private-first, on-device CRM companion. In practice, the current Foundation Models limits make that path too constrained for this agent showcase. The practical path now uses OpenAI for model reasoning while keeping CRM storage, local tools, and review-before-save behavior in the Swift app.
Foundation Models are exciting for on-device experiments, but they are not suitable for this LeadWhisper showcase today.
The issue is not the API shape. It is the usable context budget. A CRM agent needs instructions, tool schemas, structured output schemas, tool observations, model responses, and at least a short conversation history. With the current on-device context window, that budget is consumed quickly.
| Question | Short answer |
|---|---|
| Why keep Foundation Models at all? | They are useful for learning Apple's native model APIs, token counting, guided generation, and on-device tool calling. |
| Why are they not enough here? | The context window is too small for a richer CRM agent loop with tools, structured drafts, and continuity. |
| What happens in practice? | The agent becomes brittle after only a small amount of instruction, schema, tool output, and history. |
| What is the practical showcase path? | OpenAI gives the agent enough context and model behavior to demonstrate the Swift-native architecture. |
| What is the trade-off? | OpenAI is not private-first: user messages and local CRM lookup results are sent to OpenAI when that provider is selected. |
So the showcase moved up a layer. The interesting question is no longer "can Foundation Models run a full CRM agent on device today?" The practical answer is "not comfortably yet." The interesting question is how much of a useful agent runtime can be built cleanly in Swift around local data, local tools, provider abstraction, and human approval.
AgentView interaction
Review interaction
 Agent |
 Today |
 Contacts |
 Opportunities |
 Settings |
LeadWhisper explores what it means to implement an agent natively in Swift instead of wrapping a Python or JavaScript agent runtime.
The Agent tab is not a side feature bolted onto a conventional CRM. It is the center of the app. A user should be able to operate the CRM through the internal agent: create or update contacts, move opportunities, create notes and follow-ups, ask what is due, inspect pipeline state, and then approve or reject the generated local changes.
The surrounding tabs make the local data visible and editable, but the showcase is the agentic workflow itself: natural-language input, local tool use, structured draft generation, and human approval.
The model is not treated as a plain text generator. It acts as the decision point in an agent loop:
- choose whether to answer, ask, use tools, or propose changes;
- call read-only local CRM lookup tools;
- return a structured result;
- leave every write to explicit user review.
The app intentionally stays small. A compact CRM domain makes the hard agent problems visible:
- grounding in local data;
- tool output size;
- ambiguity;
- structured drafts;
- context pressure;
- provider boundaries;
- recovery;
- human approval.
The product boundary is simple: the model proposes, SwiftData changes only after review. Every proposed CRM mutation appears as a draft with old-to-new diffs and per-change selection before anything is saved.
- Use the internal agent as the primary interface for CRM work.
- Capture CRM updates by voice or text.
- Review AI-generated drafts before any local data changes are applied.
- Ask agentic questions about follow-ups, contacts, opportunities, and pipeline state.
- Manage contacts, companies, notes, and tags.
- Track opportunities by stage, expected start, budget, and related contact.
- Keep follow-up tasks visible in a Today view.
- Save an activity trail for important changes.
- Switch manually between Apple On-device and OpenAI as the agent provider.
- Store a user-provided OpenAI API key in Keychain for cloud-backed drafting.
- Watch selected-provider context-window usage while composing.
- Load demo data to try ambiguity handling and common CRM flows quickly.
LeadWhisper has two selectable providers:
| Provider | Role in this app | Data boundary | Main limitation |
|---|---|---|---|
| Apple On-device / Foundation Models | Original private-first experiment | Prompts and tool observations stay on the device | Too constrained for this showcase because the context window is quickly exhausted |
| OpenAI / Responses API | Practical showcase provider | User messages and local CRM lookup results are sent directly to OpenAI | Not private-first; this version has no backend proxy |
- CRM records are stored locally in SwiftData.
- Local lookup tools read contacts, opportunities, and follow-ups before drafting.
- The OpenAI API key is stored in Keychain and is never logged.
- The model never writes directly to SwiftData.
When OpenAI is selected, submitted agent messages and local CRM lookup results are sent to OpenAI so the cloud model can draft reviewable changes. There is no backend proxy in this version, so selecting OpenAI explicitly trades away the original private-first model.
The remaining safety boundary is product-level, not privacy-level: the agent proposes drafts and never applies changes without review. If the selected provider is unavailable, or OpenAI is selected without a saved key, the app explains the problem and drafts nothing.
Voice input uses Apple's Speech and AVFoundation APIs. On unsupported environments, you can type the transcript instead.
The Agent tab is the app's primary workflow surface. It is a Swift-native provider-backed agent where the model, not a scripted workflow, decides each turn whether to answer, ask one follow-up question, call a local lookup tool, or propose reviewable CRM changes.
flowchart TD
S[Settings<br/>provider + OpenAI key] --> E[AgentConversationEngine]
U[User message] --> E
E --> M[AgentContextMemory<br/>compact rolling continuity]
E --> P[AgentToolPlan<br/>LLM tool planning]
P --> R{ReAct loop}
R -->|Action| T[Read-only tools<br/>findContacts / findOpportunities / findFollowUps<br/>getContactDetails / getPipelineSummary]
T -->|Observation| R
R -->|Thought recorded| A[AgentTurn]
A -->|reply| C[Chat bubble]
A -->|clarify| Q[One question with options]
A -->|propose| V[Review card<br/>old-to-new diffs / per-change selection]
Q --> U
V -->|Cancel| U
V -->|Save, destructive changes reconfirmed| X[ChangeExecutor]
X --> D[(SwiftData)]
AgentConversationEngineowns memory, loop guards, draft validation, and review-before-save.- Before the main ReAct turn, the selected model returns an
AgentToolPlanwith the smallest safe read-only tool scope. - The model can call local tools for contacts, opportunities, follow-ups, contact details, and pipeline summary.
- Each turn returns an
AgentTurn: reply, clarification, or proposal.
- Apple runs the main turn in a
LanguageModelSessionwith planned tools attached. - OpenAI sends compact memory and tool roundtrips through the Responses API, using Structured Outputs for
AgentToolPlanandAgentTurn. - Provider clients own model-specific calls; the Swift agent harness stays separate from the selected model API.
- A Foundation Models session has a fixed context window exposed through
SystemLanguageModel.contextSize. - LeadWhisper reads that limit dynamically so the app can adapt to OS, model, or hardware changes.
- The engine uses iOS 26.4+ Foundation Models token-count APIs to measure Apple instructions, tools, prompts, transcript, and schema usage.
- OpenAI context usage is estimated locally so draft text is not sent to the network just to count tokens while the user is typing.
AgentContextMemorycarries only recent turns, open clarifications, relevant local IDs, and draft outcomes into the next turn.
- Every turn records a thought plus the action/observation sequence, following the ReAct pattern.
- The trace is visible behind a "Details" disclosure on each card, or always with the "Show Agent Reasoning" toggle in Settings.
- A per-turn lookup budget and a cap on consecutive clarification rounds keep the loop convergent.
ChangeDiffBuilderresolves targeted records and shows old-to-new diffs.ChangeExecutormutates SwiftData only after the user confirms the selected draft changes.
Building this in Swift is still much more hands-on than building a comparable server-side agent in Python or TypeScript:
| Constraint | What it means for LeadWhisper |
|---|---|
| Limited community patterns | Foundation Models is young, with fewer examples and production write-ups than cloud LLM stacks. |
| Foundation Models context pressure | The on-device privacy story is compelling, but the hard context limit makes real agent behavior fragile. |
| No full Swift agent framework | Foundation Models provides sessions, schemas, token counting, and tools, not a full runtime like LangChain Agents or the OpenAI Agents SDK. |
| More app-owned runtime code | LeadWhisper owns provider switching, tool planning, tool calls, loop guards, compact memory, overflow retry, trace display, draft validation, diffing, and approval. |
| Cloud providers change privacy | The V1 OpenAI path is bring-your-own-key and direct from the app to OpenAI. A production app would usually add a backend proxy for credentials, auth, quotas, logging, orchestration, and privacy controls. |
LeadWhisper now has the first provider boundary in place: Apple On-device remains available as the constrained original path, and OpenAI can be selected manually for practical cloud-backed drafting. The next architectural step would be to replace the BYO-key path with a production proxy that protects credentials, adds auth and quotas, and makes cloud usage auditable.
The OS 27 betas also point toward a more flexible Foundation Models ecosystem. Anthropic's Claude for Foundation Models package makes Claude available as a server-side LanguageModel provider for Apple's Foundation Models framework, and Apple documents PrivateCloudComputeLanguageModel as another Foundation Models type to watch. Both directions could strengthen the Swift-native provider interface and let LeadWhisper keep the same review-before-save harness while adding more provider choices later, but they do not automatically solve privacy, credential, backend, or auditability questions.
- ReAct: Synergizing Reasoning and Acting in Language Models
- Apple Foundation Models framework
- Generating content and performing tasks with Foundation Models
LanguageModelSessionTooland tool calling@GenerableSystemLanguageModel.contextSizePrivateCloudComputeLanguageModel- OpenAI Responses API
- OpenAI Function Calling
- OpenAI Structured Outputs
- LangChain Agents
- OpenAI Agents SDK
- Claude for Apple Foundation Models
- Swift 6, SwiftUI, and SwiftData
- Foundation Models and OpenAI Responses API
- Security / Keychain
- Speech and AVFoundation
- Xcode 26.5 or newer
- iOS 26.5 SDK or newer
- iPhone target or iPhone simulator
- Apple Intelligence-capable device for the Apple On-device provider
- OpenAI API key for the optional OpenAI provider
- Microphone and speech recognition permissions for voice input
Voice recording is intentionally unavailable in the simulator. You can type transcripts there instead. Drafting with Apple On-device requires a device with Apple Intelligence; drafting with OpenAI requires selecting OpenAI in Settings and saving an API key.
- Clone the repository.
- Open
LeadWhisper.xcodeprojin Xcode. - Select the
LeadWhisperscheme. - Choose an iPhone simulator or device.
- Build and run.
To try the app immediately, open Settings and tap Load Demo Data, then use the Agent tab or the floating talk button from the main CRM views. Apple On-device is selected by default. To use OpenAI, open Settings, switch the Agent provider to OpenAI, and save an API key in the OpenAI section.
For the optional OpenAI provider, create or copy an API key from OpenAI's API keys page. OpenAI's current guidance is the source of truth; see Production best practices: API keys.
In the app, open Settings, switch the Agent provider to OpenAI, paste the key into the OpenAI section, and save it. LeadWhisper stores the key in Keychain. Do not commit API keys to source code, logs, screenshots, fixtures, or README examples.
LeadWhisper is available under the MIT License. See LICENSE for details.