Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,13 @@ cython_debug/
.idea/
.vscode/

nohup.out
nohup.out

# Local clinic runtime data
clinic_appointments.db
clinic_appointments.json
clinic_appointments.csv

# Locally downloaded Whisper models; tiny.en.pt is the small tracked fixture.
whisperflow/models/base.pt
whisperflow/models/tiny.pt
80 changes: 80 additions & 0 deletions TWILIO_SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Twilio Clinic Assistant

This project includes a Czech phone assistant for clinic appointment requests.

## Run locally

```bash
cd /Users/bobbysixkiller/vioce/whisper-flow
.venv/bin/uvicorn twilio_clinic_assistant:APP --host 127.0.0.1 --port 8000
```

## Expose to Twilio

Twilio must reach your local webhook over HTTPS. The simplest local tunnel is ngrok:

```bash
ngrok http 8000
```

Use the HTTPS URL from ngrok, for example:

```text
https://example.ngrok-free.app/voice
```

## Configure Twilio

In Twilio Console, open your phone number and set:

- Voice configuration: `A call comes in`
- Webhook method: `POST`
- Webhook URL: `https://example.ngrok-free.app/voice`
- Enable call recording for this number or Voice configuration.
- Recording status callback URL: `https://example.ngrok-free.app/recording-status`
- Recording status callback method: `POST`

The webhook returns TwiML with `<Gather input="speech dtmf" language="cs-CZ">`.
Twilio sends recognized Czech speech back as `SpeechResult`.

## Appointment output

Confirmed requests are saved to:

```text
clinic_appointments.db
```

The webhook automatically stores Twilio's `From` phone number in the
`caller_phone` column so the doctor can call the patient back. VS Code-friendly
exports are refreshed after every saved request:

```text
clinic_appointments.json
clinic_appointments.csv
```

For speech quality, the webhook uses Czech speech hints and stores confidence
scores for each collected field. If Twilio reports low confidence, the assistant
asks the patient to repeat the answer once before moving on. The exported
columns `name_confidence`, `reason_confidence`, `requested_time_confidence`, and
`raw_transcript_json` help the office spot answers that may need a callback.

Whole-call recordings are stored by Twilio. The app stores recording metadata in
the appointment export when Twilio calls `/recording-status`: `recording_sid`,
`recording_url`, `recording_status`, `recording_duration`, `recording_channels`,
`recording_track`, and `recording_start_time`. The `recording_url` points to the
audio file in Twilio; access to the file requires the Twilio account credentials.

You can inspect saved requests while the server is running:

```text
http://127.0.0.1:8000/appointments
```

## Current scope

This is an appointment intake assistant. It does not yet check a real calendar,
reserve a specific slot, send SMS confirmations, or integrate with medical
software. The next production step is to connect the confirmation step to a
calendar or scheduling backend.
21 changes: 14 additions & 7 deletions run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,16 @@ elif [ $1 = "-local" ]; then
echo "Running format, linter and tests"
rm -rf .venv

# Use python3.12 if available as it's more stable for currently pinned dependencies
PYTHON_CMD="python3"
# Use Python >=3.10 for currently pinned dependencies.
if command -v python3.12 >/dev/null 2>&1; then
PYTHON_CMD="python3.12"
python3.12 -m venv .venv
elif command -v python3.11 >/dev/null 2>&1; then
python3.11 -m venv .venv
elif command -v uv >/dev/null 2>&1; then
uv venv .venv --python 3.11 --seed
else
python3 -m venv .venv
fi

$PYTHON_CMD -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel
# Pin setuptools < 70 for openai-whisper compatibility
Expand Down Expand Up @@ -63,7 +66,11 @@ elif [ $1 = "-benchmark" ]; then
kill $(lsof -t -i:8181)
elif [ $1 = "-run-server" ]; then
echo "Running WhisperFlow server"
kill $(lsof -t -i:8181)
source .venv/bin/activate
SERVER_PID=$(lsof -t -i:8181)
if [ -n "$SERVER_PID" ]; then
kill $SERVER_PID
fi
uvicorn whisperflow.fast_server:app --host 0.0.0.0 --port 8181
elif [ $1 = "-test-package" ]; then
echo "Running WhisperFlow package setup"
Expand All @@ -88,4 +95,4 @@ else
fi

trap : 0
echo >&2 '*** DONE ***'
echo >&2 '*** DONE ***'
16 changes: 16 additions & 0 deletions tests/audio/test_audio.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,28 @@
import asyncio
import pytest
import numpy as np
import pyaudio
import whisperflow.audio.microphone as mic


def has_input_device():
"""Return whether PortAudio can see any input device."""
audio = pyaudio.PyAudio()
try:
return any(
audio.get_device_info_by_index(index).get("maxInputChannels", 0) > 0
for index in range(audio.get_device_count())
)
finally:
audio.terminate()


@pytest.mark.asyncio
async def test_capture_mic():
"""test capturing microphone"""
if not has_input_device():
pytest.skip("PortAudio has no available input device")

stop_event = asyncio.Event()
audio_chunks = queue.Queue()

Expand Down
Loading
Loading