Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ This repository provides:
| [`faq-bot`](./faq-bot) | Auto-replies to inbound WhatsApp messages from configurable FAQ keyword/regex rules. | 0.1.4 | stable |
| [`group-translate`](./group-translate) | Auto-translates group messages between participants' languages via a LibreTranslate backend. Configure in-chat with /tr commands. Admin-gated; disabled until enabled. | 1.0.3 | stable |
| [`gsheets-logger`](./gsheets-logger) | Logs WhatsApp message events to a Google Sheet via a service account. | 0.2.2 | stable |
| [`voice-transcription`](./voice-transcription) | Transcribes inbound WhatsApp voice notes to text via an OpenAI-compatible speech-to-text backend (self-hosted Speaches/faster-whisper or hosted Groq/OpenAI) and delivers a `message.transcription` event to your webhook — so bots and AI can read and reply to audio. Off the message-delivery path; disabled until enabled. | 1.0.0 | beta |
<!-- END PLUGIN CATALOG -->

The table above is generated from each plugin's `manifest.json` + `CHANGELOG.md` by `npm run catalog`
Expand Down
1 change: 1 addition & 0 deletions group-translate/libretranslate.client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ function res(partial: { ok?: boolean; status?: number; json?: () => Promise<unkn
ok: partial.ok ?? true,
status: partial.status ?? 200,
headers: {},
body: '',
text: async () => '',
// PluginNetResponse.json is generic (<T>() => Promise<T>); a concrete fake needs the cast.
json: (partial.json ?? (async () => ({}))) as PluginNetResponse['json'],
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"build": "node package.mjs gsheets-logger",
"catalog": "node scripts/catalog.mjs",
"catalog:check": "node scripts/catalog.mjs --check",
"test": "node --import tsx --test \"gsheets-logger/*.test.ts\" \"faq-bot/*.test.ts\" \"after-hours/*.test.ts\" \"chat-flow/*.test.ts\" \"group-translate/**/*.test.ts\"",
"test": "node --import tsx --test \"gsheets-logger/*.test.ts\" \"faq-bot/*.test.ts\" \"after-hours/*.test.ts\" \"chat-flow/*.test.ts\" \"group-translate/**/*.test.ts\" \"voice-transcription/**/*.test.ts\"",
"typecheck": "tsc --noEmit"
},
"devDependencies": {
Expand Down
33 changes: 33 additions & 0 deletions plugins.json
Original file line number Diff line number Diff line change
Expand Up @@ -974,5 +974,38 @@
}
}
}
},
{
"id": "voice-transcription",
"name": "Voice Note Transcription",
"version": "1.0.0",
"type": "extension",
"status": "beta",
"description": "Transcribes inbound WhatsApp voice notes to text via an OpenAI-compatible speech-to-text backend (self-hosted Speaches/faster-whisper or hosted Groq/OpenAI) and delivers a `message.transcription` event to your webhook — so bots and AI can read and reply to audio. Off the message-delivery path; disabled until enabled.",
"author": "Yudhi Armyndharis <yudhi@rmyndharis.com>",
"license": "MIT",
"keywords": [
"transcription",
"speech-to-text",
"stt",
"whisper",
"voice",
"audio",
"whatsapp",
"openwa"
],
"minOpenWAVersion": "0.7.0",
"testedOpenWAVersion": "0.7.3",
"releasedAt": "2026-06-25",
"repoPath": "voice-transcription",
"repoUrl": "https://github.com/rmyndharis/OpenWA-plugins",
"homepage": "https://github.com/rmyndharis/OpenWA-plugins/tree/main/voice-transcription",
"download": "https://github.com/rmyndharis/OpenWA-plugins/releases/download/voice-transcription-v1.0.0/voice-transcription.zip",
"i18n": {
"es": {
"name": "Transcripción de Notas de Voz",
"description": "Transcribe las notas de voz entrantes de WhatsApp a texto mediante un backend de voz a texto compatible con OpenAI (Speaches/faster-whisper autoalojado o Groq/OpenAI) y entrega un evento message.transcription a tu webhook, para que los bots y la IA puedan leer y responder al audio. Fuera de la ruta de entrega de mensajes; desactivado hasta que se habilite."
}
}
}
]
1 change: 1 addition & 0 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"after-hours/**/*.ts",
"chat-flow/**/*.ts",
"group-translate/**/*.ts",
"voice-transcription/**/*.ts",
"types/**/*.d.ts"
]
}
22 changes: 21 additions & 1 deletion types/openwa.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,24 @@ export interface PluginEngineReadCapability {
export interface PluginNetRequestInit {
method?: string;
headers?: Record<string, string>;
body?: string;
// The sandbox bridges the request to the host via structuredClone, which preserves typed arrays —
// so a binary body (e.g. an assembled multipart/form-data upload) is sent intact. A string body is
// UTF-8 encoded by the host fetch, so binary MUST be passed as Uint8Array/Buffer, not a string.
body?: string | Uint8Array;
timeoutMs?: number;
}

export interface PluginNetResponse {
ok: boolean;
status: number;
statusText?: string;
headers: Record<string, string>;
// The actual field the sandbox runtime returns: the response body, read host-side (capped at 10 MiB)
// and handed back as a UTF-8 string. Parse JSON with `JSON.parse(res.body)`.
body: string;
// NOTE: these method forms are NOT provided by the sandbox runtime (functions cannot cross the
// worker structuredClone boundary). Use `body` above; the methods are retained only so older
// plugins still type-check. Calling them at runtime throws.
text(): Promise<string>;
json<T = unknown>(): Promise<T>;
arrayBuffer(): Promise<ArrayBuffer>;
Expand Down Expand Up @@ -190,4 +200,14 @@ export interface IncomingMessage {
senderPhone?: string | null;
mentionedIds?: string[];
contact?: { name?: string; pushName?: string };
// Inbound media, materialized by the adapter before the hook fires (both engines). `data` is base64
// and ABSENT when `omitted` is true (the payload exceeded the inbound size cap; `sizeBytes` is still
// set). For a voice note `type` is `'voice'` and `mimetype` is typically `'audio/ogg; codecs=opus'`.
media?: {
mimetype: string;
filename?: string;
data?: string;
omitted?: boolean;
sizeBytes?: number;
};
}
35 changes: 35 additions & 0 deletions voice-transcription/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Changelog

All notable changes to the Voice Note Transcription plugin are documented here. The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to
[Semantic Versioning](https://semver.org/).

## [Unreleased]

## [1.0.0] — 2026-06-25

### Added

- Initial release. Transcribes inbound WhatsApp voice notes via an OpenAI-compatible
`/v1/audio/transcriptions` backend (self-hosted Speaches/faster-whisper, or hosted Groq/OpenAI) and
delivers a `message.transcription` event to a configurable webhook — the integration channel for
bots/AI to read and reply to audio.
- Runs **off the message-delivery critical path**: the `message:received` hook returns immediately and
the STT call + delivery run as an un-awaited promise, so transcription never blocks or delays message
delivery (and is not bound by the host's 5s hook budget).
- Audio is uploaded as a binary multipart body (intact across the sandbox boundary); the part is labeled
`voice.ogg`/`audio/ogg` so OpenAI-compatible servers accept WhatsApp's OGG/Opus without transcoding.
- Guards: message-type filter (default `voice`), exact `maxSizeBytes` cost guard, best-effort per-session
hourly rate limit, and a best-effort idempotency guard that suppresses near-simultaneous engine re-fires.
- Status events: delivers `completed` (with transcript), `failed` (STT errored), or `skipped` (too large,
rate-limited, empty) — so a consumer always knows a voice note was received even when it can't be read.
- Optional **in-chat delivery** (`chatDelivery`: `off` | `self` | `reply`, default `off`) for operators who
want the transcript inside WhatsApp; `self` notes it to your own number without leaking to the sender.
Webhook delivery is optional too — the plugin can run chat-only.
- Webhook payloads are **HMAC-SHA256 signed** in `X-OpenWA-Signature` (same scheme as OpenWA core webhooks)
when a delivery secret is set, so existing verification reuses the same check.
- STT **circuit breaker**: after repeated failures the backend is skipped for a cooldown, so a degraded
provider isn't hammered.
- Fail-open throughout — any STT or delivery error is logged and skipped, never disrupting delivery.
- The delivered transcript is marked `untrusted: true` (`source: "speech-to-text"`): downstream LLM
consumers must treat it as user-role input.
157 changes: 157 additions & 0 deletions voice-transcription/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# Voice Note Transcription

> Transcribes inbound WhatsApp voice notes to text via an OpenAI-compatible speech-to-text backend and
> delivers a `message.transcription` event to your webhook — so bots and AI can read and reply to audio.

![type: extension](https://img.shields.io/badge/type-extension-blue.svg)
![license: MIT](https://img.shields.io/badge/license-MIT-green.svg)
![built for OpenWA](https://img.shields.io/badge/OpenWA-%E2%89%A5%200.7.0-25D366.svg)

## Details

<!-- BEGIN DETAILS (generated by scripts/catalog.mjs — do not edit by hand) -->
| Field | Value |
| ----- | ----- |
| **Identifier** | `voice-transcription` |
| **Version** | 1.0.0 |
| **Released** | 2026-06-25 |
| **Status** | beta |
| **Author** | Yudhi Armyndharis |
| **License** | MIT |
| **Type** | `extension` |
| **Requires OpenWA** | ≥ 0.7.0 (tested 0.7.3) |
| **Keywords** | transcription, speech-to-text, stt, whisper, voice, audio, whatsapp, openwa |
| **Repository** | [OpenWA-plugins/voice-transcription](https://github.com/rmyndharis/OpenWA-plugins/tree/main/voice-transcription) |
<!-- END DETAILS -->

## Features

- **Voice → text, out of band.** On each inbound voice note the plugin runs speech-to-text and POSTs a
`message.transcription` event to your webhook. It never touches the `message.received` payload and never
replies into the contact's chat, so the transcript reaches your bot/AI without polluting the conversation
or leaking back to the sender.
- **Never blocks delivery.** The hook returns immediately; STT runs as an un-awaited task. A slow or long
transcription cannot delay (or drop) WhatsApp message delivery, and it is not bound by OpenWA's 5-second
plugin-hook budget.
- **Bring your own STT.** Any OpenAI-compatible `/v1/audio/transcriptions` endpoint: self-hosted
[Speaches](https://github.com/speaches-ai/speaches)/faster-whisper (free, local, decodes WhatsApp
OGG/Opus natively — no transcoding), or hosted Groq / OpenAI by changing one URL.
- **Status events.** Delivers `completed` (with the transcript), `failed` (STT errored), or `skipped`
(too large / rate-limited / empty) — a consumer always knows a voice note arrived, even when it can't be read.
- **Optional in-chat delivery.** `chatDelivery` can also post the transcript into WhatsApp (`self` notes it to
your own number; `reply` quote-replies to the sender). Off by default; webhook delivery is optional too
(chat-only is supported).
- **Signed webhooks.** When a delivery secret is set, the body is HMAC-SHA256 signed in `X-OpenWA-Signature`
(same scheme as OpenWA's core webhooks), so existing verification reuses the same check.
- **Cost & abuse guards.** Exact `maxSizeBytes` skip, a best-effort per-session hourly cap, a message-type
filter (voice only by default), a best-effort idempotency guard against engine re-fires, and an STT
circuit breaker that backs off a degraded backend.
- **Fail-open & guarded HTTP.** Any error is logged and skipped. All outbound calls go through the host's
SSRF-guarded `ctx.net.fetch`; the STT and delivery hosts must be allow-listed (see **Security**). The STT
and delivery secrets are stored redacted.

## What it does

For every inbound message whose `type` is in **Message types to transcribe** (default `voice`) and that
carries inline audio, the plugin: decodes the audio, skips it if it was dropped over the inbound size cap
or exceeds `maxSizeBytes`, applies the idempotency + hourly-rate guards, calls your STT endpoint, and — if a
non-empty transcript comes back — POSTs this to your **Delivery webhook URL**:

```json
{
"event": "message.transcription",
"sessionId": "…",
"messageId": "<waMessageId>",
"chatId": "…@s.whatsapp.net",
"status": "completed",
"source": "speech-to-text",
"untrusted": true,
"transcription": { "text": "…", "language": "es", "provider": "faster-whisper", "model": "small" }
}
```

Correlate it to the original voice note by `messageId` (it arrives shortly **after** `message.received`,
out of order — do not assume ordering).

## Setup

1. **Run an STT backend.** Easiest local option — Speaches (faster-whisper), which exposes an
OpenAI-compatible API and transcribes OGG/Opus directly:
```bash
docker run -d --name speaches -p 8000:8000 ghcr.io/speaches-ai/speaches:latest-cpu
```
Preload/keep a small model warm for low latency. Or use hosted Groq (`https://api.groq.com/openai`,
model `whisper-large-v3-turbo`) / OpenAI with an API key.
2. **Allow the hosts.** This plugin ships `net.allow` for `localhost`, `127.0.0.1`, `api.groq.com:443`,
`api.openai.com:443`. For any **other** STT host or **delivery webhook** host, add `host:port` to
`net.allow` in `manifest.json` and re-package (`node package.mjs voice-transcription`).
3. **For a localhost STT/delivery target**, also set `SSRF_ALLOWED_HOSTS` on the OpenWA host (the SSRF guard
blocks loopback by default), e.g. `SSRF_ALLOWED_HOSTS=127.0.0.1,localhost`. Prefer a literal
`http://127.0.0.1:PORT` target (nothing to DNS-rebind).
4. **Stand up your delivery endpoint** (an n8n webhook, a Worker, your bot) and put its URL in
**Delivery webhook URL**.

## Install

```bash
# Upload the packaged zip
curl -X POST http://localhost:2785/plugins/install \
-H "X-API-Key: $OPENWA_API_KEY" -F "file=@voice-transcription.zip"

# Configure (per session, or '*' for all)
curl -X PUT http://localhost:2785/plugins/voice-transcription/config \
-H "X-API-Key: $OPENWA_API_KEY" -H 'Content-Type: application/json' \
-d '{"sttBaseUrl":"http://127.0.0.1:8000","model":"small","deliveryWebhookUrl":"http://127.0.0.1:5678/webhook/transcript"}'

# Enable
curl -X POST http://localhost:2785/plugins/voice-transcription/enable \
-H "X-API-Key: $OPENWA_API_KEY"
```

## Configuration

| Key | Required | Default | Description |
| --- | -------- | ------- | ----------- |
| `sttBaseUrl` | yes | — | OpenAI-compatible STT base URL (`/v1/audio/transcriptions` is appended). Host must be in `net.allow`; localhost also needs `SSRF_ALLOWED_HOSTS`. |
| `sttApiKey` | no | — | Bearer key for hosted STT (Groq/OpenAI). Stored redacted. |
| `model` | no | `small` | Whisper model name. |
| `language` | no | _(auto)_ | BCP-47 hint; blank = auto-detect. |
| `provider` | no | `faster-whisper` | Label recorded in the delivered event. |
| `timeoutMs` | no | `20000` | STT request timeout (max 30000). |
| `enabledMessageTypes` | no | `["voice"]` | Add `audio` to also transcribe non-PTT audio (more cost). |
| `maxSizeBytes` | no | `16777216` | Skip audio larger than this (exact cost guard). |
| `maxPerHour` | no | `60` | Best-effort per-session hourly transcription cap. |
| `deliveryWebhookUrl` | cond. | — | Endpoint receiving the `message.transcription` event. Host must be in `net.allow`. Optional if you only use `chatDelivery`. |
| `deliverySecret` | no | — | Optional. HMAC-SHA256 signs the body in `X-OpenWA-Signature: sha256=<hex>` (same as core webhooks). Stored redacted. |
| `deliveryTimeoutMs` | no | `5000` | Delivery POST timeout. |
| `chatDelivery` | no | `off` | Also post the transcript into WhatsApp: `off` (webhook only) · `self` (note to your own number) · `reply` (quote-reply to the sender — visible to them). |

## Compatibility

- Engine-neutral: both Baileys and whatsapp-web.js materialize the audio before the hook fires, so the
plugin works on either.
- **Best-effort by design (no core changes).** Because a sandboxed plugin has no host-managed background
queue, transcription runs as an un-awaited task in the worker turn: it is **at-most-once while the worker
is alive**, has no backpressure, and the hourly/idempotency guards are best-effort (a truly simultaneous
engine re-fire can still double-call STT). For exactly-once, structured-event delivery, a future core
`message.transcription` event would be the upgrade path.

## Security

- **Outbound HTTP is allow-listed.** Calls go through the host's SSRF-guarded `ctx.net.fetch`; only hosts in
`net.allow` are reachable, and internal/loopback IPs stay blocked unless the operator opts in via
`SSRF_ALLOWED_HOSTS`. This plugin never ships `net.allow: ["*"]` — it carries your voice audio and an API
key, so egress is pinned to the STT and delivery hosts you configure.
- **Secrets** are stored redacted. `sttApiKey` is sent as a Bearer token only to the STT host;
`deliverySecret` is **never transmitted** — it HMAC-signs the body (`X-OpenWA-Signature`).
- **Treat the transcript as untrusted.** `transcription.text` is attacker-controlled speech — the event
marks it `untrusted: true`. A downstream LLM auto-responder MUST place it in a **user** role, never a
system/trusted context (a caller can *speak* injection instructions a typist never would).

## Changelog

See [CHANGELOG.md](./CHANGELOG.md).

## License

MIT
Loading
Loading