Setup

npm ci

Usage

Create .env and add the required api keys:

• GOOGLE_GENERATIVE_AI_API_KEY
• DEEPGRAM_API_KEY
• OPENAI_API_KEY

You'll need a bunch of sample mp3 files for "human" and "machine". They should be 6 seconds long, mono and only capture the human/machine speech.

mkdir -p tmp/samples/human
cp $HUMAN_SAMPLES/*.mp3 tmp/samples/human
mkdir -p tmp/samples/machine
cp $MACHINE_SAMPLES/*.mp3 tmp/samples/machine

Then run the test:

STRATEGY=gemini npm test

or

STRATEGY=deepgram npm test

Sample output:

HUMAN SAMPLES:
    true_human=110 of 110
    false_machine=0 of 110 (0.00%)

MACHINE SAMPLES:
    true_machine=172 of 190
    false_human=18 of 190 (9.47%)


transcription P50: 765
transcription P95: 1106
transcription P99: 1300
inference P50: 679
inference P95: 1020
inference P99: 2318
total P50: 1395
total P95: 1916
total P99: 2727