Presentation & Soft skills evaluator

Presentation & Soft skills evaluator is a generic evaluation pipeline for recorded presentations. A user uploads a talk recording, the pipeline extracts audio, generates a timestamped transcript, captures screenshots every 20 seconds, and produces a 0-5 score using a reusable presentation rubric.

Purpose

This repository packages the infrastructure needed to assess presentation quality without embedding any private hackathon data, personal links, or vendor-specific prompts. It is intended for demo reviews, pitch practice, interview assignments, classroom presentations, and similar workflows where reviewers want a consistent scorecard.

Scope

The repository currently covers:

• local video artifact extraction with ffmpeg (or the bundled imageio-ffmpeg fallback)
• batch manifest generation for one or more recordings
• transcript generation via OpenAI-hosted transcription or a locally installed Whisper-compatible CLI
• a lightweight upload UI and polling API for single-recording evaluation
• evaluator integration for either an OpenAI model or a Wispr-hosted OpenAI-compatible endpoint
• rubric-driven JSON review output and CSV export helpers

The repository does not ship real customer data, media, transcripts, personal cloud links, or production API keys.

Repository layout

• scripts/build_submission_manifest.py: builds a generic batch manifest from a CSV file
• scripts/extract_video_artifacts.py: extracts mono audio and screenshots from a video
• scripts/process_manifest_assets.py: runs artifact extraction and transcription for a manifest
• scripts/merge_reviews_into_csv.py: merges per-recording review JSON into a CSV
• app/main.py: FastAPI entrypoint for the upload UI and evaluation API
• app/pipeline.py: orchestrates the single-upload extraction, transcription, and scoring workflow
• app/scoring.py: evaluator client for OpenAI or Wispr-compatible endpoints, with optional mock mode
• config/rubric.example.json: reusable 0-5 scoring rubric
• examples/submissions_manifest.example.csv: safe batch-input example
• examples/review_output.example.json: safe review-output example

Prerequisites

• Python 3.9+
• ffmpeg on your PATH, or allow imageio-ffmpeg to provide the fallback binary
• if you set TRANSCRIPTION_PROVIDER=whisper_cli, a Whisper-compatible CLI available as whisper
• for live hosted transcription or scoring, the provider credentials described below

Quick start

1. Clone the repository.
2. Create a virtual environment and activate it.
3. Install the Python dependencies.
4. Copy .env.example to .env.
5. Choose a configuration profile that matches how you want to run transcription and scoring.
6. Put a presentation recording somewhere outside the repository or in a gitignored local directory such as uploads/.

Example setup:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
mkdir -p uploads

If you prefer not to activate the environment, the equivalent explicit commands are:

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
cp .env.example .env
mkdir -p uploads output tmp

For the default app workflow, the web server uses:

• OpenAI audio transcription for timestamped transcript generation
• screenshots every 20 seconds
• either an OpenAI reasoning model or a Wispr-hosted OpenAI-compatible evaluator

If you want to use the older local Whisper CLI path for helper scripts, set TRANSCRIPTION_PROVIDER=whisper_cli and install openai-whisper separately.

If you need to point extraction at a non-default binary, set FFMPEG_BINARY=/absolute/path/to/ffmpeg.

Environment variables

.env.example is the public source of truth for local setup. The most common combinations are:

• MOCK_SCORING=true: skip the rubric scoring API call while keeping transcription enabled
• JUDGE_PROVIDER=openai plus OPENAI_API_KEY and OPENAI_MODEL: live evaluation with OpenAI
• JUDGE_PROVIDER=wispr plus WISPR_API_KEY, WISPR_API_BASE_URL, and WISPR_MODEL: live evaluation with a Wispr-hosted OpenAI-compatible endpoint
• TRANSCRIPTION_PROVIDER=openai plus OPENAI_API_KEY: timestamped transcription through OpenAI audio transcription
• TRANSCRIPTION_PROVIDER=whisper_cli: local transcription through the whisper CLI on your PATH

Less commonly changed but still supported:

• JUDGE_ENABLE_VISION=true to send sampled screenshots to the evaluator
• TRANSCRIPT_CHAR_LIMIT, MAX_SCREENSHOTS_FOR_JUDGE, and REQUEST_TIMEOUT_SECONDS to tune request size and runtime behavior
• UPLOADS_DIR, OUTPUT_DIR, and TMP_DIR to move local write paths outside the defaults

For a fully local smoke test with no hosted APIs, use this minimal .env and install the whisper CLI:

MOCK_SCORING=true
TRANSCRIPTION_PROVIDER=whisper_cli
WHISPER_MODEL=base

For the fastest hosted smoke test, use this minimal .env:

MOCK_SCORING=true
TRANSCRIPTION_PROVIDER=openai
OPENAI_API_KEY=replace-with-your-own-key
OPENAI_TRANSCRIPTION_MODEL=whisper-1

That keeps the real transcription path enabled while replacing rubric scoring with deterministic placeholder feedback for UI validation.

Upload UI

Run the local server:

uvicorn app.main:app --reload

Then open http://127.0.0.1:8000, upload a presentation video under 200MB, and wait for the job to move through extraction, transcription, evaluation, and result delivery.

The web app:

• validates upload size on the server
• stores uploads and generated artifacts only in gitignored directories
• polls the backend for stage updates while the multi-minute job runs
• dispatches a browser evaluation:completed event when the result arrives
• can use either OPENAI_API_KEY with an OpenAI model or WISPR_API_KEY with a Wispr-hosted OpenAI-compatible endpoint
• keeps image inputs disabled by default unless JUDGE_ENABLE_VISION=true
• can run with MOCK_SCORING=true for UI-only validation without a live judge, as long as transcription is still configured

The single-upload frontend path intentionally stays simple:

1. upload one video file
2. extract audio plus screenshots every 20 seconds
3. generate a transcript with timestamps
4. send transcript and sampled screenshots to the evaluator
5. return structured scores to the frontend

The batch scripts remain in the repo for helper workflows, but the web app does not depend on an extra audit pass.

Reproducing the local environment

From a fresh clone, this is the expected sequence:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
uvicorn app.main:app --reload

Recommended verification steps:

1. Confirm the server starts without import errors.
2. Open http://127.0.0.1:8000.
3. Upload a small sample recording under MAX_UPLOAD_MB.
4. Verify that generated files appear only under uploads/, output/, or tmp/, all of which are gitignored.

If you only want to validate the app boots, curl http://127.0.0.1:8000/api/health should return {"status":"ok"} once the server is running.

That boot-level health check does not require live evaluator credentials until you actually upload a file.

Deployment

The repository includes minimal Vercel preview scaffolding:

• api/index.py exposes the FastAPI app to Vercel
• vercel.json routes requests into the Python app
• runtime write paths automatically move to /tmp/presentation-evaluator/* when VERCEL is present

Recommended Vercel environment variables:

• JUDGE_PROVIDER
• OPENAI_API_KEY
• OPENAI_API_BASE_URL
• OPENAI_TRANSCRIPTION_MODEL
• OPENAI_MODEL
• WISPR_API_KEY
• WISPR_API_BASE_URL
• WISPR_MODEL
• JUDGE_ENABLE_VISION
• TRANSCRIPTION_PROVIDER
• MAX_UPLOAD_MB
• SCREENSHOT_INTERVAL_SEC
• SCREENSHOT_OFFSET_SEC
• TRANSCRIPT_CHAR_LIMIT
• MAX_SCREENSHOTS_FOR_JUDGE
• REQUEST_TIMEOUT_SECONDS
• UPLOADS_DIR
• OUTPUT_DIR
• TMP_DIR

Notes:

• Preview deployment is the safe default.
• The Vercel deployment is best suited for demos and smaller uploads. The local/self-hosted mode remains the more reliable option for long-running evaluations.
• No secrets belong in the repo. Configure them only in .env locally or in Vercel project env vars.

Before publishing the repository, check that .env, .env.*, .vercel/, uploads/, output/, and tmp/ are still ignored and that no deployment secrets were copied into committed examples.

Batch workflow

Create a manifest CSV following examples/submissions_manifest.example.csv, then generate the JSON manifest:

python3 scripts/build_submission_manifest.py examples/submissions_manifest.example.csv

This writes:

• output/manifest/submissions_manifest.json
• output/reviews/presentation_review_scaffold.csv

Then process the recordings:

python3 scripts/process_manifest_assets.py --manifest output/manifest/submissions_manifest.json --model base

At that point your judging service or local scorer can read the generated transcript JSON and screenshots, produce one review JSON per submission, and finally merge those into CSV:

python3 scripts/merge_reviews_into_csv.py

Each review JSON must include submission_slug so the merge step can match it back to the generated scaffold row.

Scoring flow

The default rubric scores the following criteria from 0 to 5:

• clarity
• fluidity
• ease of understanding
• visual representation

An overall presentation score is then stored alongside evidence timestamps and rationale. The rubric lives in config/rubric.example.json and can be replaced with your own version.

Privacy and secret handling

• Do not commit uploaded recordings, transcripts, screenshots, or review exports.
• Keep API keys only in local environment files or your deployment platform's secret store.
• Do not store any personal Google Drive links or presenter-identifying data in committed examples.
• Replace any sample paths, URLs, or submitter labels before publishing derived work.

Public release checklist

• Run git status --short --ignored and confirm only intentional tracked changes remain.
• Check git ls-files output for unexpected media, exports, or local config files.
• Review .env.example instead of publishing a real .env.
• Verify committed examples use placeholder names, paths, and URLs only.
• Make sure the README still matches the current env vars and startup command.

Using the repo after cloning

The repository is designed to be extended in one of two ways:

• run it as a local batch-processing utility for teams that review a set of recorded presentations
• add a server or frontend that accepts uploads, runs the same pipeline, and returns the scores

The core contract is stable: input videos go in, artifact extraction and transcript generation happen first, then a separate scoring component reads those artifacts and writes JSON review results.