Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ financial-market-analysis/*/final/*
financial-market-analysis/*/*.mp4
financial-market-analysis/*/*.mov
financial-market-analysis/*/*.m3u8
cinema-management/outputs/
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Built on [VideoDB](https://videodb.io), enabling agents to see, hear, and unders
| Agent | Description | Output | Demo |
|-------|-------------|--------|------|
| **[content-creator](content-creator/)** | Research any topic, produce a video briefing, then self-review using VideoDB's See + Understand to verify visual-narration alignment frame by frame | 1-2 min video briefing | [▶ Watch](https://player.videodb.io/watch?v=https://play.videodb.io/v1/a43d5463-a39d-4aac-994f-abdfb5b3bf2d.m3u8) |
| **[cinema-management](cinema-management/)** | Search an indexed film with Twelve Labs, frame selected scenes as management lessons, and assemble a narrated training video with VideoDB | 1-2 min management training video | [▶ Watch](https://console.videodb.io/player?url=https://play.videodb.io/v1/39b00ef8-9fbf-477a-b3b4-09ed69f62f6a.m3u8) |
| **[news-digest](news-digest/)** | Research any topic, gather multi-source evidence (YouTube, tweets, articles), deliver as broadcast-style video | 3-4 min video report | [▶ Watch](https://player.videodb.io/watch?v=https://play.videodb.io/v1/43570285-1d6e-4548-86e6-294201d2418f.m3u8) |
| **[financial-market-analysis](financial-market-analysis/)** | Investigate financial markets with charts, screenshots, and verified clips | Custom length market report | [▶ Watch](https://player.videodb.io/watch?v=https://play.videodb.io/v1/f37914dd-5239-4c10-aa2e-006f9095ac7c.m3u8) |

Expand Down Expand Up @@ -79,6 +80,11 @@ Create a video report about "climate summit 2026"
```
Create a market report for 2026-04-01
```

**Cinema Management:**
```
Create a management lesson from The Caine Mutiny on authority and blind obedience
```
The agent will autonomously research the topic, gather assets, and deliver the video.

---
Expand Down
6 changes: 6 additions & 0 deletions cinema-management/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
TWELVELABS_API_KEY=
VIDEO_DB_API_KEY=

FILM_TITLE=The Caine Mutiny
FILM_INDEX_ID=69c5ce575905babfd4fb0a9b
MANAGEMENT_THEME=authority, insubordination and the cost of blind obedience
158 changes: 158 additions & 0 deletions cinema-management/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Agent Instructions

## Goal

Build management training videos from existing Twelve Labs film indexes. The agent should search real indexed scenes, interpret them through a management theme, and compose a narrated stream with VideoDB.

## Required Inputs

Default demo values:

```env
FILM_TITLE=The Caine Mutiny
FILM_INDEX_ID=69c5ce575905babfd4fb0a9b
MANAGEMENT_THEME=authority, insubordination and the cost of blind obedience
```

Required secrets:

```env
TWELVELABS_API_KEY=
VIDEO_DB_API_KEY=
```

Do not commit real API keys.

## Execution Workflow

### 1. Pre-flight

Load `.env`, verify both API keys, and create `outputs/<slug>/` with:

```text
outputs/<slug>/
├── search_results.json
├── scenes.json
└── result.json
```

The slug should make parallel runs safe.

### 2. Twelve Labs Search

Use `client.search.query(...)` with:

```python
client.search.query(
index_id=index_id,
query_text=query,
search_options=["visual", "audio", "transcription"],
operator="or",
page_limit=10,
)
```

If the index rejects one or more options, retry in this order:

```python
["visual", "transcription"]
["visual", "audio"]
["visual"]
["transcription"]
```

Search with multiple management-oriented queries rather than relying on one broad prompt.

### 3. Scene Selection

Normalize hits into records with:

```json
{
"video_id": "...",
"start": 123.4,
"end": 139.2,
"duration": 15.8,
"query": "...",
"rank": 1,
"score": null,
"confidence": null,
"transcription": "..."
}
```

Clip rules:
- Add small padding around hits when possible.
- Keep each scene roughly `12-35` seconds.
- Deduplicate overlapping results from the same `video_id`.
- Keep at least `3` scenes and target `5`.
- Order final scenes chronologically.

### 4. Pegasus Analysis

Use `client.analyze(video_id=..., prompt=..., temperature=0.2, max_tokens=...)` when available. Prompt Pegasus to return a concise interpretation for the selected timestamp range:
- scene summary
- authority dynamic
- management lesson
- 2-3 sentence voiceover script

If Pegasus fails because the index lacks Pegasus support, keep the run going with a fallback script based on the theme, timestamp, and transcript.

### 5. HLS Retrieval

For each source `video_id`, retrieve the indexed asset:

```python
asset = client.indexes.indexed_assets.retrieve(index_id, video_id)
```

Use `asset.hls.video_url` when available. If the current SDK shape differs, inspect equivalent object or dict fields. If no HLS URL is available, fail with a clear message that the indexed asset must have video streaming enabled.

### 6. VideoDB Composition

Cut each selected HLS range into a short local MP4 first. The full film HLS can be slow or too large for direct VideoDB ingestion during iteration.

Connect with `videodb.connect()`, upload each local clip, generate scene narration with:

```python
coll.generate_voice(text=voiceover, voice_name="Default")
```

Assemble with VideoDB editor primitives:
- `Timeline`
- `Track`
- `Clip`
- `VideoAsset`
- `AudioAsset`
- `TextAsset`

Use a title card, scene labels, film clips, narration audio, and a closing card. Lower or mute source film audio under narration.

### 7. Output

Print:

```text
STREAM_URL=<raw-hls-url>
PLAYER_URL=https://console.videodb.io/player?url=<raw-hls-url>
```

Save the same URLs, selected scenes, scripts, and durations to `outputs/<slug>/result.json`.

## Validation

Run:

```bash
python -m py_compile cinema-management/agent.py
```

Then run the end-to-end script only when valid Twelve Labs and VideoDB keys are configured.

## Failure Handling

- Missing env vars: fail before any API calls.
- Search returns fewer than 3 usable scenes: fail and print the queries tried.
- Pegasus unavailable: warn and use fallback voiceover scripts.
- HLS missing: fail clearly; VideoDB cannot compose film clips without an accessible stream.
- VideoDB render failure: save intermediate `scenes.json` before exiting so the search work is preserved.
88 changes: 88 additions & 0 deletions cinema-management/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Cinema Management Video Agent

This agent turns an indexed film into a short management training video. It searches a Twelve Labs index for scenes related to a management theme, asks Pegasus to interpret those scenes, generates voiceover narration, and uses VideoDB to assemble a playable HLS stream.

## Demo

The included defaults build a management lesson from *The Caine Mutiny*:

```env
FILM_TITLE=The Caine Mutiny
FILM_INDEX_ID=69c5ce575905babfd4fb0a9b
MANAGEMENT_THEME=authority, insubordination and the cost of blind obedience
```

## Requirements

```bash
pip install twelvelabs videodb python-dotenv imageio-ffmpeg
```

Set credentials in your shell or in a repo-root `.env` file:

```env
TWELVELABS_API_KEY=your_twelve_labs_key
VIDEO_DB_API_KEY=your_videodb_key
```

The Twelve Labs index must already contain the film, have Marengo enabled for search, have Pegasus enabled for analysis, and expose an HLS stream for the indexed asset.

## Run

From the repo root:

```bash
python cinema-management/agent.py
```

With explicit values:

```bash
python cinema-management/agent.py \
--film-title "The Caine Mutiny" \
--index-id "69c5ce575905babfd4fb0a9b" \
--theme "authority, insubordination and the cost of blind obedience"
```

Successful output includes:

```text
STREAM_URL=https://play.videodb.io/v1/...
PLAYER_URL=https://console.videodb.io/player?url=https://play.videodb.io/v1/...
```

The run metadata is saved under:

```text
cinema-management/outputs/<slug>/result.json
```

## How It Works

1. Searches the Twelve Labs index with management-oriented queries.
2. Deduplicates overlapping scene hits and keeps a short chronological storyboard.
3. Uses Pegasus analysis to frame each scene as a management lesson.
4. Retrieves the indexed asset HLS URL from Twelve Labs.
5. Cuts the selected HLS ranges into short local MP4 clips, uploads those clips to VideoDB, generates narration, and builds a multi-track timeline.
6. Prints a playable HLS stream URL and VideoDB player URL.

## Output Files

```text
cinema-management/
├── README.md
├── SKILL.md
├── AGENTS.md
├── agent.py
└── outputs/
└── <slug>/
├── search_results.json
├── scenes.json
└── result.json
```

## Notes

- The agent does not upload or index the film itself; it expects an existing Twelve Labs index.
- If the indexed asset was not created with video streaming enabled, Twelve Labs will not return an HLS URL and the script will stop with a clear error.
- Pegasus analysis is used when available. If it fails, the agent still creates management-framed fallback narration from the search metadata.
Loading