I build the agentic-AI and VFX systems that put generative film into production.
I'm an AI Engineer at Collective Artists Network (Galleri5), building agentic-AI pipelines for film and OTT. My work runs in production on Mahabharat (JioHotstar) and the upcoming features Hanuman and Krishna — multi-agent systems on Azure AI Foundry, foundation-model fine-tuning, and the VFX tooling that artists use daily.
The through-line in everything below: a generative model on its own is a demo. What makes it production is the scaffolding around it — verification loops, human checkpoints, cost telemetry, and licence provenance.
colin = {
"role": "AI Engineer @ Galleri5 (Collective Artists Network)",
"building": ["agentic multi-agent systems", "VFX matting tooling",
"generative-video pipelines", "foundation-model fine-tuning"],
"stack": ["Azure AI Foundry", "PyTorch", "FastAPI", "SAM 2/3", "ViTMatte",
"Flux", "SDXL", "ComfyUI", "Blender API"],
"shipped": "Mahabharat (JioHotstar) · Hanuman · Krishna · OTT",
"learning": "MCA @ Jain University",
}| Project | What it is |
|---|---|
| ✂️ Carve | Browser-based VFX segmentation & matting — SAM 2/3/3.1 + ViTMatte + BiRefNet on an H100. Auto multi-ID, prompted and text-prompted masks, soft-alpha matting, sequence propagation, native EXR/ACES. Ships mattes on live film projects. |
| 🧪 Clean-Licence Matte Model | Retrained ViTMatte-S on a licence-clean corpus (BiRefNet pseudo-labels + P3M-10k + greenscreen alphas, per-clip provenance in MANIFEST.json) so mattes can ship to external clients. v2.1 is the production matte model. |
| 🎭 Six-Agent Creative Pipeline | Six hosted agents on Azure AI Foundry take a script → renderable shot panels. MCP-style tool calling, RAG over show-bible vector stores, and a 4-level LLM-as-judge eval harness gating output. In production. |
| 👗 Drape | Flat product photography → publishable campaign shots, with the garment's colour, print and construction held to the reference. Every frame is QC'd against the source before a human sees it. |
| 🧠 Blender Vision Agent | A vision-LLM agent that writes Blender Python, renders it headless, scores the render against the reference image, and iterates. Generate → execute → verify → refine. |
| 🌐 Vantage — World Builder | Front + back photo → an explorable 3D world (World Labs Marble), walkable in-browser via a Gaussian-splat renderer, with any camera angle recoverable as a faithful high-res still. |
| 🎥 MultiCam Control | Cross-platform GoPro fleet app — BLE discovery, COHN multi-stream live preview, synchronised record/stop, shoot/take management, parallel S3 pull. Electron · React · FastAPI. |
| 🔊 Speaker Voice Pipeline | Isolate one speaker's voice from any source in one command: yt-dlp → Demucs → pyannote → Whisper → MossFormer2 → Resemblyzer. |
| 🧱 Blender Pipeline | Headless Blender Python that turns raw AI-generated meshes into clean, posable, textured assets — multi-strategy UV unwrapping lifts texture coverage from ~10-20% to 66%. |
Agentic AI — multi-agent design · Azure AI Foundry hosted agents · MCP / tool calling · RAG · LLM-as-judge eval harnesses · DAG orchestration · human-in-the-loop checkpoints Model training — LoRA & foundation-model fine-tuning (Flux · Wan · Qwen · SDXL) · reference-image control · dataset curation & licence provenance · PyTorch Open-source models — SAM 2/3/3.1 · ViTMatte · BiRefNet · LaMa · IC-Light · OWLv2 · FBA Matting · Hermes · Gemma · pyannote · Whisper · Demucs Generative video — Seedance 2.0 · Bernini · Nanobanana · gpt-image-2 · World Labs Marble · ComfyUI · video upscaling · multi-character lipsync 3D / VFX — Blender Python API · retopology · texture baking · UV unwrapping · Gaussian splats · EXR / ACES colour Infrastructure — Azure AI Foundry · H100 GPU hosts · Cloudflare tunnels · systemd · SQLite (WAL) · Vercel · SSE streaming
Open to senior AI / ML engineering roles where I can own a slice of generative-AI infrastructure end to end.
📫 colinjoylobo16@gmail.com · 💼 linkedin.com/in/colinjoylobo · 🌐 colinjoylobo.vercel.app