Documentation and diagrams for creating a workflow to produce media, such as 2D/3D video animations.
The original motivation for this pipeline was to produce 2D/3D animations, but since it's so cross-disciplinary, I plan to generalize these workflows for many purposes.
This workflow should be able to produce 2D animations (like anime), 3D animations (like pre-rendered video game cutscenes), and both at the same time (e.g., 3D backgrounds in anime, or 2D visual effects in the Spider-Verse movie).
The timesheet-based animation workflow is tailored to the Japanese anime industry's production pipeline. But I try to unify the terminology and give analogies for Western terms.
The 3D aspect of this is planned to be used for rendering animations of 3D characters. Namely, characters from our games, so we can post fun video animations of them as another form of content and branding/marketing. We also want this to be a common pipeline for producing actual in-game cutscenes.
I think we could even combine this into being a general video-editing and CG/VFX workflow as well for live-action content.
Another possible use for this pipeline is to render still images as artwork and brand content. E.g., some video game covers are extremely complex (like Mario Kart World). Even though they are just single images, they use the 3D models, pose the models, add custom VFX (3D and 2D), overlay the game title, etc. Basically, they should be able to do anything they need (using all departments) to get a really good final artwork.
Note: Any tools we end up writing for this are planned to be developed in separate repositories. We will link to them here when they become available.
The diagrams are written in D2. To see the entire pipeline, install the latest D2 download from https://github.com/terrastruct/d2/releases, and run the command d2 Overview.d2 Overview.png (with the current working directory at this cloned repository's root).
So far, all diagrams are completely generic to software choice, but here are the tools we plan to use.
All software of interest (not all are incorporated in the workflow):
- Blender
- Open 3D Engine (O3DE)
- Clip Studio
.xdtsfile format.tdtsfile format- Toei Animation Digital Exposure Sheet
- OpenToonz
- Krita
- Meddy
- Git
- A version control solution for large binary assets
- fSpy
Any software is fine. 2D illustrations that are not used as animation cels have way more freedom as to how they are authored. It could even be hand-painted and scanned.
I decide to use Clip Studio.
Although ideally not actually required. I feel we should be able to use any illustration software for this step and rely on the compositing step to achieve anything we want in there. At the end of the day, all that matters is that we have the finished cels to send to the compositing team. Clip Studio just happens to have really good animation previewing tools, while at the same time being a top illustration experience.
Toei's file format is not completely necessary. We could just use .xdts for storing the minimal exposure information. TDTS has some nice metadata as well though, like memos for compositing instructions, episode information. It's nice to have a proper digital format for this information rather than writing it on a PDF.
Or custom scripts for syncing timesheet files to and from non-Clip-Studio software.
We will need a way to sync timesheet information to the compositing software of choice. Toei's program has tools for After Effects, but we're not using AE.
Blender's Video Sequence Editor for animating cels, implementing anime camera work, and VFX.
O3DE will allow us to
- Write custom shaders
- Write custom render passes
- Heavily customize graphics features
- Add particle systems (currently a very early feature)
- Easily use in-game content (in cases that we're rendering media for a video game developed in O3DE)
Blender's Video Sequence Editor for combining all the cuts and audio, and rendering the final output artifact to go out for delivery.
Maybe there's a better alternative, though.
Maybe there's a better alternative, though.