Long Length LTX2 I2V+Audio?

[holycowdude]

Hi thanks, amazing work on the workflows

Is it possible to create a workflow that will take a image of a person which can be driven by an audio voice track please?
All my attempts lead to a gradual degradation of the video quality which gets really bad over a long clip...

[Z-L-D]

Can you elaborate a little more on your use case? I think I understand that you want an audio-to-video workflow with a starting image where you supply your own audio track and starting image?

In terms of video degradation, the workflow will work a lot better in a future version that keeps rolling from segment to segment in latent space but I'm not quite ready to jump to that just yet (some issues to iron out).

[holycowdude]

Hi, thanks for replying

Correct, I'm looking for a way to drive an image of a person with a voice audio track over long duration clips without quality degradation gradually getting worse.
Basically the same use as Infinite Talk I2V as this works well over long duration videos but I find LTX2 looks more realistic in body / head movements and expressions compared to Infinite Talk.

I'm not sure if there is anything better out there to try instead?

[Z-L-D]

I honestly haven't done a lot yet with long duration videos that attempt to maintain consistent voices. There is a few methods I'm exploring to retain vocal tonality over long durations though. The two paths I'm currently looking at will either involve one of the following:

1. a separate workflow for generating long form audio in a special way and then you'll run the audio into the main workflow.
2. adding in longer anchor referencing but this comes with the caveat of needing more vram

With the current workflow without any audio anchoring, I've generated clips up to 30 seconds that largely retained the same tonality across three 10 second generations. I need to do more experimenting, but vocal tonality seems to be tied to the appearance of a person at least to some degree so reusing their appearance may be all that is necessary which opens up a few other doors if that is all that is necessary.

Do you have any examples you'd like to share? Maybe I can offer something more specific for your use case in the mean time.