[Feat]: Partial caching

[dxqb]

Describe your use-case.

Text embeddings have become large in recent models. Large datasets can generate text caches of hundreds of GB or even TB of data.

#1345 and #1462 are improvements but the issue remains - text caching isn't feasible anymore in those cases.

However, not caching requires the TE to be in VRAM currently, which can also be a dealbreaker.

What would you like to see as a solution?

An intermediate text cache:

1. Create text cache for an interval, for example for 1000 steps
2. Move text encoder to RAM
3. Train those 1000 steps
4. Discard text cache
5. Repeat

Have you considered alternatives? List them here.

No response

[O-J1]

Dont do this on older models and not sure how steps would work here? The problem is: its based on how large the specific models TE embeddings are, how much data the user has and how much spare space they have on the current drive

[dxqb]

Dont do this on older models and not sure how steps would work here? The problem is: its based on how large the specific models TE embeddings are, how much data the user has and how much spare space they have on the current drive

it must be an option. full cache still has its use obviously, because what I described above looses the benefit of using the cache > 1 times