[Question]: Way to summarize long text without going over rate limits?

[lukasnoeller]

What is your question?

Hi, all, was wondering if there was a way to use the summarize or extract_wisdom patterns to process a text that exceeds the token rate limit that one may have when using the lower tier accounts at an ai hosting service. I wouldn't mind if it splits the job over a few minutes. Wanted to ask here first to see if someone already found a solution to this before I try to whip up one for myself.

[charlie-romeo]

Hey @lukasnoeller . I did this with split -n 2 ... I used it a couple time last year on long YouTube transcript.

It was a little weird because split at an odd location for the transcript. I had to do a little hand jamming of a few lines of summarization/context from the first chuck into the second. I still hit a rate limit but that was not context it was req/sec

I imagine langchain or a newer agent based workflow would be better now if you regularly had to process large inputs

sorry no easy one-liner for you. good luck

[lukasnoeller]

Yeah I kind of figured, thanks for responding though!