glyphdeck will inconsistently hang during
cascade.run("title string")
Happens on most runs for data > 1000 entries, depending on the machine.
DEBUG logs show X times the following logs, which eventually stops after one of the Start | Chat completion logs.
Have waited upwards of 30minutes to no effect.
2024-11-21 16:29:32,012 | DEBUG | processors.LLMHandler | | Step | async_openai() | Start | Chat completion
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 3 | Property | active_record_title() | Start
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 4 | Property | active_record_key() | Start
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 5 | Property | latest_key() | Start
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 5 | Property | latest_key() | Finish
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 4 | Property | active_record_key() | Finish
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 4 | Function | title() | Start
2024-11-21 16:29:32,016 | DEBUG | processors.cascade | Nest 5 | Function | record() | Start
2024-11-21 16:29:32,017 | DEBUG | processors.cascade | Nest 5 | Function | record() | Finish
2024-11-21 16:29:32,017 | DEBUG | processors.cascade | Nest 4 | Function | title() | Finish
2024-11-21 16:29:32,017 | DEBUG | processors.cascade | Nest 3 | Property | active_record_title() | Finish
2024-11-21 16:29:32,017 | DEBUG | processors.LLMHandler | | Step | async_openai() | Action | chat_params = {'model': 'gpt-4o-mini', 'response_model': <class 'glyphdeck.validation.validators.SubCatsPerItemOverallSentiment'>, 'max_retries': 2, 'temperature': 0.2, 'messages': [{'role': 'system', 'content': "You are an expert customer feedback analyst system specialised in analysing feedback from hotels.Categorise into themes that represent patterns in the feedback. Don't be too specific.Analyse the feedback and return results in the correct format."}, {'role': 'user', 'content': '<INPUT_TEXT>'}]}
2024-11-21 16:29:32,017 | DEBUG | processors.LLMHandler | | Step | async_openai() | Start | Chat completion
The logs would indicate the chat completions coroutines are all being created before any can finish. So potentially a memory issue with larger workloads, but this should not be occurring regardless.
Look into the concurrency semaphores - are they being released on coroutine execution or just after the coroutine is created?
glyphdeck will inconsistently hang during
cascade.run("title string")Happens on most runs for data > 1000 entries, depending on the machine.
DEBUG logs show X times the following logs, which eventually stops after one of the
Start | Chat completionlogs.Have waited upwards of 30minutes to no effect.
The logs would indicate the chat completions coroutines are all being created before any can finish. So potentially a memory issue with larger workloads, but this should not be occurring regardless.
Look into the concurrency semaphores - are they being released on coroutine execution or just after the coroutine is created?