feat(backends): handle conditional disagg bypass annotation#11719
feat(backends): handle conditional disagg bypass annotation#11719karen-sy wants to merge 1 commit into
Conversation
Signed-off-by: Karen Chung <karenc@nvidia.com>
github-actions
Bot
added
feat
backend::vllm
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if ( | ||
| dynamo_config.disaggregation_mode == DisaggregationMode.DECODE | ||
| and not dynamo_config.enable_conditional_disagg | ||
| ): | ||
| logger.info( | ||
| "Decode worker detected (disaggregation_mode=decode): " | ||
| "kv_events_config disabled (decode workers don't publish KV events)" | ||
| "kv_events_config disabled (pass --enable-conditional-disagg to opt in)" | ||
| ) | ||
| return None |
There was a problem hiding this comment.
🔍 Conditional disagg KV event publishing requires user-provided kv_events_config
When enable_conditional_disagg=True on a decode worker, create_kv_events_config (args.py:404-407) no longer returns None early. However, it still returns None at line 429 if the user hasn't provided their own --kv-events-config. This means setup_kv_event_publisher (main.py:382-383) will see kv_events_config is None and skip publisher setup. The flag's help text says "Decode workers publish KV events so the Dynamo router can see decode-side cache" but this only works if the user also provides a kv_events_config (same requirement as prefill workers). This is consistent with how prefill workers work, but the flag's documentation might lead users to expect it's self-sufficient.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Important
This PR is [2/5] in the conditional-disaggregation stack.
Merge order is bottom-up: #11718 merges to
mainfirst, and #11723 merges last.Do not merge this PR until every earlier PR in the checklist has merged and this branch has been rebased/retargeted as needed.
Link to the DEP: #11514
Stack merge order:
Reference to original (closed) combined PR, with addressed review comments: ai-dynamo/dynamo#11357
Overview:
Details:
Where should the reviewer start?
Related Issues
🔗 This PR is linked to an issue:
🚫 This PR is NOT linked to an issue: