How to run ngram speculative decoding.

[zzhbrr]

I wonder how to run ngram speculative decoding.

I encountered an error when setting speculative.spec_strategy=ngram.

The configuration is as follows:

python3 -m verl.trainer.main_fastrl \
    speculative.enable=true \
    speculative.bs_threshold=15 \
    speculative.spec_strategy=ngram \
    speculative.eagle.spec_verify_tokens=3 \

The error message is:

�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/srt/managers/scheduler.py", line 3130, in run_scheduler_process
�[36m(WorkerDict pid=3944349)�[0m     scheduler.event_loop_overlap()
�[36m(WorkerDict pid=3944349)�[0m   File "/root/anaconda3/envs/fastrl/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
�[36m(WorkerDict pid=3944349)�[0m     return func(*args, **kwargs)
�[36m(WorkerDict pid=3944349)�[0m            ^^^^^^^^^^^^^^^^^^^^^
�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/srt/managers/scheduler.py", line 1075, in event_loop_overlap
�[36m(WorkerDict pid=3944349)�[0m     self.process_input_requests(recv_reqs)
�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/srt/managers/scheduler.py", line 1349, in process_input_requests
�[36m(WorkerDict pid=3944349)�[0m     output = self._request_dispatcher(recv_req)
�[36m(WorkerDict pid=3944349)�[0m              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/utils.py", line 496, in __call__
�[36m(WorkerDict pid=3944349)�[0m     return fn(obj)
�[36m(WorkerDict pid=3944349)�[0m            ^^^^^^^
�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/srt/managers/scheduler_update_weights_mixin.py", line 114, in release_memory_occupation
�[36m(WorkerDict pid=3944349)�[0m     self.flush_cache()
�[36m(WorkerDict pid=3944349)�[0m   File "/mnt/zzhbr/syncworkspace/fastrl/third-party/sglang/python/sglang/srt/managers/scheduler.py", line 2676, in flush_cache
�[36m(WorkerDict pid=3944349)�[0m     if self.draft_worker.adaptive_spec_threshold is not None:
�[36m(WorkerDict pid=3944349)�[0m        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[36m(WorkerDict pid=3944349)�[0m AttributeError: 'NGRAMWorker' object has no attribute 'adaptive_spec_threshold'