The "repetition_penalty" seems not to be truly enabled, is there a plan to implement it?

[neil0306]

Issue Summary

I've noticed that the repetition_penalty parameter appears to be defined in the codebase but doesn't seem to be fully functional in practice. I'm wondering if there are any plans to implement this feature completely.

Background

The repetition_penalty is an important sampling parameter for controlling text generation quality by reducing repetitive outputs. While the parameter is accepted by the framework, it seems like the actual implementation may be missing or incomplete.

Current Behavior

• The repetition_penalty parameter can be set in the API/configuration
• However, setting different values (e.g., 1.0, 1.5, 2.0) doesn't appear to produce the expected effect on reducing repetitions in generated text
• The parameter definition exists but may not be actively used during the generation process

Expected Behavior

When repetition_penalty is set to a value greater than 1.0, it should:

• Penalize tokens that have already appeared in the sequence
• Reduce repetitive patterns in generated text
• Provide more diverse and natural outputs

Questions

1. Is there a roadmap for implementing the full repetition_penalty functionality?
2. Are there any technical challenges preventing its implementation?
3. Is there a workaround or alternative approach currently available?
4. If implementation is planned, what's the estimated timeline?

Environment

• Version: Jetson AGX Orin
• Platform: Ubuntu 22.04.5 LTS
• Use case: VLM inference, like vila model

Thank you for your time and consideration!