Issues

• Any plan for integrating DPO?

  RWKV/RWKV-infctx-trainer#104 · Filter by author petergaoshan opened on Oct 14, 2024

• Is the state gradient not implemented yet for the CUDA kernel? (hence bptt_truncated_learning still forced to be True?)

  RWKV/RWKV-infctx-trainer#102 · Filter by author shouldsee opened on Jul 31, 2024
  1

• Is the training of the RWKV-6 model not yet applicable?

  RWKV/RWKV-infctx-trainer#86 · Filter by author wooks186 opened on Feb 19, 2024
  1

• [Feature] Support byte wise / character level encoding

  RWKV/RWKV-infctx-trainer#67 · Filter by author PicoCreator opened on Jan 25, 2024

• [Feature] Run OpenAI compatible API server for eval harness

  RWKV/RWKV-infctx-trainer#30 · Filter by author PicoCreator opened on Sep 14, 2023
  1

• Suggestion: trainer should do warning for low disk space

  RWKV/RWKV-infctx-trainer#27 · Filter by author PicoCreator opened on Aug 26, 2023

• [Feature] positive / negative data support (aka RFHL)

  RWKV/RWKV-infctx-trainer#21 · Filter by author PicoCreator opened on Aug 23, 2023

• [Feature] LoRA support

  RWKV/RWKV-infctx-trainer#20 · Filter by author PicoCreator opened on Aug 23, 2023
  1

• [feature] handle multiple dataset seperately, and log loss on them seperately

  RWKV/RWKV-infctx-trainer#15 · Filter by author PicoCreator opened on Aug 22, 2023