Skip to content

KV cache management #7621

Closed
Closed
KV cache management#7621
Labels
module: llmIssues related to LLM examples and apps, and to the extensions/llm/ code

Description

@iseeyuan
iseeyuan
opened on Jan 13, 2025

Although KV cache is enabled, it's not modularized.

  • Architecture can be unified or configurable for different backends.
  • APIs provided for higher-level use cases, like different eviction schemes.

Activity

  1. iseeyuan
    added this to etLLM: LLMs via ExecuTorchon Jan 13, 2025
  2. iseeyuan
    moved this to Todo in etLLM: LLMs via ExecuTorchon Jan 13, 2025
  3. mcr229
    added
    feature
    module: llmIssues related to LLM examples and apps, and to the extensions/llm/ code
    on Jan 14, 2025
  4. pytorch
    locked and limited conversation to collaborators on Feb 4, 2025
  5. digantdesai
    converted this issue into a discussion #8169 on Feb 4, 2025
  6. github-project-automation
    moved this from Todo to Done in etLLM: LLMs via ExecuTorchon Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: llmIssues related to LLM examples and apps, and to the extensions/llm/ code

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions