Skip to content

initial dynamic input support for gpu codegen #43

Closed

Description

@harz05

Context-

Dynamic inputs means an input with a symbolic dimension (e.g. batch size N) that stays a runtime value instead of being fixed at code generation time. As of now the cpu path already supports this, this issue tracks the state of the alpaka GPU path.

What was tested

A minimal model with a dynamic batch dim:
X[N,4] -> Gemm(W[4,3], B[3]) -> Relu -> Y[N,3], generated through the alpaka emitter.

Result

The infer signature already threads N through as a runtime argument and Gemm uses it correctly (op_m = N), so the basic plumbing exists. However the generated code does not compile, because the dynamic intermediate tensor allocation in RModel_ALPAKA.cxx has some issues, such as the declaration of bufDev_<name> not matching the deviceBuf_<name> used elsewhere. As a first step for the support, it would be required to fix the dynamic tensor allocation in RModel_ALPAKA.cxx and add a small batch test for it. Then probably followed by adding support for various relevant operators

Activity

  1. harz05 commented on Sep 17, 2026

    @harz05
    MemberAuthor

    Closing as completed via #45

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions