Skip to content

[Feature] Unify kernel entry convention to "ptr + int" #417

Description

@learning-chip

Summary

Follow-up #388 (comment)

The desired API style:

N = 1024   # static, closure
@pto.jit
def my_kernel(x_ptr: pto.ptr(dtype=pto.f32), batch: pto.i32):
    x = pto.make_tensor_view(x_ptr, shape=[batch, N], stride=...)  # `batch` dim is run-time dynamic

Omit rank in function arg's type annotation, which can be known from later make_tensor_view call (rank=len(shape))

区分静态&动态shape:静态轴用closure包进去(和 #388 现在写的一样),动态轴作为pto.int32入参(和我们a2 dsl现成样例一样),在kernel内部make_tensor_view -> 转化成需要的shape。

launch的时候,从torch tensor拿shape,作为额外的int参数给kernel launch接口,例如这里的total_len。ctype调用的时候,多传一个Python int

Motivation / use case

A unified kernel entry convention for static and dynamic axis.

Alternatives considered

Tilelang/CuTile-style would be like:

N = 1024  # static dim
batch = pto.symbolic("int")  # dynamic dim
@pto.jit
def my_kernel(x: pto.tensor(shape=[batch, N])):
    # no need for extra `make_tensor_view` in body

However, the total lines of code is not shorter. This adds one more level of parsing between frontend and IR, which complicates the software stack. So this style is NOT picked

The raw pto.ptr + pto.int also matches our existing C++ kernels that mostly use __gm__ T* + int32_t as kernel entry, facilitating C++ -> Python porting.

Additional context

@MirkoDeVita98 made a PR of dynamic-shape vec-add as a working example Zhendong404#3

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions