Summary
Follow-up #388 (comment)
The desired API style:
N = 1024 # static, closure
@pto.jit
def my_kernel(x_ptr: pto.ptr(dtype=pto.f32), batch: pto.i32):
x = pto.make_tensor_view(x_ptr, shape=[batch, N], stride=...) # `batch` dim is run-time dynamic
Omit rank in function arg's type annotation, which can be known from later make_tensor_view call (rank=len(shape))
区分静态&动态shape:静态轴用closure包进去(和 #388 现在写的一样),动态轴作为pto.int32入参(和我们a2 dsl现成样例一样),在kernel内部make_tensor_view -> 转化成需要的shape。
launch的时候,从torch tensor拿shape,作为额外的int参数给kernel launch接口,例如这里的total_len。ctype调用的时候,多传一个Python int
Motivation / use case
A unified kernel entry convention for static and dynamic axis.
Alternatives considered
Tilelang/CuTile-style would be like:
N = 1024 # static dim
batch = pto.symbolic("int") # dynamic dim
@pto.jit
def my_kernel(x: pto.tensor(shape=[batch, N])):
# no need for extra `make_tensor_view` in body
However, the total lines of code is not shorter. This adds one more level of parsing between frontend and IR, which complicates the software stack. So this style is NOT picked
The raw pto.ptr + pto.int also matches our existing C++ kernels that mostly use __gm__ T* + int32_t as kernel entry, facilitating C++ -> Python porting.
Additional context
@MirkoDeVita98 made a PR of dynamic-shape vec-add as a working example Zhendong404#3
Summary
Follow-up #388 (comment)
The desired API style:
Omit
rankin function arg's type annotation, which can be known from latermake_tensor_viewcall (rank=len(shape))区分静态&动态shape:静态轴用closure包进去(和 #388 现在写的一样),动态轴作为pto.int32入参(和我们a2 dsl现成样例一样),在kernel内部make_tensor_view -> 转化成需要的shape。
launch的时候,从torch tensor拿shape,作为额外的int参数给kernel launch接口,例如这里的
total_len。ctype调用的时候,多传一个Python intMotivation / use case
A unified kernel entry convention for static and dynamic axis.
Alternatives considered
Tilelang/CuTile-style would be like:
However, the total lines of code is not shorter. This adds one more level of parsing between frontend and IR, which complicates the software stack. So this style is NOT picked
The raw
pto.ptr+pto.intalso matches our existing C++ kernels that mostly use__gm__ T*+int32_tas kernel entry, facilitating C++ -> Python porting.Additional context
@MirkoDeVita98 made a PR of dynamic-shape vec-add as a working example Zhendong404#3