codegen: store constant union members inline instead of boxing

[maleadt]

Since #55045 a small isbits Union built from a constant (e.g. x = cond ? a::Int32 : 0) is represented fully boxed: the unboxed payload slot is never written and consumers read the value back through a literal_pointer_val of the boxed constant. mark_julia_const marks constants as isboxed, so the union-phi path keeps them boxed rather than storing them inline as it did before. The embedded object pointer is dead weight and breaks consumers that can't dereference host addresses (e.g. GPU back-ends). Detect a constant whose type is an unboxed leaf of the target union and store its bits inline with an unmarked tindex, restoring the pre-#55045 representation.

[gbaraldi]

Maybe a codegen test?

[maleadt]

Added a test.

[topolarity]

This is a de-optimization for normal targets, since it means that a boxed(...) downstream of this ϕ will now be forced to allocate at runtime, even though it had a (boxed) constant object already available at codegen.

Do we need a codegen param to disable carrying those opportunistic boxes around on GPU targets?

[vtjnash]

Agreed,though furthermore this feels conceptually illogical and integrated in the wrong place

[maleadt]

Okay. Implemented as a cgparam instead.

[gbaraldi]

This isn't illogical because GPUs operate under different logic (they can't access our globals). But a cgparam seems like a good 2nd option

[topolarity]

Is it possible to apply this as a transform pass instead?

Having to maintain all of the places we eagerly emit a boxed pointer is not going to age very well - We expect to do this more in future, and I don't think this covers all of the cases that we do it already

[maleadt]

Is it possible to apply this as a transform pass instead?

And how would that work?

[topolarity]

It's certainly awkward. We can lift a jl_global# to a Module-local (constant) global for immutable pointer-free constants, which I think is all you need. The problem would be that you are now pretending that the value is boxed when it is not - you can copy the typetag for any typeof logic we do, but if it interacts with the GC at all it would not work. Of course GPUs do not interact with the GC. Maybe that idea is crazy though - only trying to find a good solution here

The alternative is branching everywhere that we eagerly try to forward boxes from .constant - that happens here in the PhiNode logic but it's also done in emit_ifelse, fieldtype, and convert_julia_type_to_union (which notably eagerly boxes when we return a .constant from a function with a Union return type). I think we'd probably want to re-factor this to a maybe_boxed(...) primitive or similar that can decide whether it is appropriate to eagerly forward a box, but I'm just worried it's quite easy to break GPUCompiler given how frequently we want this optimization on the host side.

[maleadt]

Maybe I'm misunderstanding, but I think I'd rather simply emit the isbits value rather than have to deal with copying CPU boxes to the GPU and rewrite the IR to make them addressable? We already have to deal with this in GPUCompiler for e.g. type tags, and having a cgparam could be useful to tie other emission patterns to.

I pushed a commit that prototypes your mabye_boxed idea; let me know what you think.

[vchuravy]

Wording wise, it does mention boxed objects, but embed_pointers implies something about singleton values like Symbol and Type which we do want to support on the GPU.

[maleadt]

True, but I was hoping we could also avoid some of that pointer embedding over time, hence keeping the language more neutral. Although to be honest I haven't really considered how we'd do that.

[maleadt]

Trying something out in GPUCompiler.jl instead: JuliaGPU/GPUCompiler.jl#874
@topolarity Would love to hear your thoughts on that.

[maleadt]

Done in GPUCompiler.jl instead: JuliaGPU/GPUCompiler.jl#874