core: support odd-bit primitive integers, add Core.bitsizeof by fingolfin · Pull Request #61359 · JuliaLang/julia

fingolfin · 2026-03-18T14:35:51Z

Allow declaring primitive integer types with non-byte logical widths while keeping their storage byte-rounded. Track the declared bit width in datatype layout metadata, use it in width-sensitive runtime and codegen paths, and add Core.bitsizeof for public introspection.

Update bitstring and the manual to reflect logical bit widths, and add coverage for UInt63 and Int63 declarations, conversions, and LLVM lowering.

Some non-goals for this initial step:

packed array storage
reinterpret/array layout changes
broader ABI guarantees for odd-bit primitive integers
actually defining UInt63 or similar types anywhere other than the test suite
adding full-blown arithmetics and interop for these these types (doing so seems to be opening a can of worms, while in my applications, I will mostly end up coercing or bitshifting between these types and "ordinary" Int64/UInt64 types anyway).

This pull request was written with the assistance of generative AI.

Co-authored-by: Codex codex@openai.com

Motivated by discussion on PR #61262.

Also constitutes partial progress on issue #45486 (but no attempts at bit packed arrays, or "arbitrary size" integers, just enabling odd size).

JeffBezanson · 2026-03-18T23:22:33Z

Thanks for working on this, this would be a great feature.

topolarity · 2026-03-19T16:37:31Z

Thanks for starting this initiative. Not sure how useful this is to you, but I went ahead and spun up a bot to explore an implementation for #61361 along with this change: 8a8a6ac

That commit also punts on the issue of bit-level padding in reinterpret, but the bot did try to implement intrinsics, etc. to avoid regressing e.g. UInt24 . I tried to keep the robot mostly on the rails and prod it for useful tests, but it's in major need of human hands / clean-up / comprehensive review still. Anyway I thought it might be a useful reference for you to compare against. I do think it found at least a few places to update that are missing from this PR as-is.

Allow declaring primitive integer types with non-byte logical widths while keeping their storage byte-rounded. Track the declared bit width in datatype layout metadata, use it in width-sensitive runtime and codegen paths, and add Core.bitsizeof for public introspection. Update bitstring and the manual to reflect logical bit widths, and add coverage for UInt63 and Int63 declarations, conversions, and LLVM lowering. Co-authored-by: Codex <codex@openai.com>

Handle byte extraction in bitstring by logical bit width rather than rounded storage size. Sub-byte primitives need zero-extension to UInt8, while wider odd-bit primitives still truncate to their low byte. Add regression coverage for a 6-bit primitive to keep bitstring working below one byte. Co-authored-by: Codex <codex@openai.com>

Store odd-bit primitive width metadata in an explicit unused_bits field instead of overloading the layout struct padding bits. This keeps the layout description self-documenting while preserving the same packed footprint. Adjust the layout builder and runtime metadata accessors to use the new field name, and update the corresponding runtime_internals bitfield notes. Co-authored-by: Codex <codex@openai.com>

Reject byte-oriented reinterpret helpers for non-byte-aligned primitive types, fix APInt sign, zero-extension, and truncation for sub-byte widths, and use storage width for ABI integer rewrites. Extend the odd-width intrinsics and reinterpret tests. Co-authored-by: Codex <codex@openai.com>

Rewrite the primitive type description to say that the declared bit count is the logical width, while storage remains byte-rounded. This keeps the manual aligned with odd-bit primitive support and Core.bitsizeof. Co-authored-by: Codex <codex@openai.com>

Add regressions for odd-width primitive equality and object identity masking, integer-float conversion round trips, Ref and field loads, and storage-metadata behavior. Extend reinterpret coverage for odd-width primitive layout metadata as well. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-03-27T17:13:26Z

@topolarity we (Codex and me) substantially revised this, also based on some of the things you did in your PR.

Perhaps triage can make a decision on issue #61361 so I know whether to take that into account or not...

fingolfin · 2026-03-27T17:50:11Z

BTW there is at least one genuine test failure, I just did not yet have time to look into it:

Error in testset intrinsics:
Error During Test at /cache/build/tester-amdci4-15/julialang/julia-master/julia-553f547e5a/share/julia/test/testdefs.jl:25
  Got exception outside of a @test
  LoadError: MethodError: no method matching Main.Test60Main_intrinsics.Int63(::Int32)
  The type `Main.Test60Main_intrinsics.Int63` exists, but no method is defined for this combination of argument types when trying to construct it.
  Closest candidates are:
    (::Type{T})(::T) where T<:Number
     @ Core boot.jl:1087
    Main.Test60Main_intrinsics.Int63(::Int64)
     @ Main.Test60Main_intrinsics /cache/build/tester-amdci4-15/julialang/julia-master/julia-553f547e5a/share/julia/test/intrinsics.jl:200
    (::Type{<:Integer})(::Sockets.IPAddr)
     @ Sockets /cache/build/tester-amdci4-15/julialang/julia-master/julia-553f547e5a/share/julia/stdlib/v1.14/Sockets/src/IPAddr.jl:11
    ...

topolarity · 2026-04-03T13:33:38Z

Perhaps triage can make a decision on issue #61361 so I know whether to take that into account or not...

I think it might be possible to proceed without the triage decision, at least for a while. If we can make progress making the codebase consistent about allocation vs. storage vs. register (bit) size then accepting #61361 is a small change to make allocation size == storage size but most of codegen, intrinsics, etc. shouldn't be affected too badly.

I do think it might be useful to create an "action list" of areas that need non-byte-multiple integer support and to tackle them piece-by-piece. Specifically this piece:

adding full-blown arithmetics and interop for these these types (doing so seems to be opening a can of worms, while in my applications, I will mostly end up coercing or bitshifting between these types and "ordinary" Int64/UInt64 types anyway).

doesn't seem that far out-of-reach to me TBH (not that you have to / should worry about it in this PR).

I think the LLVM support should "just work" as long as we add the "cast storage integer type to / from register integer type" on the boundary with memory from 8a8a6ac, and I'd be happy to do an "odd-bit" follow-up to #61399 for this. The improved test coverage from that PR should be a help too.

Keno · 2026-07-04T06:17:39Z

Can we move this forward? What is missing?

Port odd-width APInt conflict resolution to APInt.c. Keep datatype unused_bits alongside master layout updates. Co-authored-by: Codex <codex@openai.com>

Allow the local Int63 test constructor to accept signed integers through Int64 so default Int32 literals on 32-bit runners exercise the same truncation path. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-09T20:34:02Z

    if (!dt->layout->nfields && !dt->layout->npointers)
        return NULL;
-    return Type::getIntNTy(ctx, jl_datatype_nbits(dt));
+    return Type::getIntNTy(ctx, jl_datatype_size(dt) * 8);


For ABIs we need byte store, so here we move away from jl_datatype_nbits

fingolfin · 2026-07-09T20:34:21Z

I've just resolve the merge conflicts. Had to adapt to the APInt-C.cpp -> APInt.c migration.

I've asked Codex to summarize what it thinks of the state of this PR -- sorry, I don't mean to be rude by doing this, but I wasn't sure based on the previous comments what might be expected or not, so I thought I'd ask it ... here is what it wrote after I pointed it at the discussion here:

Locally I verified:

make -j4

JULIA_TEST_FAILFAST=1 ./julia --startup-file=no test/runtests.jl intrinsics apint

JULIA_TEST_FAILFAST=1 ./julia --startup-file=no test/runtests.jl intfuncs reinterpretarray core

The earlier Int63(::Int32) intrinsics failure is no longer reproducing on the current branch.

I don’t see inline review comments currently. The remaining question I’m
aware of is whether to include the broader #61361 storage/alignment
change here. My current view is that it should remain a follow-up:
this PR keeps existing byte-rounded storage behavior and focuses on
logical odd-bit primitive widths plus Core.bitsizeof.

Anyway: there was also a failure on 32bit systems that I hope is fixed now.

Of course I (with Codex/ChatGPT) can do more work on this, but right now, I am indeed not quite sure else is expected / necessary? I mean: of course more work is needed, but would it perhaps make sense to get this (or perhaps just parts?) in sooner rather than later to avoid further conflicts caused by code drift, and then we can do follow-up PRs, e.g. to change the storage behavior?

Keno · 2026-07-09T20:46:26Z

I'm happy with incremental progress, especially for new features, so once CI goes through, I'm happy to get it merged. I'll also take another review pass.

Mask the partial storage byte when zero-extending odd-bit APInt values, and make reinterpret padding checks inspect isbits union members so odd-width primitives cannot bypass the guard. Add focused coverage for runtime odd-width zero extension and reinterpret rejection through an isbits union field. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-09T21:33:20Z

It would definitely be good if others checked this -- while I tried my best to discuss all changes with Codex, we may very well have missed something.

Resolve the bitstring conflict by preserving odd-bit primitive extraction while adopting master wrapping arithmetic. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-10T09:20:18Z

Another merge conflict popped up, should be resolved now.

Use declared primitive bit widths when modeling bitcast and integer conversion exceptions, so invalid casts remain observable and valid odd-width conversions are recognized. Co-authored-by: Codex <codex@openai.com>

Share size transfer logic with Core.sizeof and register bitsizeof for constant folding, return inference, and builtin effect analysis. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-10T11:36:11Z

Realized there are some more issues with this. But now need to attend some talks. Will update PR later tonight

Pad the leading partial byte to its logical hexadecimal width so low-valued 5-, 6-, and 7-bit primitives keep a stable representation. Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-11T08:02:35Z

OK pushed three more commits.

fingolfin · 2026-07-11T08:08:32Z

One fixed a somewhat serious issue where intrinsic_exct in the compiler still compared primitive types based on their sizeof value, not bitsizeof. That could lead to the compiler eliding a check it shouldn't have.

Another commit improves support in the compiler for bitsizeof to match that for sizeof, so that e.g. bitsizeof can be const folded, correctly type inferred etc.

Make the expected values explicitly Int64 so identity comparisons do not depend on the host-sized Int literal type. CI: https://buildkite.com/julialang/julia-pr/builds/333#019f5038-9b13-473b-9287-582d1b7113b4 Co-authored-by: Codex <codex@openai.com>

fingolfin · 2026-07-15T20:04:11Z

I've once again resolved merge conflicts.

The repeating build failure on x86_64-apple-darwin seems to be unrelated to this PR (see discussion on Slack)

Keno · 2026-07-15T20:06:24Z

Yeah, the Darwin builders have been having trouble, fine to ignore for now.

fingolfin marked this pull request as ready for review March 18, 2026 14:35

This was referenced Mar 18, 2026

WIP: implement ImmediateOrRef{T} aka "tagged pointers" #61262

Draft

Support arbitrary bitwidth integers #45486

Open

topolarity mentioned this pull request Mar 18, 2026

Consider making size of non-power-of-two primitive types always aligned #61361

Open

fingolfin and others added 6 commits March 20, 2026 21:27

fingolfin force-pushed the codex/int63-primitive-support branch from e8cd519 to 553f547 Compare March 20, 2026 20:28

fingolfin and others added 2 commits July 9, 2026 16:04

Merge master into odd-bit primitive support

a8ac5d1

Port odd-width APInt conflict resolution to APInt.c. Keep datatype unused_bits alongside master layout updates. Co-authored-by: Codex <codex@openai.com>

test: fix Int63 intrinsics on 32-bit hosts

2d4c60f

Allow the local Int63 test constructor to accept signed integers through Int64 so default Int32 literals on 32-bit runners exercise the same truncation path. Co-authored-by: Codex <codex@openai.com>

fingolfin commented Jul 9, 2026

View reviewed changes

simplify code

7dccab8

Keno reviewed Jul 9, 2026

View reviewed changes

Comment thread base/essentials.jl

topolarity self-requested a review July 9, 2026 21:59

fingolfin and others added 2 commits July 10, 2026 00:24

Add Core.bitsizeof(MyUInt63) example to docstring

fc3b47a

Merge master into odd-bit primitive support

adc4e22

Resolve the bitstring conflict by preserving odd-bit primitive extraction while adopting master wrapping arithmetic. Co-authored-by: Codex <codex@openai.com>

fingolfin and others added 2 commits July 10, 2026 13:15

compiler: Model intrinsic logical widths

57f50c9

Use declared primitive bit widths when modeling bitcast and integer conversion exceptions, so invalid casts remain observable and valid odd-width conversions are recognized. Co-authored-by: Codex <codex@openai.com>

compiler: Model Core.bitsizeof

bd9c68e

Share size transfer logic with Core.sizeof and register bitsizeof for constant folding, return inference, and builtin effect analysis. Co-authored-by: Codex <codex@openai.com>

fingolfin marked this pull request as draft July 10, 2026 11:36

show: Pad partial primitive bytes

8bebe5c

Pad the leading partial byte to its logical hexadecimal width so low-valued 5-, 6-, and 7-bit primitives keep a stable representation. Co-authored-by: Codex <codex@openai.com>

fingolfin marked this pull request as ready for review July 11, 2026 08:02

comments

263277a

fingolfin changed the title ~~core: support odd-bit primitive integers~~ core: support odd-bit primitive integers, add Core.bitsizeof Jul 11, 2026

fingolfin force-pushed the codex/int63-primitive-support branch from 0682913 to 4d43c94 Compare July 12, 2026 08:41

fingolfin added 2 commits July 15, 2026 10:03

Merge branch 'master' into codex/int63-primitive-support

e703880

Merge branch 'master' into codex/int63-primitive-support

a4eb7d4

Uh oh!

Uh oh!

Conversation

fingolfin commented Mar 18, 2026

Uh oh!

JeffBezanson commented Mar 18, 2026

Uh oh!

topolarity commented Mar 19, 2026

Uh oh!

fingolfin commented Mar 27, 2026

Uh oh!

fingolfin commented Mar 27, 2026

Uh oh!

topolarity commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Keno commented Jul 4, 2026

Uh oh!

fingolfin Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

fingolfin commented Jul 9, 2026

Uh oh!

Keno commented Jul 9, 2026

Uh oh!

fingolfin commented Jul 9, 2026

Uh oh!

Uh oh!

fingolfin commented Jul 10, 2026

Uh oh!

fingolfin commented Jul 10, 2026

Uh oh!

fingolfin commented Jul 11, 2026

Uh oh!

fingolfin commented Jul 11, 2026

Uh oh!

fingolfin commented Jul 15, 2026

Uh oh!

Keno commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

topolarity commented Apr 3, 2026 •

edited

Loading