Skip to content
 
 

Repository files navigation

BitsKit

CI Benchmarks NuGet Version License

This is the community-maintained fork of barncastle/BitsKit. The assembly and namespaces remain BitsKit; maintained NuGet releases use the package ID RejectKid.BitsKit.

BitsKit is a lightweight C# library that provides efficient bit-level reading, writing and manipulation. As well as adding bit-field support to C#, not dissimilar to C/C++ languages.

All features support integral and memory types, as well as targeting both, Little Endian (LE) Least Significant Bit (LSB) and Big Endian (BE) Most Significant Bit (MSB).

Install the maintained package:

dotnet add package RejectKid.BitsKit

Existing source code continues to use the original namespaces, for example BitsKit.BitFields and BitsKit.Primitives. Do not reference both the original BitsKit package and RejectKid.BitsKit in the same project because they provide the same assembly and namespaces.

Features

Changes in this fork

This fork preserves the original public API while maintaining it for current .NET and C# versions. New APIs, including non-seekable stream writing and unsafe generated access, are additive. The detailed version-by-version history is in the changelog.

Area Maintained fork changes
Source generator Restored BITSKIT003, fixed inline-array setters for modern compilers, removed retained Roslyn state, and expanded generator regression coverage.
Generated-code performance Specialized integral, memory, span, array, fixed-buffer, Boolean, inline-array, aligned, and common-width accessors instead of routing every operation through general primitives.
Streams Correct partial-read and EOF behavior, pooled reader/writer buffering, large stream positions, consistent disposal errors, and forward-only/non-seekable BitStreamWriter output.
Runtime targets Targets netstandard2.1, .NET 8, and .NET 10; the end-of-life .NET 6 and .NET 7 assets were removed.
Validation Cross-platform CI, CodeQL, modern tests, package validation, release provenance, scheduled benchmarks, and an original-fork performance regression workflow.
Distribution Automated tagged GitHub releases and the maintained RejectKid.BitsKit package.
Optional unsafe access Generated byte-storage accessors can explicitly skip safety checks for trusted, sufficiently padded buffers; checked behavior remains the default.
Batch processing Packed and record-strided fields can be read or written in bulk directly or through opt-in generated helpers.

Migrating from the original package normally requires only changing the package reference:

<PackageReference Include="RejectKid.BitsKit" Version="1.6.1" />

No namespace changes are required. Applications that still target .NET 6 or .NET 7 can consume the netstandard2.1 asset when their target supports it, but those runtimes are no longer built or tested directly by this project.

Usage

BitPrimitives

BitsKit.Primitives.BitPrimitives is the workhorse of the library containing all of the read and write logic. This class is the bit equivalent of System.Buffers.Binary.BinaryPrimitives and contains a MSB and LSB read and write method for each of the below integral types:

sbyte, byte, short, ushort, int, uint, long, ulong, nint, nuint

Each type has two overloads allowing the source/destination to be either a Span<byte> or T e.g.,

// reads a range of bits from a uint as MSB
static uint ReadUInt32MSB(uint source, int bitOffset, int bitCount);

// reads a range of bits from a span as MSB
static uint ReadUInt32MSB(ReadOnlySpan<byte> source, int bitOffset, int bitCount);

// writes a range of bits to a uint as MSB
static void WriteUInt32MSB(ref uint destination, int bitOffset, uint value, int bitCount);

// writes a range of bits to a span as MSB
static void WriteUInt32MSB(Span<byte> destination, int bitOffset, uint value, int bitCount);

This class also provides a ReverseBitOrder method for each integral type which inverts the bit order of each byte within the value, but not the order (endianness) of the bytes themselves e.g.

// reverses the bit order of each byte
static uint ReverseBitOrder(uint value);

//7......0 7......0      0......7 0......7
0b11001000_00111011 => 0b00010011_11011100

Batch Processing

BitsKit.Primitives.BitBatchPrimitives reads and writes repeated fields with one upfront range validation and no allocations. It supports signed and unsigned 8-, 16-, 32-, and 64-bit values, native integers, Booleans, and both LSB and MSB order.

The destination span determines how many values are read. This example decodes 1,000 packed 12-bit samples:

ReadOnlySpan<byte> packetData = GetPacketData();
Span<ushort> samples = new ushort[1_000];

BitBatchPrimitives.ReadUInt16LSB(
    packetData,
    bitOffset: 0,
    bitCount: 12,
    destination: samples);

The matching write API packs values back-to-back:

ReadOnlySpan<ushort> samples = GetSamples();
Span<byte> packetData = GetOutputBuffer();

BitBatchPrimitives.WriteUInt16LSB(
    packetData,
    bitOffset: 0,
    bitCount: 12,
    values: samples);

Use bitStride when the same field appears in a repeated record. Here each record is 32 bits wide and the desired 11-bit field starts at bit 5:

BitBatchPrimitives.ReadUInt16LSB(
    packetData,
    bitOffset: 5,
    bitCount: 11,
    bitStride: 32,
    destination: values);

This reads bits 5–15, 37–47, 69–79, and so on. The stride must be at least the field width, so batch writes cannot overlap. The full batch is validated before any value is processed; an invalid offset, width, stride, count, or buffer throws ArgumentOutOfRangeException. The final accessed bit position is limited to Int32.MaxValue, matching the span-oriented primitive APIs.

Bit Fields

BitsKit provides the ability to generate bit-fields within types and aims to be as feature complete as the C and C++ implementations. This is achieved through the use of attributes applied to backing fields, which describe the structure and layout. These are converted into properties via a source generator.

Bit fields can be added to class, struct and record types, supporting all of their variants too e.g., readonly struct, record struct etc. Objects containing bit-fields are declared by the [BitObjectAttribute(BitOrder)] attribute which also declares the default bit order for the type. Types must be partial and not nested.

Due to the nature of source generators, the user must generate the backing fields. Whilst this is more verbose than in C, it does provide much more granularity and control opening up some interesting dynamics. Backing fields must be a field and either; one of integral types above or one of the memory types below.

byte[], fixed byte[], byte*, Span<byte>, ReadOnlySpan<byte>, Memory<byte>, ReadOnlyMemory<byte>

Fixed byte buffers have a compile-time capacity and support checked access. A raw byte* does not carry a length, so it is accepted only when the bit object explicitly selects unsafe access:

[BitObject(
    BitOrder.LeastSignificant,
    AccessMode = BitObjectAccessMode.Unsafe)]
public unsafe partial struct NativePacket
{
    [BitField("Id", 12, BitFieldType.UInt16)]
    public byte* Data;
}

The caller must keep Data non-null and ensure that it addresses enough accessible bytes for every generated field. Use a fixed buffer, array, span, or memory when checked bounds validation is required.

Bit fields are declared using the [BitFieldAttribute] attribute which describes their name, size, bit order and properties. Each attribute defines a new bit-field sequential from the previous. A backing field can have as many bit-fields as desired, limited only by field boundaries.

Notes:

  • If the backing field is an integral type, the bit-field will be of the same type. Type Casting is supported via FieldType.
  • If the backing field is a memory type, the FieldType is required as it cannot be inferred.
  • If the backing field is readonly or represents a readonly type, the bit-field will also be readonly.
  • Inline Arrays with an integral member type are supported.
// Constructor for integral backed bit-fields
[BitFieldAttribute(string name, byte size)]
// Constructor for memory backed bit-fields
[BitFieldAttribute(string name, byte size, BitFieldType fieldType)]

Fields:
// The name of the bit-field
public string? Name { get; }
// The number of bits the field occupies 
public byte Size { get; }
// The integral type of the field if backed by a memory type
public BitFieldType? FieldType { get; }
// Uses the opposite bit order than declared on the type
public bool ReverseBitOrder { get; set; }
// Modifiers that change the source generation
public BitFieldModifiers Modifiers { get; set; }

Padding Fields

Like C, an unnamed bit-field generates a set of inaccessible "padding" bits. These are primarily used for alignment or to map reserved/unused bits. There is a constructor overload dedicated to these fields.

// Constructor for integral padding bit-fields
[BitFieldAttribute(byte size)]
// Constructor for boolean padding bit-fields
[BooleanFieldAttribute]
// Constructor for enum padding bit-fields
[EnumFieldAttribute(byte size)]

Boolean Bit Fields

Boolean bit-fields are supported by the [BooleanFieldAttribute] helper attribute. Boolean fields consume a single bit and return if it is set or not. This attribute can be applied to any valid backing field and inherits from the [BitFieldAttribute] attribute. Boolean fields can be mixed with integer and enum fields without incurring a new unit.

// Constructor for boolean bit-fields
[BooleanFieldAttribute(string name)]

Enum Bit Fields

Enum bit-fields are supported by the [EnumFieldAttribute] helper attribute. This attribute can be applied to any valid backing field and inherits from the [BitFieldAttribute] attribute. The enum type must be passed as a type argument i.e., typeof(MyEnum). Enum fields can be mixed with integer and boolean fields without incurring a new unit.

// Constructor for enum bit-fields
[EnumFieldAttribute(string name, byte size, Type enumType)]

Modifiers

The BitFieldModifiers enum allows alterations to the way that the source generator produces the bit-fields. By default all bit-fields are generated as a public read/write or public readonly properties relative to their backing field's accessibility. The Modifiers field allows control over this and provides the ability to change a bit-field's accessibility and if it is readonly, init only (.NET 6.0) and/or required (.NET 7.0).

For valid fixed-width integral backing fields, generated LSB and MSB getters and setters use direct masks, shifts, and byte-order operations. Eligible memory, span, array, and byte inline-array layouts also use specialized generated accessors. Other layouts continue through BitPrimitives to preserve their established semantics.

Generated Batch Accessors

Set GenerateBatchAccessors = true to generate static batch helpers from the field names, offsets, widths, types, and bit orders already declared on a bit object:

public enum PacketKind : byte
{
    Data,
    Control,
    Acknowledgement
}

[BitObject(
    BitOrder.LeastSignificant,
    GenerateBatchAccessors = true)]
public partial struct Packet
{
    [BitField("Id", 12, BitFieldType.UInt16)]
    [BooleanField("Enabled")]
    [EnumField("Kind", 3, typeof(PacketKind))]
    private ushort _layout;
}

The generator adds packed and strided overloads such as:

const int recordCount = 100;
Span<byte> records = new byte[recordCount * 2];
Span<ushort> ids = new ushort[recordCount];
Span<bool> enabled = new bool[recordCount];

Packet.ReadIdBatch(records, bitStride: 16, destination: ids);
Packet.ReadEnabledBatch(records, bitStride: 16, destination: enabled);

Packet.WriteIdBatch(records, bitStride: 16, values: ids);

Generated batch methods operate on external Span<byte>/ReadOnlySpan<byte> buffers; they do not construct a bit-object for every record. Methods use the corresponding property accessibility, omit writers for readonly fields, and expose strongly typed enum and Boolean spans. Batch generation is opt-in to avoid adding public methods to existing types. Generated batch helpers always use checked BitBatchPrimitives, even when instance accessors select BitObjectAccessMode.Unsafe.

Unsafe Access Mode

Generated accessors are bounds-checked by default. No changes are required for existing bit objects:

[BitObject(BitOrder.LeastSignificant)]
public partial struct CheckedPacket
{
    [BitField(3)]
    [BitField("Value", 20, BitFieldType.UInt32)]
    private Memory<byte> _buffer;
}

Applications that control every backing buffer and have measured a meaningful benefit can opt an entire bit object into unchecked byte access. Validate the buffer once at the trust boundary, before storing it in the bit object:

[BitObject(
    BitOrder.LeastSignificant,
    AccessMode = BitObjectAccessMode.Unsafe)]
public partial class TrustedPacket
{
    [BitField(3)]
    [BitField("Value", 20, BitFieldType.UInt32)]
    private Memory<byte> _buffer;

    public TrustedPacket(Memory<byte> buffer)
    {
        // A generated UInt32 unsafe accessor can touch an 8-byte window.
        if (buffer.Length < 8)
            throw new ArgumentException("The buffer must contain at least 8 bytes.", nameof(buffer));

        _buffer = buffer;
    }
}

var storage = new byte[8];
var packet = new TrustedPacket(storage);
packet.Value = 0xABCDE;
Console.WriteLine(packet.Value); // 703710

Unsafe access mode affects byte-addressable backing fields only; integral backing fields are unchanged. It permits the generator to remove length and argument validation and use raw references through UnsafeBitPrimitives. Already-specialized layouts retain their checked intrinsic when benchmarks show it is faster.

The minimum accessible window, measured from the byte containing the field's first bit, is based on the declared BitFieldType, not only the logical field size:

Generated field type Required accessible window
Boolean 1 byte
SByte, Byte, Int16, UInt16 4 bytes
Int32, UInt32 8 bytes
Int64, UInt64 16 bytes

For example, a 20-bit UInt32 field still requires an accessible 8-byte window. Account for the field's starting byte as well: if it begins in byte 5, the backing storage must remain accessible through byte 12. These access widths can exceed the bytes occupied by the logical bit-field.

UnsafeBitPrimitives can also be called directly after performing the same validation:

Span<byte> buffer = stackalloc byte[8];
ref byte first = ref MemoryMarshal.GetReference(buffer);

UnsafeBitPrimitives.WriteUInt32LSB(ref first, bitOffset: 3, value: 0xABCDE, bitCount: 20);
uint value = UnsafeBitPrimitives.ReadUInt32LSB(ref first, bitOffset: 3, bitCount: 20);

The direct methods do not validate buffer length, bit offset, or bit count. Prefer generated checked accessors or BitPrimitives unless the caller owns the complete memory-safety contract.

Warning: An undersized, empty, invalid, or concurrently moved backing buffer can cause out-of-bounds reads or writes, data corruption, information disclosure, or process failure. Unsafe mode is never inferred and should not be enabled merely because it benchmarks faster. Checked access remains the supported default for untrusted or variable-sized input.

Note: Currently both the getter and setter share the same accessibility therefore you cannot have public bit-fields with private setters.

Examples

Putting this into action with the following C struct:

struct S
{
    // occupies 2 bytes:
    unsigned char b1 : 3;  // 1st 3 bits (in 1st byte) are b1
    unsigned char    : 2;  // next 2 bits (in 1st byte) are unused "padding"
    unsigned char b2 : 1;  // next 1 bit (in 1st byte) is b2
    unsigned char b3 : 6;  // 6 bits for b3 - doesn't fit into the 1st byte => starts a 2nd
    unsigned char b4 : 2;  // 2 bits for b4 - next (and final) bits in the 2nd byte
};

Converted to its BitsKit representation:

[BitObject(BitOrder.LeastSignificant)]
public partial struct S 
{
    [BitField("b1", 3)]     // 1st 3 bits (in 1st byte) are b1
    [BitField(2)]           // next 2 bits (in 1st byte) are unused "padding"
    [BitField("b2", 1)]     // next 1 bit (in 1st byte) is b2
    private byte _backingField1;
    
    [BitField("b3", 6)]     // 6 bits for b3 - doesn't fit into the 1st byte => use a 2nd
    [BitField("b4", 2)]     // 2 bits for b4 - next (and final) bits in the 2nd byte
    private byte _backingField2;
}

Which produces a new generated partial class containing:

public partial struct S 
{
    public byte b1 { get => ..; set => ..; }; // _backingField1 0..2
    public byte b2 { get => ..; set => ..; }; // _backingField1 5
    public byte b3 { get => ..; set => ..; }; // _backingField2 0..5
    public byte b4 { get => ..; set => ..; }; // _backingField2 6..7
}

Straddling Unit Boundaries

Some C compilers support straddling storage-unit boundaries. An example of this would be the "b3" field in the above example occupying the last 2 bits in the first byte and the first 4 bits in the second byte. BitsKit enforces unit boundaries for integral types however memory types do allow this.

[BitObject(BitOrder.LeastSignificant)]
public unsafe partial struct S 
{
    [BitField("b1", 3)] // 1st 3 bits (in 1st byte) are b1
    [BitField(2)]       // next 2 bits (in 1st byte) are unused "padding"
    [BitField("b2", 1)] // next 1 bit (in 1st byte) is b2
    [BitField("b3", 6)] // next (and final) 2 bits in 1st byte and 1st 4 bits in 2nd byte
    [BitField("b4", 4)] // 4 bits for b4 - next (and final) bits in the 2nd byte
    private fixed byte _backingField[2];
}

Errors

Below are the diagnostics BitsKit produces. Additionally, an ArgumentOutOfRangeException exception is thrown if the bit offset or count exceed the bounds of the backing field/source.

Rule ID Severity Notes
BITSKIT001 Error BitsKit object must be partial
BITSKIT002 Error BitsKit object must not be a nested type
BITSKIT003 Error Cannot infer FieldType
BITSKIT004 Warning Conflicting accessibility modifiers
BITSKIT005 Warning Conflicting setter modifiers
BITSKIT006 Error Enum type argument expected
BITSKIT007 Error Invalid bit-object option
BITSKIT008 Error Unsupported backing-field type
BITSKIT009 Error Raw pointer backing requires unsafe access
BITSKIT010 Error Invalid generated member name
BITSKIT011 Error Generated member conflicts with an existing or generated member
BITSKIT012 Error Bit-field width exceeds its value type
BITSKIT013 Error Bit-field layout exceeds fixed backing storage
BITSKIT014 Error Invalid modifier combination

Generation is isolated per bit object. An invalid declaration reports its own diagnostic without suppressing generated accessors for unrelated valid types. Generated type, namespace, field, enum, and framework references are escaped and fully qualified so legal C# keywords and consumer-defined type names do not corrupt generated source.

IO Classes

There are a number of IO types available under the BitsKit.IO namespace built to sequentially read/write regions of bit data. Each of these classes expose all the BitPrimitives methods whilst supporting seeking and writing in-place.

BitReader/BitWriter - Classes for reading/writing to a byte array. MemoryBitReader/MemoryBitWriter - Ref structs for reading/writing to a Span<byte>. BitStreamReader/BitStreamWriter - Classes for reading/writing to a stream.

Notes:

  • The array and span backed types support up to int.MaxValue bits as they use a signed integer for positioning to boost performance. This limits the source to being less than 0x10000000 bytes.
  • BitStreamReader uses a pooled read-ahead buffer for sequential reads. When it is disposed with leaveOpen: true, the underlying seekable stream position is restored to the reader's logical bit position, rounded up to the next byte.
  • BitStreamReader throws EndOfStreamException when the source cannot supply all the bits requested by an operation.
  • BitStreamWriter supports sequential output to forward-only streams. For non-seekable destinations, Position and Length are relative to the bytes written through the writer, while seeking is unavailable.
  • Seeking and writing in-place require a seekable destination. In-place writes also require it to be readable so existing data can be preserved.
  • BitStreamWriter buffers sequential output. Call Flush (or dispose the writer) before reading the new data directly from the underlying stream.

Utility Methods

Additional utilities for common bit processing tasks are provided under the BitsKit.Utilities namespace. Many of these functions have been taken from the awesome Bit Twiddling Hacks page created by Sean Eron Anderson.

BitUtilities.InterleaveBits (See) Interleaves the bits of two integral numbers. This is also known as "Morton numbers" or "Morton codes" e.g.,

static uint InterleaveBits(ushort a, ushort b)

//  a          b          baba_baba_baba
0b00_1111, 0b10_0010 => 0b1000_0101_1101

BitUtilities.MergeBits (See) Merges the bits of two integral numbers according to a mask. If the mask bit is a 0, the bit is taken from a otherwise it is taken from b e.g.,

static uint MergeBits(uint a, uint b, uint mask)

//   a           b          mask        bbbbaaaa
0b10101110, 0b11001010, 0b11110000 => 0b11001110

BitUtilities.NegateBits Negates a range of bits within an integral number e.g.,

static uint NegateBits(uint value, int bitOffset, int bitCount)

//  value   offset count      xxxx
0b1100_1101   4     4    => 0b0011_1101

BitUtilities.ReverseBits (See) Reverses both the bit and byte order (endian) of an integral number e.g.,

static uint ReverseBits(uint value)

//   a        b             b        a
//7......0 7......0      0......7 0......7
0b11001000_00111011 => 0b11011100_00010011

BitUtilities.SwapBits (See) Swaps the positions of two ranges of bits within an integral number e.g.,

static uint SwapBits(uint value, int offsetA, int offsetB, int bitCount)

//  value   offsetA offsetB bitCount       bbaa
0b1100_1101    4       6       2      => 0b0011_1101

ZigZag An integer encoding used to convert signed integers to unsigned integers whilst maintaining a relative sized bit count. This is achieved by making the least significant bit the sign bit thus making the bit count proportional to the magnitude. This encoding is particularly useful for deltas with a small range e.g.,

static uint Encode(int value)
static int Decode(uint value)

//   -2         -1         0         1         2
// Two's complement
0b11111110 0b11111111 0b00000000 0b00000001 0b00000010
// ZigZag Encoded
0b00000011 0b00000001 0b00000000 0b00000010 0b00000100

Benchmarks

The benchmark workflow measures the library's features on .NET 10 every week and on demand. The report covers LSB/MSB bit primitives, direct and generated batch operations, generated scalar, memory, and inline-array accessors, and the array-, span-, and stream-backed readers and writers across supported bit widths. Relevant pull requests run one focused dry benchmark to validate the harness. Each run includes a readable, categorized results table in its workflow summary and downloadable Markdown, JSON, logs, and environment metadata for 90 days.

The published Mean values are normalized to one library operation, even though each benchmark processes a larger batch internally for measurement stability. This keeps primitive reads and writes, generated accessors, and the reader/writer types on the same nanoseconds-per-operation scale. Use stable local hardware and attach its generated report when making a performance-regression claim.

Run the complete feature suite locally:

./eng/Run-Benchmarks.ps1

Use BenchmarkDotNet filters for a quicker focused run, for example:

./eng/Run-Benchmarks.ps1 -Category BitStreamReader

About

A C# library for efficient bit-level reading and writing also adding bit field support

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages