refactor(dapi,dpp)!: move dapi-client and Identifier off Buffer to Uint8Array

[PastaPastaPasta]

Summary

Replaces Buffer with Uint8Array across @dashevo/dapi-client's public API. Rewrites @dashevo/wasm-dpp's Identifier to extend Uint8Array directly (instead of Buffer) and removes Buffer.from from the wasm-dpp module loader. After this PR, both packages' module-load and core construction paths are Buffer-free; only the deprecated Identifier#toBuffer() shim still requires a Buffer global at call time. The SPV path through dashcore-lib will become Buffer-free once dashpay/dashcore-lib#315 lands and dapi-client picks up the new release.

This is PR 2 of 3 in the stacked series. PR 1 (#3679) is merged.

Why

Removes Buffer from the dapi-client and wasm-dpp surfaces so consumers don't need a Buffer polyfill in browser bundles. Buffer is Node-specific; Uint8Array works everywhere.

What changes

@dashevo/dapi-client

• New lib/utils/bytes.js: hexToBytes, bytesToHex, base64ToBytes, bytesToBase64, concatBytes, bytesEqual (CJS exports). Input-validated with informative TypeErrors; uses String.prototype.slice not the deprecated substr.
• Exposed via DAPIClient.bytes so the migration path is reachable from the package's public surface (no deep imports).
• All Buffer.* call sites in lib/ converted to Uint8Array per the translation table below.
• createGrpcTransportError: explicit dual-format handling that the previous Buffer.from(x, 'base64') provided implicitly — typeof x === 'string' ? base64ToBytes(x) : new Uint8Array(x) for the three metadata-bin fields (grpc-js sends raw bytes; grpc-web sends base64 strings).
• One production exception remains where Buffer.from is retained: BlockHeadersReader → new BlockHeader(Buffer.from(header)) because dashcore-lib's BufferReader still requires Buffer. Tracked upstream: dashpay/dashcore-lib#315. Once that lands and we bump the dep here, this wrap goes away.

@dashevo/wasm-dpp

• Identifier extends Uint8Array instead of Buffer. Buffer.isBuffer(id) now returns false; id instanceof Uint8Array returns true.
• New id.toBytes() returns a plain Uint8Array copy.
• id.toBuffer() retained as @deprecated compatibility shim — calls Buffer.from(this). Requires Buffer only at call time, no longer at module load.
• id.toString(encoding) now handles 'base58'/'base64'/'hex' explicitly via inline byte helpers — no Buffer method dependency. Throws IdentifierError: Unsupported encoding: ... for anything else.
• Identifier.from(value, encoding) accepts Uint8Array directly and string with 'base58' (default) or 'base64'. The encoding parameter type is narrowed to only the supported values.
• id.equals(other) byte-by-byte loop, no Buffer methods.
• wasm-dpp module loader (lib/index.ts) replaces Buffer.from(wasmBase, 'base64') with an inline atob-based decode. loadDpp() no longer requires a Buffer global.

@dashevo/wallet-lib

• One production callsite: IdentitySyncWorker.js:98 migrates Buffer.isBuffer(identityBuffer) → identityBuffer instanceof Uint8Array. Comment updated.

Test migrations

All internal test code that called .toBuffer() on an Identifier-derived value is migrated to .toBytes() (36 test files, ~100 mechanical one-line swaps). The deprecated .toBuffer() shim is intentionally exercised in one place — wasm-dpp/test/unit/Identifier.spec.js #toBuffer describe block — with an explanatory comment.

README

New "Browser usage" section in dapi-client documenting that response fields are Uint8Array, that DAPIClient.bytes is the conversion helper, and that one internal path (BlockHeadersProvider via dashcore-lib) still requires a Buffer polyfill until upstream changes land.

Breaking changes

@dashevo/dapi-client public API

Before	After
response.field.toString('hex')	DAPIClient.bytes.bytesToHex(response.field)
response.field.toString('base64')	DAPIClient.bytes.bytesToBase64(response.field)
Buffer.isBuffer(response.field)	response.field instanceof Uint8Array
response.field.slice(a, b)	response.field.subarray(a, b)

In Node, Buffer extends Uint8Array, so anything that just reads bytes (passes to crypto, writes to a stream, length checks, indexed access) continues to work identically.

@dashevo/wasm-dpp Identifier

Before	After
Buffer.isBuffer(id) returned true	now returns false
id instanceof Buffer returned true	now returns false
id.readUInt32LE(...) etc. (Buffer methods)	gone — id no longer has Buffer-only methods
id.toBuffer()	still works (deprecated); prefer id.toBytes()
id.toString('hex') / id.toString('base64')	still works (now implemented natively)
id.toString('utf8'), 'ascii', etc.	throws IdentifierError: Unsupported encoding (was silently inherited from Buffer)
Error message 'Identifier expects Buffer'	'Identifier expects Uint8Array'

The remaining transitive Buffer dependency in wasm-dpp is bs58 (via safe-buffer) — eliminating it requires replacing bs58 with a Uint8Array-native base58 library and is out of scope here.

Test plan

• CI: JS packages (@dashevo/dapi-client) / Tests
• CI: JS packages (@dashevo/wasm-dpp) / Tests
• CI: JS packages (@dashevo/wallet-lib) / Tests
• CI: JS packages (dash) / Tests

Stack

• PR 1 — Cleanup (merged)
• ➡️ PR 2 — Buffer → Uint8Array (this PR)
• PR 3 — ESM conversion (dapi-client, wallet-lib, js-dash-sdk + platform-test-suite)

Summary by CodeRabbit

• New Features

  • Added a bytes utility on the client for hex/base64 conversions, concatenation, and comparisons.
  • Improved browser compatibility with native Uint8Array-based handling.

• Breaking Changes

  • Binary inputs/outputs and related method signatures now use Uint8Array instead of Buffer; update callers accordingly.

• Documentation

  • Added browser usage guidance for Uint8Array handling and optional Buffer polyfills.

[coderabbitai]

📝 Walkthrough

Walkthrough

Project-wide migration from Buffer to Uint8Array for binary data. Adds shared byte helpers, updates core/platform methods and transport, exposes DAPIClient.bytes, refactors wasm-dpp Identifier and wasm loader, adjusts wallet worker guard, and updates tests/docs accordingly.

Changes

js-dapi-client Uint8Array migration

Layer / File(s)	Summary
Bytes utilities and DAPIClient export packages/js-dapi-client/lib/utils/bytes.js, packages/js-dapi-client/lib/index.js, packages/js-dapi-client/README.md	Adds Uint8Array byte helpers, exposes as DAPIClient.bytes, and documents browser usage and Buffer polyfill notes.
Core methods and SML provider use Uint8Array packages/js-dapi-client/lib/SimplifiedMasternodeListProvider/..., packages/js-dapi-client/lib/methods/core/*	Returns/handles Uint8Array for blocks, status, transactions, and SML diffs; uses hex/base64 helpers.
Platform response classes return Uint8Array packages/js-dapi-client/lib/methods/platform/*Response.js, packages/js-dapi-client/lib/methods/platform/response/Proof.js, packages/js-dapi-client/lib/methods/platform/getStatus/GetStatusResponse.js, packages/js-dapi-client/lib/methods/platform/waitForStateTransitionResult/*	Stores/returns Uint8Array in response objects and uses bytesToHex for string fields.
Platform request factories accept Uint8Array packages/js-dapi-client/lib/methods/platform/*Factory.js, packages/js-dapi-client/lib/methods/platform/getProtocolVersionUpgradeVoteStatus/*	Accepts/clones Uint8Array inputs and converts hex/base64 via helpers for request building.
Transport error decoding and test fixtures packages/js-dapi-client/lib/transport/GrpcTransport/createGrpcTransportError.js, packages/js-dapi-client/lib/test/fixtures/*, packages/js-dapi-client/lib/test/mocks/*	Decodes metadata as base64 or raw bytes; fixtures/mocks produce Uint8Array with helpers.
Core integration/unit tests use Uint8Array packages/js-dapi-client/test/integration/methods/core/*, packages/js-dapi-client/test/unit/methods/core/*	Rewrites tests to encode bytes with TextEncoder and assert Uint8Array.
Platform integration/unit tests use Uint8Array packages/js-dapi-client/test/integration/methods/platform/*, packages/js-dapi-client/test/unit/methods/platform/*	Updates tests to pass/expect Uint8Array and use bytes helpers.
wallet-lib IdentitySyncWorker accepts Uint8Array packages/wallet-lib/src/plugins/Workers/IdentitySyncWorker.js	Type guard updated to Uint8Array for identity buffer.

wasm-dpp Uint8Array refactor

Layer / File(s)	Summary
Identifier class migrates from Buffer to Uint8Array packages/wasm-dpp/lib/identifier/Identifier.ts	Identifier now stores bytes as Uint8Array, adds toBytes, deprecates toBuffer, updates encoding/equality/APIs.
WASM loader decodes base64 to Uint8Array packages/wasm-dpp/lib/index.ts	Adds base64ToBytes and uses it to construct wasm bytes for init.
wasm-dpp tests switch to toBytes and Uint8Array packages/wasm-dpp/test/**/*	Updates affected tests to compare via toBytes and use Uint8Array inputs/expectations.

Sequence Diagram(s)

(skipped)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Suggested reviewers

• QuantumExplorer

Poem

A rabbit nibbles bytes so fine,
From Buffer fields to typed-array shine.
Hex and base64 now play,
In Uint8 songs along the way.
Identifier hops, new trails it writes—
CBOR meadows, wasm lights. 🐇✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/esm-2-uint8array

[github-actions]

✅ DashSDKFFI.xcframework built for this PR.

• Workflow run: https://github.com/dashpay/platform/actions/runs/26121048045
• Artifacts: DashSDKFFI.xcframework (folder), DashSDKFFI.xcframework.zip, xc_checksum.txt

SwiftPM (host the zip at a stable URL, then use):

.binaryTarget(
  name: "DashSDKFFI",
  url: "https://your.cdn.example/DashSDKFFI.xcframework.zip",
  checksum: "bfa0ce328feabc7b4f6d66a72c6cdc47229bcc9ec155eaa86e41747b372d302b"
)

Xcode manual integration:

• Download 'DashSDKFFI.xcframework' artifact from the run link above.
• Drag it into your app target (Frameworks, Libraries & Embedded Content) and set Embed & Sign.
• If using the Swift wrapper package, point its binaryTarget to the xcframework location or add the package and place the xcframework at the expected path.

[thepastaclaw]

Code Review

Both prior findings are confirmed FIXED at HEAD 4ed8428. Remaining in-scope items are: (1) inconsistent dashcore-lib boundary in wallet-lib getTransaction.js — the new Uint8Array transaction payload is passed straight into new Transaction(...), while every other in-tree call site at the same boundary still normalizes via Buffer.from(...); (2) the protobuf bytes-decoding pattern new Uint8Array(proto.getX()) used in several PR-touched response classes will silently corrupt bytes under grpc-web (the very target this PR enables) — the author already fixed one instance in GetProtocolVersionUpgradeVoteStatusResponse.js with a comment explaining the issue; (3) a JSDoc/.d.ts contract mismatch on getIdentityByPublicKeyHash.js. None are strictly blocking — codex-ffi-engineer verified dashcore-lib 0.22 accepts Uint8Array via BufferUtil — but the inconsistency and grpc-web pattern should be addressed.

🟡 2 suggestion(s) | 💬 1 nitpick(s)

1 additional finding(s) omitted (not in diff).

Out-of-scope follow-up suggestions (2)

These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.

• Audit all dapi-client response classes for grpc-web byte-decoding compatibility — The new Uint8Array(proto.getXxx()) pattern that the author fixed in GetProtocolVersionUpgradeVoteStatusResponse.js exists across many sibling response classes (GetStatusResponse.js nodeId/proTxHash/latestBlockHash/latestAppHash/earliestBlockHash/earliestAppHash, WaitForStateTransitionResultResponse.js, response/Proof.js grovedbProof/quorumHash/signature, core/getTransaction/GetTransactionResponse.js blockHash, getBlockchainStatusFactory.js). Some of these are touched by this PR; the broader audit (and choosing between _asU8() everywhere vs. a base64-string-aware helper) is bigger than this PR.
  • Follow-up: Open a separate issue to systematically migrate all dapi-client bytes-field decoders to _asU8() getters (or a wrapper that handles both representations).
• Verify wallet-lib → dashcore-lib byte boundaries now that responses are Uint8Array — packages/wallet-lib/src/transport/DAPIClientTransport/methods/getBlockByHash.js and getBlockByHeight.js pass the dapi-client response (now Uint8Array) straight into new Block(...) from @dashevo/dashcore-lib. The PR explicitly preserved Buffer.from(header) at the analogous BlockHeadersReader boundary. Worth confirming whether dashcore-lib.Block accepts a Uint8Array, or whether explicit Buffer.from(...) wraps should be added for consistency with the rest of the codebase.
  • Follow-up: Audit wallet-lib → dashcore-lib byte boundaries (Block, BlockHeader, Transaction inputs) and either standardize on Buffer.from(...) wrappers or wait for upstream dashcore-lib#315 to widen the inputs.

[thepastaclaw]

🟡 Suggestion: getTransaction() passes raw Uint8Array to dashcore-lib.Transaction while sibling call sites still normalize via Buffer.from(...)

@dashevo/dapi-client now constructs GetTransactionResponse.transaction as a plain Uint8Array (packages/js-dapi-client/lib/methods/core/getTransaction/GetTransactionResponse.js:78), and this wrapper hands the result directly to new Transaction(response.getTransaction()). Every other raw-transaction entry point in this same package still routes through Buffer.from(...) before crossing into dashcore-lib: packages/wallet-lib/src/plugins/Workers/TransactionsSyncWorker/utils.js:56 does new Transaction(Buffer.from(rawTransaction)), and packages/wallet-lib/src/transport/FixtureTransport/methods/getTransaction.js:6 does new Transaction(Buffer.from(txFile.transaction, 'hex')). The PR also intentionally preserves Buffer.from(...) at the analogous BlockHeadersReader boundary. The dapi-client unit tests still build GetTransactionResponse with Buffer values, so the new typed-array runtime shape isn't actually exercised by tests. Either wrap with Buffer.from(...) here for consistency and safety, or drop the wraps at the other two sites with a comment.

Suggested change

	transaction: new Transaction(response.getTransaction()),
	transaction: new Transaction(Buffer.from(response.getTransaction())),

_{source: ['codex']}

[thepastaclaw]

🟡 Suggestion: new Uint8Array(proto.getIdentity()) will silently corrupt bytes under grpc-web — use getIdentity_asU8()

The google-protobuf JS binding returns bytes fields either as Uint8Array (grpc-js) or as a base64 string (grpc-web, depending on deserialization). The author already addressed this exact boundary at packages/js-dapi-client/lib/methods/platform/getProtocolVersionUpgradeVoteStatus/GetProtocolVersionUpgradeVoteStatusResponse.js:44-48 by switching to getProTxHash_asU8(), with a comment stating the default getter would yield a base64 string under grpc-web and new Uint8Array(string) does NOT base64-decode it. Same issue applies here and in GetIdentityByPublicKeyHashResponse.js:29-32, GetStatusResponse.js:96-106, WaitForStateTransitionResultResponse.js:41, response/Proof.js:51-53, and core/getTransaction/GetTransactionResponse.js:78-79. Since this PR's stated purpose is to enable browser/grpc-web bundles, the _asU8() accessor should be applied consistently at the bytes-field boundaries this PR touches. Granted, the pre-PR Buffer.from(string) was also wrong in this scenario, but this PR's whole point is to make these paths actually work in the browser — so the fix belongs here.

Suggested change

	return new GetIdentityResponse(
	Buffer.from(proto.getV0().getIdentity()),
	new Uint8Array(proto.getV0().getIdentity()),
	metadata,
	proof,
	);
	return new GetIdentityResponse(
	proto.getV0().getIdentity_asU8(),
	metadata,
	proof,
	);

_{source: ['claude']}