Skip to content

feat(pic): add chunked WASM upload and canisterStatus method - #239

Merged
nikosxenakis merged 9 commits into
mainfrom
nikosxenakis/SDK-2149
Mar 6, 2026
Merged

feat(pic): add chunked WASM upload and canisterStatus method#239
nikosxenakis merged 9 commits into
mainfrom
nikosxenakis/SDK-2149

Conversation

@nikosxenakis

@nikosxenakis nikosxenakis commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Support large WASM installation in pic-js. Users need to be able to deploy any WASM module, including large ones that exceed the 2 MB IC ingress message size limit. Without this, large canisters simply fail to install.

SDK-2149

Chunked WASM upload

When the combined size of WASM + install args exceeds 2 MB, installCode, reinstallCode, upgradeCanister, and setupCanister now automatically use the chunked upload path:
clear_chunk_storeupload_chunk (parallel batches of 12, 1 MB each) → install_chunked_code

This is transparent to the caller, no API changes required. Existing calls automatically switch to the chunked path when needed.

Implementation follows the same pattern used in production by other projects (e.g., Juno's IC management test utils).

canisterStatus method

New first-class method on PocketIc to query canister status (status, settings, module hash, cycles, memory size, query stats, etc.).

IC management canister interface

All chunk-related calls and canister_status follow the IC interface spec candid definitions.

Test plan

  • pnpm run build
  • pnpm run test:pic
  • pnpm audit — no vulnerabilities
  • pnpm run format:check — clean

@nikosxenakis
nikosxenakis requested a review from a team as a code owner March 4, 2026 12:17
Copilot AI review requested due to automatic review settings March 4, 2026 12:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the @dfinity/pic SDK to support automatic chunked WASM uploads for large installs (to avoid the IC ingress size limit) and adds a first-class PocketIc.canisterStatus() API for querying management canister status data.

Changes:

  • Add chunked WASM installation path (clear store → parallel chunk uploads → install_chunked_code) and route large installs automatically.
  • Add PocketIc.canisterStatus() plus supporting Candid encode/decode types.
  • Update dependencies/tooling versions and refresh changelog/docs.

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/pic/src/pocket-ic.ts Adds chunked install implementation + size-based fallback; introduces canisterStatus() method.
packages/pic/src/management-canister.ts Adds Candid types/encoders/decoders for canister_status and chunked install endpoints.
packages/pic/src/pocket-ic-types.ts Adds public TS types for canister status options/results.
packages/pic/src/util/wasm.ts Introduces splitIntoChunks and sha256 helpers.
packages/pic/src/util/index.ts Re-exports new WASM utilities.
packages/pic/tests/src/util/wasm.spec.ts Unit tests for chunking and hashing utilities.
packages/pic/tests/src/pocket-ic.spec.ts Integration coverage for normal vs chunked installs + canisterStatus() verification.
package.json Updates devDependencies and removes overrides.
pnpm-lock.yaml Lockfile updates reflecting dependency bumps.
CHANGELOG.md Documents new feature + chore changes; fixes “BREAKING CHANGE” header.
.github/CONTRIBUTING.md Minor formatting tweaks around commit instructions.
Comments suppressed due to low confidence (5)

packages/pic/src/pocket-ic.ts:406

  • The chunked-install decision compares only wasm.byteLength + arg.byteLength against the ingress limit, but the actual install_code payload also includes Candid/record overhead (mode, canister_id, length prefixes, etc.). This can still exceed the limit when wasm+arg is close to the threshold. Consider computing encodeInstallCodeRequest(...).byteLength (or using a safety margin below 2MB) to decide when to fall back to chunked upload.
    if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
      return this.installCodeChunked({
        wasm: new Uint8Array(wasm),
        arg: new Uint8Array(arg),
        canisterId,
        mode: { install: null },
        sender,
        targetSubnetId,
      });
    }

packages/pic/src/pocket-ic.ts:476

  • Same as installCode: the decision to use chunked install is based on wasm.byteLength + arg.byteLength, but the ingress limit applies to the fully encoded install_code message. Using encoded payload size (or a conservative margin) would avoid borderline cases where install_code still exceeds the limit.
    if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
      return this.installCodeChunked({
        wasm: new Uint8Array(wasm),
        arg: new Uint8Array(arg),
        canisterId,
        mode: { reinstall: null },
        sender,
      });
    }

packages/pic/src/pocket-ic.ts:542

  • Same issue as the other install paths: the fallback to chunked upload uses wasm.byteLength + arg.byteLength, which ignores Candid encoding overhead for the install_code message. Consider basing the comparison on the encoded payload size (or leaving headroom) to prevent occasional ingress-limit failures near the threshold.
    if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
      return this.installCodeChunked({
        wasm: new Uint8Array(wasm),
        arg: new Uint8Array(arg),
        canisterId,
        mode: { upgrade: optional(upgradeModeOptions) },
        sender,
      });
    }

packages/pic/src/pocket-ic.ts:1764

  • Variable name encodeInstallChunkedCodeRequestPayload reads like a function and is easy to confuse with encodeInstallChunkedCodeRequest. Renaming it to something like installChunkedCodePayload would improve readability (same applies to other request/payload locals if you want consistency).
    const encodeInstallChunkedCodeRequestPayload =
      encodeInstallChunkedCodeRequest({
        mode,
        target_canister: canisterId,
        store_canister: [],
        chunk_hashes_list: chunkHashes,
        wasm_module_hash: sha256(wasm),
        arg,
        sender_canister_version: [],
      });

package.json:65

  • package.json removes the overrides block, but pnpm-lock.yaml still contains an overrides: section (e.g. for path-to-regexp / esbuild). If overrides are truly no longer needed, regenerate the lockfile so it no longer pins them; otherwise, keep the overrides in package.json to avoid config drift.
    "npm-run-all": "^4.1.5",
    "prettier": "3.8.1",
    "ts-node": "^10.9.2",
    "typescript": "^5.9.3",
    "vitest": "^4.0.18"
  }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

packages/pic/src/pocket-ic.ts:640

  • The canisterStatus JSDoc example calls pic.canisterStatus({ canisterId }) without specifying sender, but canister_status is controller-only on the IC management canister. This example is likely to reject for most real canisters unless the anonymous principal is a controller; consider updating the example to show passing a controller principal (or creating the canister with the same identity as controller).
   * @example
   * ```ts
   * import { Principal } from '@icp-sdk/core/principal';
   * import { PocketIc, PocketIcServer } from '@dfinity/pic';
   *
   * const canisterId = Principal.fromUint8Array(new Uint8Array([0]));
   *
   * const picServer = await PocketIcServer.start();
   * const pic = await PocketIc.create(picServer.getUrl());
   *
   * const status = await pic.canisterStatus({ canisterId });
   *
   * await pic.tearDown();
   * await picServer.stop();
   * ```

packages/pic/tests/src/pocket-ic.spec.ts:40

  • padWasm assumes there is always room for an 8-byte custom section header (contentSize = paddingSize - 8). If targetSize is only slightly larger than wasm.byteLength (paddingSize < 8), contentSize becomes negative and the generated section will be malformed (and likely produce invalid WASM). Consider either enforcing a minimum paddingSize (e.g., return original wasm unless paddingSize >= 8) or implementing a header that can fit in smaller padding sizes.
function padWasm(wasm: Uint8Array, targetSize: number): Uint8Array {
  const paddingSize = targetSize - wasm.byteLength;
  if (paddingSize <= 0) return wasm;

  // Custom section: id (1 byte) + size as fixed u32 LEB128 (5 bytes) +
  //                 name length (1 byte) + name "p" (1 byte) + content
  const contentSize = paddingSize - 8;
  const section = new Uint8Array(paddingSize);
  section[0] = 0x00; // custom section id
  // Encode content size + 2 (name length + name) as 5-byte LEB128
  let size = contentSize + 2;
  for (let i = 1; i <= 5; i++) {
    section[i] = (size & 0x7f) | (i < 5 ? 0x80 : 0);
    size >>>= 7;
  }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/pic/src/util/wasm.ts
Comment thread packages/pic/src/pocket-ic.ts
Comment thread packages/pic/src/pocket-ic.ts Outdated
Comment thread packages/pic/src/pocket-ic.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/pic/tests/src/pocket-ic.spec.ts
@nikosxenakis
nikosxenakis enabled auto-merge March 6, 2026 07:53
@nikosxenakis
nikosxenakis added this pull request to the merge queue Mar 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Mar 6, 2026
@nikosxenakis

Copy link
Copy Markdown
Contributor Author

@adamspofford-dfinity re-requesting review as a new change was needed for the tests to pass after merging to the latest main 9289887

@nikosxenakis
nikosxenakis added this pull request to the merge queue Mar 6, 2026
Merged via the queue into main with commit ab68d2c Mar 6, 2026
14 checks passed
@nikosxenakis
nikosxenakis deleted the nikosxenakis/SDK-2149 branch March 6, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants