feat(pic): add chunked WASM upload and canisterStatus method - #239
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the @dfinity/pic SDK to support automatic chunked WASM uploads for large installs (to avoid the IC ingress size limit) and adds a first-class PocketIc.canisterStatus() API for querying management canister status data.
Changes:
- Add chunked WASM installation path (clear store → parallel chunk uploads →
install_chunked_code) and route large installs automatically. - Add
PocketIc.canisterStatus()plus supporting Candid encode/decode types. - Update dependencies/tooling versions and refresh changelog/docs.
Reviewed changes
Copilot reviewed 10 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
packages/pic/src/pocket-ic.ts |
Adds chunked install implementation + size-based fallback; introduces canisterStatus() method. |
packages/pic/src/management-canister.ts |
Adds Candid types/encoders/decoders for canister_status and chunked install endpoints. |
packages/pic/src/pocket-ic-types.ts |
Adds public TS types for canister status options/results. |
packages/pic/src/util/wasm.ts |
Introduces splitIntoChunks and sha256 helpers. |
packages/pic/src/util/index.ts |
Re-exports new WASM utilities. |
packages/pic/tests/src/util/wasm.spec.ts |
Unit tests for chunking and hashing utilities. |
packages/pic/tests/src/pocket-ic.spec.ts |
Integration coverage for normal vs chunked installs + canisterStatus() verification. |
package.json |
Updates devDependencies and removes overrides. |
pnpm-lock.yaml |
Lockfile updates reflecting dependency bumps. |
CHANGELOG.md |
Documents new feature + chore changes; fixes “BREAKING CHANGE” header. |
.github/CONTRIBUTING.md |
Minor formatting tweaks around commit instructions. |
Comments suppressed due to low confidence (5)
packages/pic/src/pocket-ic.ts:406
- The chunked-install decision compares only
wasm.byteLength + arg.byteLengthagainst the ingress limit, but the actualinstall_codepayload also includes Candid/record overhead (mode, canister_id, length prefixes, etc.). This can still exceed the limit whenwasm+argis close to the threshold. Consider computingencodeInstallCodeRequest(...).byteLength(or using a safety margin below 2MB) to decide when to fall back to chunked upload.
if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
return this.installCodeChunked({
wasm: new Uint8Array(wasm),
arg: new Uint8Array(arg),
canisterId,
mode: { install: null },
sender,
targetSubnetId,
});
}
packages/pic/src/pocket-ic.ts:476
- Same as
installCode: the decision to use chunked install is based onwasm.byteLength + arg.byteLength, but the ingress limit applies to the fully encodedinstall_codemessage. Using encoded payload size (or a conservative margin) would avoid borderline cases whereinstall_codestill exceeds the limit.
if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
return this.installCodeChunked({
wasm: new Uint8Array(wasm),
arg: new Uint8Array(arg),
canisterId,
mode: { reinstall: null },
sender,
});
}
packages/pic/src/pocket-ic.ts:542
- Same issue as the other install paths: the fallback to chunked upload uses
wasm.byteLength + arg.byteLength, which ignores Candid encoding overhead for the install_code message. Consider basing the comparison on the encoded payload size (or leaving headroom) to prevent occasional ingress-limit failures near the threshold.
if (wasm.byteLength + arg.byteLength > MAX_INSTALL_CODE_PAYLOAD_SIZE) {
return this.installCodeChunked({
wasm: new Uint8Array(wasm),
arg: new Uint8Array(arg),
canisterId,
mode: { upgrade: optional(upgradeModeOptions) },
sender,
});
}
packages/pic/src/pocket-ic.ts:1764
- Variable name
encodeInstallChunkedCodeRequestPayloadreads like a function and is easy to confuse withencodeInstallChunkedCodeRequest. Renaming it to something likeinstallChunkedCodePayloadwould improve readability (same applies to other request/payload locals if you want consistency).
const encodeInstallChunkedCodeRequestPayload =
encodeInstallChunkedCodeRequest({
mode,
target_canister: canisterId,
store_canister: [],
chunk_hashes_list: chunkHashes,
wasm_module_hash: sha256(wasm),
arg,
sender_canister_version: [],
});
package.json:65
package.jsonremoves theoverridesblock, butpnpm-lock.yamlstill contains anoverrides:section (e.g. forpath-to-regexp/esbuild). If overrides are truly no longer needed, regenerate the lockfile so it no longer pins them; otherwise, keep the overrides inpackage.jsonto avoid config drift.
"npm-run-all": "^4.1.5",
"prettier": "3.8.1",
"ts-node": "^10.9.2",
"typescript": "^5.9.3",
"vitest": "^4.0.18"
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 12 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
packages/pic/src/pocket-ic.ts:640
- The
canisterStatusJSDoc example callspic.canisterStatus({ canisterId })without specifyingsender, butcanister_statusis controller-only on the IC management canister. This example is likely to reject for most real canisters unless the anonymous principal is a controller; consider updating the example to show passing a controller principal (or creating the canister with the same identity as controller).
* @example
* ```ts
* import { Principal } from '@icp-sdk/core/principal';
* import { PocketIc, PocketIcServer } from '@dfinity/pic';
*
* const canisterId = Principal.fromUint8Array(new Uint8Array([0]));
*
* const picServer = await PocketIcServer.start();
* const pic = await PocketIc.create(picServer.getUrl());
*
* const status = await pic.canisterStatus({ canisterId });
*
* await pic.tearDown();
* await picServer.stop();
* ```
packages/pic/tests/src/pocket-ic.spec.ts:40
padWasmassumes there is always room for an 8-byte custom section header (contentSize = paddingSize - 8). IftargetSizeis only slightly larger thanwasm.byteLength(paddingSize < 8),contentSizebecomes negative and the generated section will be malformed (and likely produce invalid WASM). Consider either enforcing a minimum paddingSize (e.g., return original wasm unless paddingSize >= 8) or implementing a header that can fit in smaller padding sizes.
function padWasm(wasm: Uint8Array, targetSize: number): Uint8Array {
const paddingSize = targetSize - wasm.byteLength;
if (paddingSize <= 0) return wasm;
// Custom section: id (1 byte) + size as fixed u32 LEB128 (5 bytes) +
// name length (1 byte) + name "p" (1 byte) + content
const contentSize = paddingSize - 8;
const section = new Uint8Array(paddingSize);
section[0] = 0x00; // custom section id
// Encode content size + 2 (name length + name) as 5-byte LEB128
let size = contentSize + 2;
for (let i = 1; i <= 5; i++) {
section[i] = (size & 0x7f) | (i < 5 ? 0x80 : 0);
size >>>= 7;
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 12 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@adamspofford-dfinity re-requesting review as a new change was needed for the tests to pass after merging to the latest main 9289887 |
Summary
Support large WASM installation in pic-js. Users need to be able to deploy any WASM module, including large ones that exceed the 2 MB IC ingress message size limit. Without this, large canisters simply fail to install.
SDK-2149
Chunked WASM upload
When the combined size of WASM + install args exceeds 2 MB,
installCode,reinstallCode,upgradeCanister, andsetupCanisternow automatically use the chunked upload path:clear_chunk_store→upload_chunk(parallel batches of 12, 1 MB each) →install_chunked_codeThis is transparent to the caller, no API changes required. Existing calls automatically switch to the chunked path when needed.
Implementation follows the same pattern used in production by other projects (e.g., Juno's IC management test utils).
canisterStatusmethodNew first-class method on
PocketIcto query canister status (status, settings, module hash, cycles, memory size, query stats, etc.).IC management canister interface
All chunk-related calls and
canister_statusfollow the IC interface spec candid definitions.Test plan
pnpm run buildpnpm run test:picpnpm audit— no vulnerabilitiespnpm run format:check— clean