Skip to content

Latest commit

 

History

History
137 lines (124 loc) · 34 KB

File metadata and controls

137 lines (124 loc) · 34 KB

AGENTS Guide for floccus

Note: All AI contributions will be carefully reviewed by the project maintainers before being merged.

Scope and source files

  • This file documents discoverable project behavior for coding agents.
  • AI-instruction scan performed with glob **/{.github/copilot-instructions.md,AGENT.md,AGENTS.md,CLAUDE.md,.cursorrules,.windsurfrules,.clinerules,.cursor/rules/**,.windsurf/rules/**,.clinerules/**,README.md}.
  • Result: only README.md matched (no existing agent-specific rules files were found).

Big picture architecture

  • floccus is a cross-platform bookmarks sync engine with two runtimes: browser extension and Capacitor mobile app.
  • Entrypoints are minimal: src/entries/background-script.js (browser controller), src/entries/options.js (web UI), src/entries/native.js (native UI), src/entries/test.js (in-extension tests).
  • Runtime abstraction is via src/lib/Controller.ts: browser UI talks to service worker/runtime messages; native uses direct controller implementation.
  • Sync orchestration is centered in src/lib/Account.ts:
    • creates adapter + local tree + storage
    • runs strategy (default / merge / unidirectional)
    • persists cache, mappings, and continuation state
    • applies failsafes and error normalization
  • Core sync algorithm lives in src/lib/strategies/Default.ts (multi-stage diff/reconcile/execute pipeline, resumable from an incrementally persisted continuation — see "Continuation persistence and resume").

Data flow and boundaries

  • Flow: UI action/event -> BrowserController/NativeController -> Account.sync() -> strategy -> local tree + server adapter.
  • The native UI's own reads are a separate path: src/ui/store/native/actions.js keeps only the account's folders in the Vuex store (state.folderTree, no bookmarks in it, because the hierarchy is needed synchronously while rendering) and answers everything else with NativeTreeQuery -- LOAD_CHILDREN, LOAD_TAGS, SEARCH_ITEMS, FIND_BOOKMARK_BY_URL return their result to the component instead of committing it. Every write action re-commits a freshly built folder tree, which is what tells Tree.vue to re-query its children, tags and search results.
  • Storage is per-account and platform-specific:
    • browser: src/lib/browser/BrowserAccountStorage.js — browser.storage.local for account data, the sync cache (bookmarks[<id>].cache), the mappings (bookmarks[<id>].mappings) and the logs; the continuation lives in IndexedDB (BrowserContinuationStore, see below), with bookmarks[<id>].continuation only as fallback/legacy location.
    • native: src/lib/native/NativeAccountStorage.js — @capacitor/preferences only for account data (accounts), lastFolders and the legacy continuation key; the local tree, the mappings, the sync cache, the continuation and the logs are SQLite rows (see "Native SQLite storage").
  • Both storages offer setEntry (write) and changeEntry (read-modify-write), each holding the same per-key AsyncLock. Use changeEntry only when the new value is derived from the old one (accounts, lastFolders); a caller that replaces an entry wholesale — the browser cache blob, the continuation blob — must use setEntry, because changeEntry reads and parses the previous value before discarding it, and during a sync those entries can be megabytes. The two write the same representation, so either can read the other (getEntry also still parses legacy stringified values).
  • Adapter implementations are server boundary points under src/lib/adapters/ (Nextcloud, WebDAV, Git, Dropbox, Google Drive, Linkwarden, Karakeep, Fake).

Native SQLite storage

  • Since #2357 the native local tree and the sync mappings live in a shared SQLite database (@capacitor-community/sqlite) instead of JSON blobs in @capacitor/preferences; the sync cache, the continuation and the logs have followed.
  • src/lib/native/NativeDatabase.ts owns the single connection, the schema (account_meta, folders, bookmarks, cache_folders, cache_bookmarks, continuations, continuation_actions, mappings, logs) and an AsyncLock that serializes every query/batch — the plugin accepts concurrent calls, but their transactions would interleave. CREATE TABLE IF NOT EXISTS does not add columns to an existing table, so later columns are backfilled by addMissingColumns.
  • src/lib/native/NativeTreeStore.ts keeps one account's rows in lockstep with the in-memory tree that NativeTree (still a CachingAdapter) holds, so a change costs one row write instead of re-serializing the whole tree. Writes are queued and flushed in a microtask (enqueue/flushNow); NativeTree#save() therefore only persists hashes and awaits store.flush(), and NativeTree#load() hydrates the tree from rows.
  • Sibling order lives in the position column: CachingAdapter only ever appends to a folder's children, so new rows get an ever increasing position (nextPosition); only orderFolder and bulk imports renumber a folder's children.
  • src/lib/native/NativeTreeQuery.ts is the read-only side of the same rows and what the native UI browses: getFolderTree, getChildren, getTags, findBookmarkByUrl and search are a query each, so opening a folder or searching never materializes the tree. It talks to NativeDatabase directly, so writes still sitting in NativeTreeStore's microtask queue are invisible to it -- a caller that just changed something has to await NativeTree#save() first (the store actions do).
  • Consequently NativeTree hydrates lazily: NativeAccount.get()/create() only construct it, and ensureLoaded() at the top of every tree-touching method reads the rows the first time something actually needs the tree in memory (a sync, a UI edit). Account.getAllAccounts() on startup therefore reads no tree at all.
  • folders.search_text/bookmarks.search_text hold the item's title (plus url and tags for bookmarks) lowercased in JS -- SQLite's lower() and LIKE only fold ASCII, so a query for apfel would otherwise miss Apfel the moment a letter is non-ASCII. Every write path fills the column in (folderSearchText/bookmarkSearchText in NativeTreeStore); rows from before the column existed are filled in once per account by NativeTreeQuery#backfillSearchText, recorded in account_meta.search_backfilled. SQL narrows a search to the rows containing every tag and term somewhere, and the predicates are then applied in JS, which is also what ranks the results.
  • A search query is a mix of tags and free text in any order, parsed by parseSearchQuery (exported from NativeTreeQuery, so Tree.vue reads the same query the search does). Every #tag has to be on the item and narrows the results down further; every free-text term has to turn up in the title, the url or one of the tags, each term judged on its own — so #recipes pasta and a plain recipes pasta both find the bookmark tagged recipes with pasta in its title. Only bookmarks carry tags, so a query naming one returns no folders. Tags and terms containing spaces are quoted (#"read later"), which is what formatSearchToken produces when the tag bar writes a chip into the query — the chips accumulate, each one adding or removing its own #tag and leaving the rest of the query alone. Ranking is by how many of the query's tags the item carries exactly, then by title match quality.
  • src/lib/native/NativeMappingsStore.ts gets the full in-memory mappings on every persist, diffs them against what it knows to be stored, and writes only the difference. Both ids are TEXT plus a *_numeric flag, because a remote id like '007' must not come back as 7; LocalToServer is authoritative and ServerToLocal is rebuilt on load.
  • src/lib/native/NativeCacheStore.ts holds the sync cache in cache_folders/cache_bookmarks (see "Sync cache persistence"). hash_value stores the whole hashValue map as JSON — unlike folders.hash/hash_settings, since the cache never hashes anything itself.
  • src/lib/native/NativeContinuationStore.ts holds the continuation in continuations (one row per account: strategy, created_at, the structure JSON) and continuation_actions (one row per (diff_id, seq)); see "Continuation persistence and resume".
  • src/lib/native/NativeLogStore.ts holds the debug log in logs(seq, message) — the only table not keyed by account — trimmed to the newest LOG_RETENTION (1000, src/lib/Logger.js) lines in the same transaction as each append. Logger buffers lines and appends only the new ones every 3s (Storage.appendLogs); on the browser, BrowserAccountStorage.appendLogs does a read-concat-trim-write of the logs key under the same lock key as setEntry. NativeController clears the stored log on start.
  • The stores migrate once per account from the old preferences keys (bookmarks[<id>].tree, bookmarks[<id>].highestId, bookmarks[<id>].mappings, bookmarks[<id>].cache) and then remove them; the *_migrated flags in account_meta record that this happened. The legacy bookmarks[<id>].continuation is read as a fallback and removed after the first row write; the old logs preferences key is dropped, not migrated.
  • Rows of a deleted account would otherwise stay in the shared database forever, so NativeAccountStorage#deleteAccountData explicitly clears the tree, cache, continuation and mapping rows.
  • src/lib/native/ScreenWakeLock.ts (@capacitor-community/keep-awake) keeps the screen on while a native sync runs, because the WebView is throttled once it goes dark. It is reference counted (acquire/release around account.sync() in NativeController#syncAccount, several accounts may sync at once) and re-asserts itself on visibilitychange, since iOS drops it in the background.

Sync cache persistence

  • src/lib/interfaces/CacheStore.ts (ICacheStore, obtained via IAccountStorage#getCacheStore()) is what CacheTree persists through: native NativeCacheStore (rows), browser BrowserCacheStore (still one JSON blob, bookmarks[<id>].cache; its recorders are no-ops and save() writes toStorageJSON), and NullCacheStore (everything a no-op; the default for new CacheTree()/new CachingTreeWrapper(inner) in unit tests). Production always passes storage.getCacheStore().
  • Every ICacheStore method except load/save/clear only records a change; nothing reaches storage before save(). That keeps the cache in step with the mappings and continuation persisted on the same progress tick — a cache running ahead of the mappings would, after an interrupt, claim items the mappings don't know.
  • CacheTree (a CachingAdapter) forwards each mutation to the store after applying it in memory; createBookmarkAs/createFolderAs create items under the id the live tree allocated, importSubtree mirrors a bulk import, and onHashesInvalidated forwards to store.invalidateHashes. setTree (called at the start of each sync) is a diff on native: only changed rows are upserted and vanished ids deleted.
  • NativeCacheStore#save sends the queued statements as one batch; if that throws, the statements are put back in front of the queue and the error rethrown. Dropping them would leave the rows permanently wrong, because later saves only write what is queued and the next sync reads the rows before setTree could correct them.
  • Dirty tracking: CachingAdapter counts mutations (endMutation/mutated, getMutationCount). Any new mutation path must count, or isCacheDirty() stays false and the change is never persisted. Account#persistProgress saves the cache only when dirty, reading getCacheRevision() before saveCache() and calling markCachePersisted(revision) after, so a change landing during the write leaves it dirty.
  • Unaccepted bookmarks are not filtered on write: Account#sync applies filterUnacceptedBookmarks (CacheTree.ts) right after loading the cache (the browser's toStorageJSON strips them when writing the blob).

Continuation persistence and resume

  • A continuation is no longer one JSON blob. src/lib/Continuation.ts splits it into one row per action, keyed by (diffId, seq), plus one structure record (strategy, createdAt, meta, members, diffIds). SyncProcess#toContinuationUpdateAsync builds an IContinuationUpdate holding only the actions changed since the last acknowledged write; rows of diffs no longer listed in diffIds are deleted by the store. assembleContinuation rebuilds the ISerializedSyncProcess shape fromJSON expects.
  • Stores: browser src/lib/browser/BrowserContinuationStore.ts (IndexedDB floccus_continuations, object stores meta and actions; falls back to the bookmarks[<id>].continuation blob via continuationUpdateToJSON only when IndexedDB is unavailable or a write failed — a failing write switches to the blob and rethrows, so the update stays unacknowledged and the next one is full), native NativeContinuationStore (no fallback). On the browser, getCurrentContinuation prefers the blob only if its createdAt is newer than the rows'.
  • Diff (src/lib/Diff.ts) tracks what to persist per action (seqs, changed/removed/in-flight sets). commit/retract are the only structural mutators; any other in-place change to an action that stays in its diff needs markChanged(action) (today: removeItemFromReorders), or the stale row survives. getPendingChangesAsync iterates copies of actions/seqs, because an executor's getActions() may compact() the diff mid-serialization. markPersisted — via markContinuationPersisted — runs only after a successful write; anything unacknowledged is written again next time.
  • Diff ids carry a per-process random prefix, so a resumed run's diffs never adopt the previous run's rows; its first update writes everything and the stale rows are dropped.
  • Account#persistProgress writes the continuation only for non-atomic servers (atomic adapters rely on cache + mappings), serially under the module-level continuationLock — an older update landing last would put executed actions back into their plan. Account#clearContinuation takes the same lock (else a fire-and-forget tick could land after the clear) and stores null, never a bare { createdAt }, since Account#sync hands any non-null entry to SyncProcess.fromJSON.
  • createdAt is stamped by the stores on every update; Account#sync drops continuations older than half an hour by it. A continuation without one compares as NaN > x, i.e. never stale.
  • Progress ticks are throttled by progressInterval(items) (Default.ts: 0.2ms per item, clamped to 1.5–10s; retuneProgressInterval() rebuilds the throttle once the trees are loaded, so read this.throttledProgressCb at call time). progressCallback persists nothing before the first executed action. Under isTest the throttled tick is a no-op; the continuation is then only persisted at an interrupt or by tests calling account.progressCallback.
  • Which members are persisted is decided by getMembersToPersist(): scan results until stage-1 plans exist, stage-1 plans until stage 2, stage-2 plans until both stage-3 plans exist (stage 3 is built lazily), stage-3 plans/done plans/prelim reorders/actionsPlanned until both localReorders and serverReorders exist, and the reorders always. The stage-3 condition must not be based on actionsDone (reset every run, and lagging behind bulk imports' REORDERs): a member that drops out for one tick loses its rows, and when it comes back only its later changes are written. Unidirectional persists scanResult/revertPlan/revertDonePlan until revertReorders exists.
  • Resume in Account#sync: a continuation that can't be read (corrupt row, IndexedDB gone) or can't be hydrated is treated as none — failing the sync would re-init() the account and wipe cache and mappings. It is only resumed if continuationMatchesStrategy holds: slave resumes unidirectional towards LOCAL, overwrite towards SERVER, a forced sync (strategy === null) anything, other strategies default/merge. If both reorder lists were restored, sync() skips straight to executeReorderingStage(), because re-planning would run a sync of its own whose reorders then lose to the stored ones.
  • A restored continuation gives each member its own copy of a diff that was shared (e.g. planStage3Server.CREATE and serverPlanStage2.CREATE), and only one copy gets drained. So Default applies the failsafes only when the stage-2 plans were built in this run (plannedInThisRun); a re-check would count executed removals against the shrunken tree. Unidirectional re-checks on resume, which is harmless there: it keeps no second copy of revertPlan, whose executors drain it in place, so a resumed check sees only the remaining changes against a tree they have already been applied to — a lower ratio than the one that passed.
  • On failure, keepsContinuation (E026 interrupts, network errors, LocalFolderNotFoundError, the failsafe errors) triggers persistFinalProgress() (non-atomic servers, only once actions were done): the last throttled tick may be up to 10s behind, and re-executing those actions on resume duplicates items. Any other error clears the continuation and re-init()s.

Folder hashes and the tree index

  • folders.hash/folders.hash_settings persist each folder's subtree hash, so a sync only re-hashes the subtrees that changed. IHashSettings are negotiated per sync; a hash stored under different settings is ignored (hashCacheKey in src/lib/Tree.ts). The sync cache carries its hashes too — CacheTree and CachingTreeWrapper clone(true)/copy(true), and the native cache stores them in cache_folders.hash_value.
  • Because hashes survive across syncs now, every mutation has to invalidate them. CachingAdapter#invalidateHashes(folderId) walks Folder#invalidateHashUpwards to the root and calls the onHashesInvalidated hook, which NativeTree and CacheTree override to drop the stored hashes. Any new mutation path in src/lib/adapters/Caching.ts — and any place that rewrites children directly (Scanner, filterOut* in Default.ts, loadServerTree in Merge/Unidirectional) — must do the same, or the scanner concludes that nothing changed.
  • Folder#updateIndex/#removeFromIndex maintain the index incrementally along the path from the root to the item rather than rebuilding it (a rebuild is O(items × depth) on every change); anything they can't make sense of falls back to a full createIndex(), which is always correct. updateIndex always rebuilds the item's own index first, because adapters rewrite ids after inserting an item.
  • Callers that replace a folder's children wholesale must removeFromIndex the old children first — the folders above still list them (see Caching#bulkImportFolder, NextcloudBookmarks#loadFolderChildren). NextcloudBookmarks#bulkImportFolder is the opposite case: the endpoint adds to the folder and answers with only what it imported, so the adapter appends those children to its tree (skipping ones the folder already lists) rather than replacing them -- otherwise a chunked import leaves only the last chunk in the in-memory tree.
  • FLOCCUS_VERIFY_INDEX=true enables Folder#assertIndexConsistent, which cross-checks the incrementally maintained index against a full rebuild after every CachingAdapter mutation. It is a no-op otherwise.
  • Async tree serialization (toJSONAsync) and hashing yield to the event loop by module-global counters (SERIALIZE_ITERATIONS, HASH_ITERATIONS in Tree.ts, every 1000 items) and walk children sequentially; don't reintroduce per-call counters or unbounded Parallel.map.

Build, run, and test workflows

  • Install/build: npm install, npm run build.
  • Dev watch loop: npm run watch (also syncs Capacitor assets; see gulpfile.js).
  • Release artifacts: npm run build-release -> zip/xpi/crx in builds/.
  • Static checks: npm run lint, npm run typecheck.
  • Selenium integration tests: npm test (expects Selenium server + env vars; runner in test/selenium-runner.js). CI (.github/workflows/tests.yml, also android-appium.yml) runs against Nextcloud 35.
  • Node.js test harness: npm run build:test-node bundles src/entries/test-node.js to dist/node-tests/fake-tests.js via webpack.node-tests.js.
  • Node.js test execution: npm run test:node:fake runs the bundled Mocha suite without a browser/WebDriver. Defaults are FLOCCUS_TEST_ACCOUNTS=fake, FLOCCUS_TEST_BROWSER=node, and CI=true; useful knobs include FLOCCUS_TEST (grep), FLOCCUS_TEST_INVERT=true, FLOCCUS_TEST_ACCOUNTS=..., FLOCCUS_TEST_SEED=..., FLOCCUS_VERIFY_INDEX=true, and FLOCCUS_NODE_INCLUDE_BENCHMARK=true (npm run test:node:fake:benchmark).
  • Fake account types available to the harness: fake (atomic, cached), fake-noCache (atomic; the tests call disableCachePersistence(account) from src/test/utils.js, which swaps in a NullCacheStore, and stub setMappings themselves, so every sync starts without persisted state — stubbing setCache alone no longer disables the cache) and fake-nc-bookmarks (FakeNcBookmarksAdapter, isAtomic() === false, ids that embed the parent, bulkImportAppendsChildren with an answer of only the imported items — the stand-in for nextcloud-bookmarks, so it also exercises the chunked bulk import). The nodejs-fake-test CI workflow runs a matrix of fake and fake-nc-bookmarks.
  • Tests supply their own local folder: Account.create({...ACCOUNT_DATA, ...(await createTestLocalRoot())}). Production code never creates one — BrowserAccount#init throws LocalFolderNotFoundError (E038) for a profile without a local folder.
  • expectTreeEqual(tree1, tree2, ignoreEmptyFolders, checkOrder = true, checkTags = false): pass Boolean(account.server.orderFolder) as checkOrder so adapters without ordering (Linkwarden, Karakeep, ...) aren't held to an order. checkOrder === false sorts both trees' children in place.
  • The node harness shims Capacitor plugins via webpack aliases in webpack.node-tests.js; @capacitor-community/sqlite resolves to src/test/node-shims/capacitor-sqlite.js, an in-memory sql.js database that lives for the length of the process. sql.js is kept as a webpack external so it can locate its own wasm file at runtime. IndexedDB comes from fake-indexeddb via src/test/node-shims/indexeddb.js, which the tests import themselves (not an alias).
  • src/test/native_storage.test.js (in src/test/node-suite.js) exercises the native stores directly: NativeTreeStore/NativeTreeQuery/NativeMappingsStore (tree round-trips, folder hash persistence/invalidation, browsing, search ranking/predicates/LIKE escaping/non-ASCII folding/search_text backfill, mapping id types), the preferences entries (setEntry round-trips, replaces rather than merges, agrees with changeEntry), the legacy continuation entry (NativeAccountStorage continuation), NativeAccountStorage incremental continuations (only changed actions are written, in-flight/unacknowledged rows, markChanged, shared diffs, no adoption of a previous run's rows, legacy blob resume) and Native SQLite sync cache (writes only what changed, subtree removal, moves, bulk import, hashes, the preferences migration, a failed save is written again).
  • src/test/browser_continuation_store.test.js covers BrowserContinuationStore: round trip, incremental updates, key-range isolation between diffs and accounts, clearing, reopening after the database was deleted, and a real DefaultSyncProcess driven through toContinuationUpdateAsync → update → markContinuationPersisted.
  • src/test/continuation_resume.test.js runs only with FLOCCUS_TEST_ACCOUNTS=fake-nc-bookmarks (continuations are only persisted for non-atomic servers): no re-execution after a failure, resuming at the reorder stage without re-planning (default/overwrite, plain/forced/explicit strategy), not resuming for the opposite direction, syncing normally over an unreadable continuation, and no failsafe trip over removals already executed.
  • src/test/caching_tree_wrapper.test.js covers CachingTreeWrapper's bulk import (capability only exposed when the wrapped tree has it, cache mirrors the import under the live tree's ids, a subtree well over the nextcloud chunk size still goes in one call, the cache stays out of later changes) and cache dirty tracking (persisting the cache).
  • src/test/diff.test.js covers Diff (retract, peekActions, markChanged, persistence across compaction) and async tree serialization; src/test/progress_interval.test.js covers progressInterval.
  • Appium/native Android harness: npm run test:appium runs test/appium-runner.js, which waits for an Appium server, creates an Android UiAutomator2 session, switches into the app's WEBVIEW, opens the native #/test route, and streams Mocha logs until a FINISHED marker is emitted.
  • Appium prerequisites: the Android app/APK must already be built and installed, and an Appium server with the uiautomator2 driver must be running. Common env vars are APPIUM_SERVER, APPIUM_DEVICE_NAME, either APPIUM_APP or (APPIUM_APP_PACKAGE + APPIUM_APP_ACTIVITY), plus the same test-selection env used by the browser harness (FLOCCUS_TEST, FLOCCUS_TEST_SEED, APP_VERSION, TEST_HOST, adapter-specific credentials/tokens such as Google/Dropbox/Linkwarden/Karakeep).
  • Browser-local test mode is destructive to bookmarks unless using a dedicated profile (see README.md test section).

Project conventions (specific to this repo)

  • Mixed JS/TS/Vue2 codebase (allowJs: true in tsconfig.json); keep edits consistent with surrounding file language.
  • Lint style is strict and legacy-standard-like: single quotes, no semicolons, 2-space indent (.eslintrc.json).
  • Adapters are registered centrally in src/lib/Account.ts via AdapterFactory.register(...) (dynamic imports).
  • Sync reliability relies on continuation persistence and mapping GC; avoid "simplifying" this flow without preserving resume semantics.
  • IS_BROWSER compile-time flag (webpack define) is the platform switch; do not branch on ad-hoc runtime checks when an existing IS_BROWSER path exists.

Integration notes for safe changes

  • Browser manifests differ (manifest.firefox.json is MV2 background page; manifest.json/manifest.chrome.json are MV3 service worker).
  • gulpfile.js contains a guard to prevent browser-api leakage into native chunk (webpackCheck).
  • Nextcloud adapter (src/lib/adapters/NextcloudBookmarks.ts) is the most feature-rich reference for locking, sparse tree loading, ordering, and request handling.
  • If adding/changing adapters, implement interfaces/Resource.ts capabilities (getCapabilities, isAtomic, optional orderFolder/bulkImportFolder/loadFolderChildren) and verify strategy interactions.
  • bulkImportFolder comes in two flavours and the difference decides how Default#executeCreate calls it. NextcloudBookmarks adds the given children to the folder but refuses more than 75 bookmarks per request, so a large subtree is imported in chunks (of 70 bookmarks; subfolders become their own CREATEs); CachingAdapter — and thus NativeTree and the CachingTreeWrapper around it — replaces the folder's children, so chunking it would keep nothing but the last chunk. bulkImportAppendsChildren on BulkImportResource is what says which, and only the appending kind is ever chunked.
  • The sub-scanner after each bulk import creates the mappings, and Scanner#addMapping only maps a LOCAL/SERVER pair — a scan of two items from the same location records nothing, silently. Scan the source-side tree (action.oldItem, or a chunk restamped to its location) against what the import returned.
  • CachingTreeWrapper is what the strategy gets as the local tree (Account#sync), so a capability the wrapper doesn't forward is a capability the sync can't use — it detects bulk import with 'bulkImportFolder' in resource, and the wrapper therefore only defines that method (in its constructor) when the tree it wraps has one. It also forwards isUsingBrowserTabs, which picks the failsafe thresholds.
  • Failsafes: applyDeletionFailsafe/applyAdditionFailsafe are async and must be awaited; getFailsafeThresholds() is laxer for tab sync (#2318).
  • SyncProcess#loadChildren (sparse server trees) must recurse into folders already marked loaded too: the sparse listing marks one layer at a time, so returning early leaves grandchildren unloaded.
  • BrowserTree#updateBookmark/#updateFolder skip browser.bookmarks.move when the parent doesn't change — a move without an index appends to the end of the folder (#2361); positions are the REORDERs' job.
  • DropboxAdapter#request retries 5xx only for callers passing retriable (the API is all POST, so the verb says nothing about whether a repeat is harmless).
  • GitAdapter may only delete its own lightning-fs IndexedDB databases (isDisposableFsDatabase): deleting another one (e.g. floccus_continuations) blocks, a blocked delete stays pending, and from then on indexedDB.databases() never resolves for the origin. BrowserContinuationStore closes its connection on versionchange for the same reason.
  • Profile import/export strips volatile account data (VOLATILE_ACCOUNT_DATA in Account.ts) and no longer disables imported profiles; ImportExport.vue sends the user to the first profile that still needs authorization or a local folder.
  • i18n strings live in _locales/en/messages.json; UI text should use i18n helpers rather than hardcoded strings.

Sync algorithm internals (diff/reconcile)

  • Scanner (src/lib/Scanner.ts) diffs cacheTreeRoot (always local-located) against a live tree using a mergeable predicate chosen per scanner: the main getDiffs scanners accept Mappings.mappable pairs, or canMergeWith (weak: bookmarks by URL, server scanner only; folders by title only when syncing tabs). The sub-scanners (concurrent creations in reconcileDiffs, both bulk-import paths, Merge) accept canMergeWith && !Mappings.wouldEvictUnrelatedMapping(snapshot, a, b).
  • Scanner#addMapping evicts a conflicting entry when it adds one, so a pairing can't strand a mapping. But an item that vanishes without a synced deletion can, and the sub-scanners then refuse to re-bind its counterpart. The pre-scan prunes clean that up, and they only run on a fresh plan, never on a resumed one:
    • Default#dropDeadMappings drops mappings whose local item is in neither the local tree nor the cache. Keep the cache check: an item missing from the tree but still in the cache was deleted locally, and this sync needs the mapping to delete it on the server.
    • Default#repairServerMappings re-points a mapping whose server item is gone to a unique canMergeWith item in the mapped parent, or drops it. It exists for nc-style ids (<id>;<folderId>), which change when a folder is re-created without any hash noticing. It walks folders before bookmarks and parents before children (SyncProcess.indexTree/mappedIdsInTreeOrder), and is skipped for sparse servers ('loadFolderChildren' in this.server).
    • Unidirectional#dropMappingsOfVanishedSlaveItems drops mappings whose slave-side item is gone.
  • After a successful sync, Mappings#gc (Account.ts) drops mappings of local items missing from the post-sync cache, and — only for atomic servers — of server items missing from the server tree; a sparse tree would drop unloaded items.
  • Mappings#getSnapshot() returns a cached, shared snapshot that mutators invalidate (invalidateSnapshot) rather than modify, so a caller keeps seeing the state it asked for. Treat it as read-only (use getMutableSnapshot() to write), call getSnapshot() again to see later changes, and make any new mutator invalidate.
  • reconcileDiffs in Default.ts builds the per-target plan; it must never plan an UPDATE/MOVE against an item that's absent from the freshly-fetched target tree (executes as E002 UnknownBookmarkUpdateError / E004 UnknownMoveTargetError). Concurrent-removal detection via REMOVE actions + Diff.findChain is best-effort; a target-tree existence check (targetTree.findItem(type, mapId(...))) is the robust guard.
  • Stage-3 MOVEs and the final REORDERs are mapped with Diff#map(..., skipErroneousActions = true): an action whose id or parent can't be mapped is dropped (Failed to map id: ... dropping action from plan) instead of throwing E048 MappingFailureError, whose reset-and-resync recovery duplicates items on non-atomic servers. The drop is silent to the user, so a missing mapping now shows up as a change that never reaches the other side — look for that log line. Stage-2 mapping still throws.
  • Scanner#createsFolderLoop lets a folder move into a folder that used to be its descendant if that folder has moved out in the new tree (nestingSurvives). Refusing the pairing would turn the move into REMOVE + CREATE and delete everything moved into it.
  • Diff#getActions() returns a copy the caller owns (use it to mutate the list or to retract while iterating); peekActions() returns the diff's own array, which is valid only for reading and only until the next commit/retract — keep actions from it, never the array.

Debugging the node benchmark suite

  • The fake-nc-bookmarks benchmark interrupt test simulates nextcloud-bookmarks: both accounts are wired to one shared server bookmarksCache and the adapter reports isAtomic() === false; setInterrupt() aborts syncs mid-flight (recoverable errors are E026/E027 only — see syncAccountWithInterrupts in src/test/utils.js). The fake/fake-noCache accounts copy the server db at sync boundaries and are atomic.
  • Logs are noisy and misleading: the fuzzers (randomTreeManipulationWithDeletion) wrap their own NativeTree mutations in try/catch and console.log the errors, so most E001/E002/E004 lines (stack via NativeTree.updateBookmark) are expected noise. The real failure is the line Syncing failed with ... (stack through FakeAdapter + SyncProcess).
  • CI job logs interleave real-time stdout with a buffered Logger dump at the end, and util.inspect truncates trees/actions ([Bookmark], [Array]) — scan-result/plan contents are not fully recoverable from logs; trace by item id and the Mapping <server|local> plan markers instead.
  • A benchmark that ends with a tree mismatch but no sync error is often a silently dropped action: grep the sync's log for Failed to map id, then trace that id back through the sync that should have created its mapping.