Mobile, Desktop, Cli: Resolves #16020: Automatically merge non-overlapping note changes on sync conflicts

[varunkumar-22]

Fixes #16020

PR Summary:

As described in issue #16020 :
When the sync loop detects a note conflict, this new code runs inside handleConflictAction, just after the existing mustHandleConflict check and before the conflict note is created. So, the automatic conflict resolution is performed after the conflict detection and just before the note creation. Read-only notes, encrypted notes , and locked notes fall back to the existing conflict note behavior.

The three versions

A three-way merge needs base, local, and remote, and all three already exist:

• base :- the last version this device [ in which conflict is detected] uploaded, kept in sync_items.base_body/base_title. Saved only on upload, never on download, so it's always something this device really agreed with the server. [ PR: Chore: Desktop, Mobile: Resolves #15570: Populate base and conflict note state during sync #15669 ]
• local :- the current note.
• remote :- the incoming version (remoteContent).
  If there's no base (a note only ever downloaded ), it falls back to the plain conflict note

How the merge works (diffNotes.ts):

autoMerge(base, local, remote):

• Fixed a bug where a single trailing space could make diff3 detect a false conflict. Diffs now run on a normalised copy of the text, while unchanged parts of the merged output keep their original formatting; conflicting sections still use that normalised text, and the Markdown hard line breaks (two trailing spaces) are preserved
• Uses node-diff3 library to split the three versions into parts: parts in which only one side changed, and parts both sides changed
• Each part becomes a section marked as unchanged, auto-merged, or conflict. Merging is line based, so if both sides changed the same line it stays a conflict

Turning that into two notes (autoMergeNote.ts)

From those sections it builds two versions that are same everywhere except where a real conflict is happened:

• resolvedLocal : keeps the user's side for conflicts
• resolvedCurrent : keeps the incoming [remote] side for conflicts
• fullyMerged is true only when nothing is left as a conflict
• titles work the same way: merged if only one side changed it, conflict if both did changed

What sync does with this:

• Fully merged : no conflict note will be created. The merged note is saved with anew updated_time, so it is uploaded during the next sync and reaches the other device
• Partially merged : a conflict note is still created, but the changes that can be merged are added to both the conflict note (local version) and the updated original note (remote version); this means they only differ where there is a real genuine conflict. The base version is saved in conflict_note_states, along with the remote note's updated_time so the resolution UI can tell whether the original note has changed since the conflict was created. The remote_title and remote_body are kept for future partial resolution purpose.

E2EE support

Initially, automerge not used to run when E2EE is enabled as we can read encrypted remote note content until it is decrypted and normal conflict behaviour is continued. So, now this case is handled by decrypting the notes which are found to be conflicts, E2E encrypted and local is not encrypted. the encrypted remote note is decrypted in memory and it is used for running automerge. If everything is merged cleanly, then the merged note is saved directly over the local note and If incase a real conflict exists, decrypted remote (along with some auto-merged changes) is used to update the original note , which helps upcoming conflict resolution UI to read the actual remote content from it instead of encrypted or any empty data
The local note is not decrypted; it is expected to already be decrypted during a normal sync, and if incase it is still encrypted, or if the remote note cannot be decrypted (for ex, the master key has not been unlocked), then the behaviour safely falls back to the existing conflict note behavior

• Testing for E2EE handling:
  added additional tests in Synchronizer.autoMerge.test.ts for cases : merging an encrypted remote, keeping the decrypted remote on the original note when partially merged, and the fallback when decryption is not available

Settings

A new setting controls this:

sync.autoMergeConflicts - a normal public setting under the Synchronisation, named as "Automatically merge non-conflicting note changes". It is enabled by default, so automatic merging is available immediately, while users can still disable it at any time.
The setting is read through a static helper (isAutoMergeEnabled()) instead of checking the setting name in different places

Testing:

Tests are added:

• diffNotes.test.ts - the merge engine: clean merges, real conflicts, line level merges, trailing whitespace, no-base fallback
• autoMergeNote.test.ts - title rules and the two resolved versions
• Synchronizer.autoMerge.test.ts - two real synced devices: clean merge makes no conflict note and reaches the other device, partial merge keeps shared changes into both, locked notes skipped

Tested the automatic merge implementation across various Markdown edge cases, including images, links, tables, code blocks, lists, headings, blockquotes, checklists, nested formatting, emojis, whitespace-only changes, insertions, deletions, and mixed content to verify that merges behave correctly without breaking the document structure and all are working correctly.

limitation:

• Auto-merging is line based only, so any two edits to the same line are always treated as a conflict even when they touch different words. Edits on adjacent lines are also treated as one region by the diff3 library, so they also conflict together.
• Word level merging was implemented initially but dropped after review [ commet ] and A short write up for word-level diff will be written in automerge documentation, so it will be useful if word-level automerge is considered in future. Word level highlighting inside the conflict resolution UI is unaffected.
• The remote_body and remote_title columns in conflict_note_states are left in the schema but no longer written at sync time [ comment ]. As at that point they only duplicate what original note alreadyhas, and so they will be invalid as soon as any changes are made to note. They are kept for storing a partial resolution later.

AI Assistance Disclosure:

AI was used in researching & learning about the library implementation, in writing some tests and also for data while testing various edgecases

[coderabbitai]

.

[mrjo118]

There's a test failure:

[@joplin/lib]: Summary of all failing tests
[@joplin/lib]: FAIL services/synchronizer/Synchronizer.basics.test.js (41.127 s)
[@joplin/lib]: ● Synchronizer.basics › should handle items that are read-only on the sync target
[@joplin/lib]:
[@joplin/lib]: expect(received).toBe(expected) // Object.is equality
[@joplin/lib]:
[@joplin/lib]: Expected: "un"
[@joplin/lib]: Received: "un mod"
[@joplin/lib]:
[@joplin/lib]: 332 |
[@joplin/lib]: 333 | const noteReload = await Note.load(note.id);
[@joplin/lib]: > 334 | expect(noteReload.title).toBe(note.title);
[@joplin/lib]: | ^
[@joplin/lib]: 335 |
[@joplin/lib]: 336 | const conflictNote: NoteEntity = (await Note.all()).find((n: NoteEntity) => !!n.is_conflict);
[@joplin/lib]: 337 | expect(conflictNote).toBeTruthy();
[@joplin/lib]:
[@joplin/lib]: at Object.toBe (services/synchronizer/Synchronizer.basics.test.ts:334:28)
[@joplin/lib]:
[@joplin/lib]:
[@joplin/lib]: Test Suites: 1 failed, 170 passed, 171 total
[@joplin/lib]: Tests: 1 failed, 1 skipped, 1674 passed, 1676 total
[@joplin/lib]: Snapshots: 34 passed, 34 total
[@joplin/lib]: Time: 452.207 s
[@joplin/lib]: Ran all test suites.
The command failed in workspace @joplin/lib@workspace:packages/lib with exit code 1
Failed with errors in 9m 11s

[coderabbitai]

.

[laurent22]

Please review and trim down the LLM generated comments

[varunkumar-22]

Thanks for the review, I have removed unwanted comments and shortened the necessary ones

[varunkumar-22]

think I know what the problem is now. File system sync (but not most other targets) will always fetch an item after uploading it (in the delta step), on the same run of the sync. The delta step invokes BaseItem.updateSyncTimeQueries which will delete the sync_item row and re-create it, which will re-create it with the base version fields cleared out.

Thanks for finding it, that was the problem with Base cleared on every sync run with file system as sync target, which is why automerge was not performed.

Add some mandatory new parameters base_body and base_title to the updateSyncTimeQueries function. Amend usages of the function to select the existing base_body and base_title prior to calling the function, to pass the values in. The saveSyncTime and saveSyncDisabled functions are async, so they are able to await fetching the values within the function, to pass down to updateSyncTimeQueries without requiring additional parameters adding to themselves. These changes ensure the base version is never lost from the sync_item record

I fixed it slight differently. Instead of adding parameters and updating every caller as you suggested , I made the insert to carry the existing base values over from row it is replacing, so nothing lost here whichever sync step call it. I also carried base_conflict_note_id, which was losing the same way

On 2nd one, I think it's not required because base is recorded only on upload step from the local note and not on download, so it's always plaintext and it never needs decrypting [you can see in below video about E2EE on a file system target]

Screen.Recording.2026-08-12.at.6.mp4

Testing info: Base was recorded in left profile before recording the video

[mrjo118]

On 2nd one, I think it's not required because base is recorded only on upload step from the local note and not on download

It's not technically required to record the base on download (and was not in the original proposal), but I realise now that it is advisable to record it on download as well, to get the least amount of conflicts. Take this example:

1. Device A and B has note with contents 'a\nb\nc'. The base version is the same value in the db on both clients
2. Device A changes the note contents to be 'a1\nb\nc'. It syncs and the base version is 'a1\nb\nc'
3. Device B syncs. The local contents are now 'a1\nb\nc', but the base version is still 'a\nb\nc'
4. Device A changes the note contents to be 'a1\nb\nc2'. It syncs and the base version is 'a1\nb\nc2'
5. Device B changes the note contents to 'a12\nb\nc'
6. Device B syncs, but hits the conflict logic before it can upload the change. This evaluates a conflict using the following:
   Base version: 'a\nb\nc'
   Local version: 'a12\nb\nc'
   Remote version: 'a1\nb\nc2'

A conflict must be created because it could not be fully resolved automatically. If the base version was stored on download as well, then the conflict would have been able to be automatically resolved completely.

It's worth noting as well, if E2EE is not enabled it should also update the base version on download as well: If encryption is not applied update it via the Synchronizer, if encryption is applied, that would be skipped, and it will update it via the BaseItem.decrypt flow instead

[mrjo118]

You can't do this. Ids in Joplin are UUIDs, so using MAX will not work. Also in the insert, you're doing many sub queries vs 1 additional select which I proposed. From a performance perspective it's probably better to follow my suggested approach instead (plus in include the base_conflict_note_id column as well which I forgot about)

[varunkumar-22]

sync_items.id is an INTEGER PRIMARY KEY so max do work but I think it's fair considering performance and to avoid subqueries, so I followed your approach. I added base_conflict_note_id as well. Verified it manually.

Screen.Recording.2026-08-12.at.9.mp4

[varunkumar-22]

Fixed a bug in commit:
when E2EE is enabled, on a partial merge, the note was saved with merged content but encryption_applied was still with old cipher text, so the decryption worker later decrypt that and overwrites the merge. The merged result appears in conflict note but was lost from the original. I fixed it by using the whole decrypted copy rather than just its title and body, since that copy already has encryption fields cleared

Issue:

Screen.Recording.2026-08-12.at.7.mp4

Fix

Screen.Recording.2026-08-12.at.7.1.mp4

[mrjo118]

On 2nd one, I think it's not required because base is recorded only on upload step from the local note and not on download

It's not technically required to record the base on download (and was not in the original proposal), but I realise now that it is advisable to record it on download as well, to get the least amount of conflicts. Take this example:

1. Device A and B has note with contents 'a\nb\nc'. The base version is the same value in the db on both clients
2. Device A changes the note contents to be 'a1\nb\nc'. It syncs and the base version is 'a1\nb\nc'
3. Device B syncs. The local contents are now 'a1\nb\nc', but the base version is still 'a\nb\nc'
4. Device A changes the note contents to be 'a1\nb\nc2'. It syncs and the base version is 'a1\nb\nc2'
5. Device B changes the note contents to 'a12\nb\nc'
6. Device B syncs, but hits the conflict logic before it can upload the change. This evaluates a conflict using the following:
   Base version: 'a\nb\nc'
   Local version: 'a12\nb\nc'
   Remote version: 'a1\nb\nc2'

A conflict must be created because it could not be fully resolved automatically. If the base version was stored on download as well, then the conflict would have been able to be automatically resolved completely.

It's worth noting as well, if E2EE is not enabled it should also update the base version on download as well: If encryption is not applied update it via the Synchronizer, if encryption is applied, that would be skipped, and it will update it via the BaseItem.decrypt flow instead

@varunkumar-22 What are your thoughts about this message? Additionally, without saving base version on download, when a user does the full initial sync for a new device, the base version won't be populated for any note initially.

Also, what do you think about altering the description of the setting, to highlight the possibility of duplication and changes to formatting when enabled?

[varunkumar-22]

What are your thoughts about this message? Additionally, without saving base version on download, when a user does the full initial sync for a new device, the base version won't be populated for any note initially.

Agreed, this makes sense and I initially discussed to record base for downloads as well, but it was ruled out considering that it's not safe to do it. It will avoid unnecessary conflicts when automerge is enabled. Yet, I feel it's safe to write base for downloads as

update_remote runs before delta step, and itemsThatNeedSync takes all the things where sync_time < updated_time, so when a note is modified locally it is always handled as a conflict first and never reaches UpdateLocal. This means what is getting written on download really is the version both sides agreed on, that is what the base needs to be.

But I still would like to confirm it whether it is safe to do or not.

I think in any case, the duplicate line ambiguity issue is likely to be a lot rarer than repeated text / intent on the same line. So maybe we should follow the approach of Obsidian and just include a disclaimer on the setting?

I also think the whitespace limitation would be acceptable. A code comment would suffice, and a general disclaimer similar to the Obsidian one on the setting also implies this limitation for the user facing text.

I discussed about this with Caleb, and we agree to go with a disclaimer as it can take a lot of time working on this issue with no guarantee of finding solution. I will update the setting in upcoming commit and will give an update after it is done.

[mrjo118]

update_remote runs before delta step, and itemsThatNeedSync takes all the things where sync_time < updated_time, so when a note is modified locally it is always handled as a conflict first and never reaches UpdateLocal. This means what is getting written on download really is the version both sides agreed on, that is what the base needs to be.

But I still would like to confirm it whether it is safe to do or not.

This code comment is what indicates you can never have a conflict in the delta step (implicitly stated that it is due to the upload step always running before delta), and therefore you can rely on any version saved locally in the delta step as being a common base. If the delta step would ever be moved before the upload, then the sync algorithm would become fundementally broken.

joplin/packages/lib/Synchronizer.ts

Lines 894 to 895 in 2654b33

	// At this point all the local items that have changed have been pushed to remote
	// or handled as conflicts, so no conflict is possible after this.

My initial pushback was because I thought it was best to change the Synchronizer code as little as possible, but I see now that there is a safe way to do this, even with E2EE, and it does provide a notable benefit.

[coderabbitai]

.

[varunkumar-22]

With commit, now the base is recorded on downloads as well. When E2EE is enabled, the base is skipped to avoid recording encrypted content and it is recorded once the decryption worker ran and the content ends up as plain text

So I'd propose to modify BaseItem.decrypt so that if the item is a note, after decrypting the item, it should call a new BaseItem function which will perform an update for just the base_body and base_title columns on the matching sync_item record, without updating any of the timestamps. This change does not need to be feature flagged, as the expensive part (the decryption) is part of the existing code already

Implemented as you described, but with just one difference. Instead of adding a new function I reused existing saveSyncBaseContent, which will also clears base_conflict_note_id. That's like a difference in code but since decrypt only runs on notes that are still cipher text and a note with the active conflict link is already stored decrypted, so the decryption worker never run for it.

Without E2EE:

Screen.Recording.2026-08-14.at.3.mp4

With E2EE:

Screen.Recording.2026-08-14.at.2.mp4

Testing info: file system is used as sync target

[varunkumar-22]

Low severity (whitespace-only, and the merged note is what uploads so the loss propagates rather than being flagged), but it's a real silent-content-alteration path in the fully-merged branch. Worth a comment acknowledging it as a known limitation if not fixed.

I think in any case, the duplicate line ambiguity issue is likely to be a lot rarer than repeated text / intent on the same line. So maybe we should follow the approach of Obsidian and just include a disclaimer on the setting?

In commit I added a code comment for white-space only edit loss and also duplication when adjacent identical lines are edited. This also covers disclaimer about this limitations.

Desktop:

Mobile:

[mrjo118]

In commit I added a code comment for white-space only edit loss and also duplication when adjacent identical lines are edited. This also covers disclaimer about this limitations.

I think the user facing note below the setting is a bit too technical and long to be honest. How about this?:
"When the same note is edited on two devices, changes made to different lines are usually merged automatically. In rare cases, automatic merging may result in duplicated content or formatting changes"

[mrjo118]

My apologies, I think actually making an extra select before calling this query is less efficient than your previous solution. I didn't realise on a sqlite db that making extra round trips to the db is still much more expensive than running multiple queries in a single transaction. While a single select in it's own transaction is very cheap (maybe 1-10 ms depending on the device), considering that the delta step may iterate over 100s of thousands of items, the overall impact of adding this additional select may not be insignificant.

You were correct that the id of sync_items is indeed an integer, and it would work to match the latest row using MAX. So I think it would be better to restore your previous change in commit 359654b, but AI suggested a more optimised query for the INSERT query:

INSERT INTO sync_items (
    sync_target,
    item_type,
    item_id,
    item_location,
    sync_time,
    remote_item_updated_time,
    sync_disabled,
    sync_disabled_reason,
    base_body,
    base_title,
    base_conflict_note_id
)
SELECT
    ?,
    ?,
    ?,
    ?,
    ?,
    ?,
    ?,
    ?,
    COALESCE(existing.base_body, ''),
    COALESCE(existing.base_title, ''),
    COALESCE(existing.base_conflict_note_id, '')
FROM (SELECT 1) AS seed
LEFT JOIN sync_items AS existing
    ON existing.id = (
        SELECT MAX(candidate.id)
        FROM sync_items AS candidate
        WHERE candidate.sync_target = ?
          AND candidate.item_type = ?
          AND candidate.item_id = ?
    );