Skip to content

Fix duplicate patient creation on MPI import - #60

Merged
ibacher merged 11 commits into
IsantePlus:mainfrom
mherman22:fix/duplicate-patient-on-mpi-import
Apr 14, 2026
Merged

Fix duplicate patient creation on MPI import#60
ibacher merged 11 commits into
IsantePlus:mainfrom
mherman22:fix/duplicate-patient-on-mpi-import

Conversation

@mherman22

@mherman22 mherman22 commented Apr 9, 2026

Copy link
Copy Markdown

Summary

When importing a patient from the MPI (OpenCR) into iSantePlus, two local patient records are created instead of one. This happens because:

  1. importMpiPatient()savePatient() triggers PatientSynchronizationAdvice AOP
  2. The AOP spawns PatientUpdateWorker which re-queries the MPI
  3. The worker discovers golden record seealso links and triggers a second importMpiPatient() call
  4. The second import creates a duplicate patient with a new iSantePlus ID

Changes

  • PatientSynchronizationAdvice: Add ThreadLocal<Boolean> SUPPRESS flag — when set, the AOP advice skips spawning PatientUpdateWorker, breaking the re-entrant import loop
  • FhirMpiClientServiceImpl.importPatient(): Implement the method (was returning null), with duplicate detection via matchWithExistingPatient() and AOP suppression during savePatient()
  • HL7MpiClientServiceImpl.importPatient(): Add the same AOP suppression to prevent duplicates on the HL7 code path
  • PatientUpdateWorker: Add dedup lock (HashSet) to prevent concurrent exports of the same patient (matching existing PatientSyncWorker pattern)
  • PatientUpdateWorker + PatientSyncWorker: Add proxy privileges to prevent APIAuthenticationException in background threads
  • MpiClientService / MpiClientWorker / MpiClientServiceImpl: Remove duplicate getPatientList() method definitions and fix parseFhirPatient() call signature

Test plan

  • Import a patient from the MPI into one iSantePlus instance — verify only ONE local record is created
  • Verify the imported patient is correctly exported back to OpenCR
  • Import the same patient again — verify it updates the existing record instead of creating a new one
  • Verify PatientUpdateWorker still runs normally for non-import savePatient() calls (e.g. editing a patient in the UI)
  • Verify cross-reference identifiers are still resolved after patient creation
  • Check OpenHIM transactions — no duplicate POST to /CR/fhir/Patient during import

@mherman22
mherman22 force-pushed the fix/duplicate-patient-on-mpi-import branch 5 times, most recently from 9aa07ca to 025993f Compare April 9, 2026 19:55
Comment thread .github/workflows/ci.yml
boolean hasId = false;
if (id.getIdentifierType() == null) continue;
for (PatientIdentifier eid : patientRecord.getIdentifiers())
hasId |= eid.getIdentifier().equals(id.getIdentifier())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly too clever. You actually want:

Suggested change
hasId |= eid.getIdentifier().equals(id.getIdentifier())
hasId = hasId || eid.getIdentifier().equals(id.getIdentifier())

The |= is a bitwise or and assignment operator similar to +=. The thing is that while the bytecode for hasId |= ... is somewhat more efficient byte-code wise, it will always evaluate both operands whereas hasId = hasId || ... will short-circuit, i.e., once hasId has been set to true it will no longer invoke the eid.getIdentifier().equals(id.getIdentifier()) part.

To demonstrate, a simplified version based on:

boolean first = true;
boolean second = true;

first |= second;

Results in something like this:

1 iload_1  // load the first "true"
2 iload_2 // load the second "true"
3 ior         // bitwise "or" of 1 and 2

Whereas:

boolean first = true;
boolean second = true;

first = first || second;

Is much more verbose, but in a key way:

1 iload_1   // load the first "true"
2 ifne 5    // if the first value != 0 jump to line 5
3 iload_2 // load the second "true"
4 ifeq 7    // if the second value = 0 jump to line 7
5 iconst_1 // the value 1, i.e., false
6 return
7 iconst_0 // the value 0, i.e., true
8 return

Which is more bytecodes but in the real example, every where we have iload_2 in the dummy code, we'd actually have something like:

1 aload_0         // load eid
2 invokevirtual // run getIdentifier()
3 astore_2       // save the result to register 2
4 aload_1         // load id
5 invokevirtual // run getIdentifier(), result on stack
6 aload_2        // load result of first identifier call
7 invokevirtual // call equals()

In the second case we can skip those steps. And if none of that makes any sense, the short version is || short-circuits, i.e., only evaluates the left-side if the left-side is true, but | does not, i.e., it always evaluates both sides.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! this was just copied from

public Patient importPatient(MpiPatient patient) throws MpiClientException
so perhaps a rewrite of that is required. Bringing this method here is somewhat not the fix. I noticed registration-core module has https://github.com/IsantePlus/openmrs-module-registrationcore/blob/5af0ddfc383379b0d42de352b9643e0e6aab6a0c/api/src/main/java/org/openmrs/module/registrationcore/api/impl/RegistrationCoreServiceImpl.java#L417-L487 and i am still trying to figure out these two calls and how they complement each other.

For now i will revert this!

@mherman22
mherman22 force-pushed the fix/duplicate-patient-on-mpi-import branch from 025993f to 43a2f66 Compare April 9, 2026 20:13
Add a ThreadLocal SUPPRESS flag to PatientSynchronizationAdvice that
callers can set to skip the AOP advice during patient import, preventing
the re-entrant loop where PatientUpdateWorker triggers a second import.

Add a deduplication lock (HashSet) to PatientUpdateWorker to prevent
concurrent exports of the same patient, matching the existing pattern
in PatientSyncWorker.

Add proxy privileges to PatientUpdateWorker and PatientSyncWorker to
prevent APIAuthenticationException in background threads.

Fix pre-existing compile errors: remove duplicate getPatientList()
definitions, fix parseFhirPatient() call signature.

Update CI: replace deprecated GitHub Actions (checkout v4, setup-java
v4 with cache: maven, cache v4, setup-maven v5), drop
s4u/maven-settings-action in favor of setup-java built-in server config.
@mherman22
mherman22 force-pushed the fix/duplicate-patient-on-mpi-import branch from 43a2f66 to 217902b Compare April 9, 2026 20:59
mherman22 added a commit to mherman22/openmrs-module-registrationcore that referenced this pull request Apr 9, 2026
When importMpiPatient() saves a patient via createImportedMpiPatient(),
the mpi-client PatientSynchronizationAdvice AOP fires and spawns a
PatientUpdateWorker. This worker re-queries the MPI, discovers golden
record seealso links, and triggers a second import — creating a
duplicate local patient.

Wrap the savePatient() call with PatientSynchronizationAdvice.SUPPRESS
to skip the AOP during import. The export back to MPI is already
handled explicitly by exportPatient() at the end of importMpiPatient(),
so the AOP-triggered export is redundant anyway.

Depends on IsantePlus/openmrs-module-mpi-client#60 which adds the
SUPPRESS ThreadLocal flag.
mherman22 added a commit to mherman22/openmrs-module-registrationcore that referenced this pull request Apr 9, 2026
When importMpiPatient() saves a patient via createImportedMpiPatient(),
the mpi-client PatientSynchronizationAdvice AOP fires and spawns a
PatientUpdateWorker. This worker re-queries the MPI, discovers golden
record seealso links, and triggers a second import — creating a
duplicate local patient.

Wrap the savePatient() call with PatientSynchronizationAdvice.SUPPRESS
to skip the AOP during import. The export back to MPI is already
handled explicitly by exportPatient() at the end of importMpiPatient(),
so the AOP-triggered export is redundant anyway.

Depends on IsantePlus/openmrs-module-mpi-client#60 which adds the
SUPPRESS ThreadLocal flag.
mherman22 added a commit to mherman22/sedish that referenced this pull request Apr 10, 2026
…e patient import

- santedb-mpiclient-1.1.5-SNAPSHOT.omod: adds SUPPRESS ThreadLocal flag
  to PatientSynchronizationAdvice, dedup lock on PatientUpdateWorker,
  and proxy privileges for background threads
  (IsantePlus/openmrs-module-mpi-client#60)

- registrationcore-2.2.0.omod: sets SUPPRESS=true around savePatient()
  in createImportedMpiPatient() to prevent AOP from spawning
  PatientUpdateWorker during MPI import
  (IsantePlus/openmrs-module-registrationcore#49)

- Dockerfile: copies patched registrationcore to distribution path
  to override the base image's unpatched version
mherman22 added a commit to mherman22/sedish that referenced this pull request Apr 10, 2026
…e patient import

- santedb-mpiclient-1.1.5-SNAPSHOT.omod: adds SUPPRESS ThreadLocal flag
  to PatientSynchronizationAdvice, dedup lock on PatientUpdateWorker,
  and proxy privileges for background threads
  (IsantePlus/openmrs-module-mpi-client#60)

- registrationcore-2.2.0.omod: sets SUPPRESS=true around savePatient()
  in createImportedMpiPatient() to prevent AOP from spawning
  PatientUpdateWorker during MPI import
  (IsantePlus/openmrs-module-registrationcore#49)

- Dockerfile: copies patched registrationcore to distribution path
  to override the base image's unpatched version
mherman22 added a commit to mherman22/iSantePlus that referenced this pull request Apr 10, 2026
- mpi-client: updated to 1.1.5-SNAPSHOT from fix/duplicate-patient-on-mpi-import
  branch (IsantePlus/openmrs-module-mpi-client#60) — adds SUPPRESS
  ThreadLocal, dedup lock, proxy privileges, CI fixes

- registrationcore: updated from fix/suppress-mpi-sync-on-import branch
  (IsantePlus/openmrs-module-registrationcore#49) — sets SUPPRESS=true
  around savePatient() during MPI import, bumps mpiClientVersion to
  1.1.5-SNAPSHOT

- Fixed dead repo references and github-packages profile in
  registrationcore POM
- Aligned xdsSenderVersion to 2.5.9 in registrationcore
mherman22 added a commit to mherman22/iSantePlus that referenced this pull request Apr 10, 2026
- Add 'Adding a New Module' guide: importing from external repo,
  creating from scratch, vendoring dependencies
- Add 'Contributing' section with workflow
- Add 'Downloading OMODs from CI' section
- Expand build instructions with install command
- Update mpi-client to 1.1.5-SNAPSHOT with SUPPRESS flag, dedup lock,
  and proxy privileges (IsantePlus/openmrs-module-mpi-client#60)
- Update registrationcore with SUPPRESS during MPI import
  (IsantePlus/openmrs-module-registrationcore#49)
- Fix dead repo references in registrationcore POM
- Align registrationcore xdsSenderVersion to 2.5.9
- Vendor everest-core 1.1.0 JAR and POM into lib/maven-repo/
- Add .mvn/settings.xml to mirror dead repos (te.marc-hi.ca,
  santesuite.org) to Maven Central
- Copy vendored JARs to ~/.m2 and clear cached failures before build
- Remove dependency on github-packages profile (no auth needed)
- Upload OMOD as build artifact
The everest-core POM declares te.marc-hi.ca and santesuite.org as
repositories. Maven reads it and tries to resolve transitive deps
from those dead repos. Shipping only the JAR avoids this — Maven
treats it as a local artifact without transitive resolution.
Added !lib/maven-repo/**/*.jar exception to .gitignore so the
vendored JAR is tracked by git and available in CI.
@mherman22
mherman22 force-pushed the fix/duplicate-patient-on-mpi-import branch from 4ca6319 to 59f14d1 Compare April 10, 2026 21:53
Comment thread .github/workflows/ci.yml Outdated
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- uses: actions/checkout@v4

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, checkout is currently on v6 or something like that...

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +28 to +34
mkdir -p ~/.m2/repository
# Clear any cached everest resolution state
rm -rf ~/.m2/repository/org/marc/everest
cp -r lib/maven-repo/* ~/.m2/repository/
find ~/.m2/repository -name "*.lastUpdated" -delete 2>/dev/null || true
echo "Installed vendored JARs:"
find ~/.m2/repository/org/marc/everest -name "*.jar"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use mvn install:install-file here to install everest-core? This will ensure that the Maven metadata is also updated.

* When set to true, the advice will skip spawning PatientUpdateWorker,
* preventing the re-entrant import loop that creates duplicate patients.
*/
public static final ThreadLocal<Boolean> SUPPRESS = ThreadLocal.withInitial(() -> false);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this value set anywhere?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see... Ok... Weird, but ok

Comment on lines +138 to +142
Context.removeProxyPrivilege("Get Identifier Types");
Context.removeProxyPrivilege("Get Patients");
Context.removeProxyPrivilege("Get Patient Identifiers");
Context.removeProxyPrivilege("Edit Patient Identifiers");
Context.removeProxyPrivilege("Add Patient Identifiers");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd wrap these in try...catch blocks to ignore any exceptions...

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably same above.

Comment on lines +143 to +145
synchronized (s_lock) {
s_lock.remove(patientUuid);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might make this lock removal the first thing we do in the finally block since it's the least likely to fail.

Comment thread .mvn/settings.xml Outdated
<settings>
<mirrors>
<mirror>
<id>marc-te-mirror</id>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, we can use the Maven repo https://oscarmcmaster.sourceforge.net/m2/ to find Everest-Core. However, it's weird to do this in settings instead of just... adding a repo in the POM

Everest-core was vendored as a JAR because its original Maven repos
(marc-te, santesuite) are dead. Ian pointed out it is available at
https://oscarmcmaster.sourceforge.net/m2/ so we add that repo directly
in the POM, remove the vendored JAR, and drop .mvn/settings.xml which
was only there to mirror the dead repos.

Also wraps proxy privilege setup and teardown in try-catch blocks so a
failure in one does not prevent the others from running, and moves lock
removal to the top of the finally block since it is the least likely to
fail.
@mherman22
mherman22 requested a review from ibacher April 14, 2026 15:18
@ibacher
ibacher merged commit 25c428b into IsantePlus:main Apr 14, 2026
1 check passed
@mherman22
mherman22 deleted the fix/duplicate-patient-on-mpi-import branch April 15, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants