test: guard per-simulation topic isolation with two concurrent simulations - #1901
Open
craigpnnl wants to merge 3 commits into
Open
test: guard per-simulation topic isolation with two concurrent simulations#1901craigpnnl wants to merge 3 commits into
craigpnnl wants to merge 3 commits into
Conversation
Adds testRunTwoConcurrentSimulationsWithTopicIsolation, which issues two simulation requests concurrently on independent client connections and proves per-simulation topic isolation: every message received on simulation A's own output/log topic subscription carries A's own simulation_id/processId, never B's, and vice versa. This guards the regression fixed in faa2d4e (PR 1758): SimulationManager and helics_goss_bridge sharing one topic across simulations so concurrent instances corrupted each other's status. The existing single-simulation test (testRunSimulationWithContainers) is untouched. Its commented-out completion assertions are left as-is: a baseline run against this environment (gridappsd/gridappsd:local, built from this branch) showed GridLAB-D/HELICS never starts here because Proven has no seeded load-schedule timeseries data (GLDZiploadScheduleConfigurationHandler throws before SimulationManagerImpl.startSimulation() is reached), so completion is genuinely unreachable in this environment, not flaky. The new test's load-bearing assertion instead targets the log topic, which fires reliably from the earliest, always-executed phase of simulation request processing regardless of that gap, and best-effort checks the output topic in case that infra gap is ever closed.
|
Code formatting check failed! Please run |
The Blazegraph & Simulation Tests job started a full stack but never invoked any JUnit container test, so SimulationContainerTest (@tag("container", "simulation")) never ran on CI. A PR adding a container test would go green without that test ever executing. Adds a `Run container integration tests` step invoking a new Makefile target `test-simulation-container`, with no continue-on-error so a failure gates the job. No new test tag was needed. build.gradle already defines simulationTest (includeTags 'simulation') and messagingTest (includeTags 'messaging'), which correctly separate the two incompatible container strategies: SimulationContainerTest expects an externally started stack via GridAppsDTestEnvironment, while GridAppsDContainerTest manages its own Testcontainers lifecycle on the same fixed ports. The existing containerTest task selects both and would collide. The new Makefile target selects only the 'simulation' tag and guards on a running gridappsd container, matching the test-blazegraph and test-stomp-topics pattern. Verified locally against a stack started the same way CI starts it: 6 tests run, 0 failures, including testRunTwoConcurrentSimulationsWithTopicIsolation, and no GridAppsDContainerTest report is produced, confirming it is not selected. Also converts pre-existing non-ASCII characters in Makefile section headers and the test-check target to ASCII. No logic change; required for the edit to land.
There was a problem hiding this comment.
Pull request overview
Adds a regression guard to ensure per-simulation topic suffixing remains intact when two simulations are started concurrently, preventing cross-talk on simulation output/log topics. This closes a long-standing coverage gap around the multi-simulation topic isolation fix.
Changes:
- Add
SimulationContainerTest.testRunTwoConcurrentSimulationsWithTopicIsolationto assert per-simulation log/output topic isolation under concurrency. - Add a dedicated
make test-simulation-containertarget to run only the externally-started-stack simulation container tests. - Wire the new simulation-container test target into the Integration Tests GitHub Actions workflow.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| Makefile | Adds test-simulation-container target and help text to run only simulation-tagged container tests against an externally started stack. |
| gov.pnnl.goss.gridappsd/test/gov/pnnl/goss/gridappsd/SimulationContainerTest.java | Introduces a two-concurrent-simulation topic-isolation regression test with dedicated topic watchers and DataResponse unwrapping. |
| .github/workflows/integration-tests.yml | Runs make test-simulation-container in CI after bringing the docker-compose stack up and verifying readiness. |
Suppressed comments (1)
gov.pnnl.goss.gridappsd/test/gov/pnnl/goss/gridappsd/SimulationContainerTest.java:479
- This assignment references
watcher.expectedSimulationId, but that field is otherwise unused. If the field is removed, this line should be dropped as well.
watcher.expectedSimulationId = simulationId;
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+376
to
+378
| final String label; | ||
| volatile String expectedSimulationId; | ||
| final List<JsonObject> outputMessages = new CopyOnWriteArrayList<>(); |
| * that fix, SimulationManager and helics_goss_bridge published to one shared | ||
| * topic across simulations, so concurrent instances corrupted each other's | ||
| * status. The fix suffixes every per-simulation topic with "." + simulationId | ||
| * (SimulationManagerImpl.java:224, SimulationProcess. java:217,232,255,290, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a regression guard for per-simulation topic isolation, the behaviour
delivered by @poorva1209 in
faa2d4e0via #1758 (merged 2024-01-29). That fixhas been running in
developsince January 2024 with no test covering it.Closes the coverage gap noted when #1757 was closed.
What it guards
SimulationManagerandhelics_goss_bridgeoriginally shared one topic acrosssimulations, so concurrent instances corrupted each other's status. The fix
suffixes the simulation id:
gov.pnnl.goss.gridappsd/src/gov/pnnl/goss/gridappsd/simulation/SimulationManagerImpl.java:224gov.pnnl.goss.gridappsd/src/gov/pnnl/goss/gridappsd/simulation/SimulationProcess.java:217,232,255,290services/helicsgossbridge/service/helics_goss_bridge.py:440,642,681Nothing asserted that, so dropping the suffix would ship silently.
The test
SimulationContainerTest.testRunTwoConcurrentSimulationsWithTopicIsolationStarts two simulations concurrently on separate
Clientconnections (a sharedJMS
Sessionis not safe across threads), subscribes each to its owntopic_simulationOutput.<simId>andtopic_simulationLog<simId>using exactstrings rather than wildcards, and asserts every message arriving on simulation
A's subscription carries A's own id and never B's.
How it fails if the suffix is removed: the exact-string subscriptions stop
matching anything the platform publishes, so the "at least one log message
received" assertion fails outright. If the suffix is present but a shared or
wrong id is used, a message intended for the other simulation lands on this
subscription and the per-message id equality assertion fails, printing the
mismatched id.
That failure mode is not hypothetical. The first run failed against a real bug in
the test itself:
subscribe()callbacks arrive wrapped in aDataResponseenvelope rather than as raw text. The assertions caught it, which is direct
evidence they are live rather than vacuous.
Concurrency uses a two-thread executor, a
CountDownLatchper watcher, andbounded shared-deadline waits. No sleeps, no polling. Both simulations use the
ieee123feeder, confirmed present by SPARQL before the run.TWO_SIM_TIMEOUT_SECONDS = SIMULATION_DURATION + 240.What it deliberately does NOT assert
Simulation completion. The existing single-simulation test has its completion and
measurement assertions commented out as infrastructure-dependent, and that turns
out to be accurate rather than flaky: running it unmodified gives
Completed: false, Total measurements received: 0, because the simulation diesabout five seconds in at
GLDZiploadScheduleConfigurationHandler.java:209withNo load schedule data in time series data store. Proven/InfluxDB has no seededload-schedule timeseries data, so
simulationManager.startSimulation()is neverreached.
That is a real, reproducible environment gap and is tracked separately. Asserting
completion here would produce a test that fails for reasons unrelated to topic
isolation and gets disabled again. The isolation assertion stands on its own and
is the behaviour this PR exists to protect.
Running it without building anything
CI published an image for this branch. Drop this file into
gridappsd-docker/docker-compose.d/and run./run.sh -t develop:Only the
gridappsdservice is repointed; blazegraph, viz, proven and influxdbstay on
develop.pull_policy: alwaysmatters, since a stale local layer underthe same tag must not win over the registry.
Note that only one
docker-compose.d/*.ymloverride should target thegridappsdservice at a time; disable any other before bringing the stack up.Image:
gridappsd/gridappsd:test_gadp-061-two-simulations, also tagged26080203_af05161, digestsha256:4bbf0005b8295caabd0260501e67f444b5f57776e1daf5d17c633360d6b23ab6.Verified by running the full stack from this override alone, with no bind mounts,
and confirming the container's goss-core jars are 16.0.1.
Results
New test passes; simulation ids distinct; log-topic isolation held. Full
SimulationContainerTestclass, 6 tests, passes.Test code only. No production changes.
An unrelated pre-existing failure surfaces on a full unfiltered
containerTestrun:
GridAppsDContainerTest.initializationErrorconflicts on fixed ports when astack is already running, because it manages its own Testcontainers lifecycle
while
GridAppsDTestEnvironmentexpects an externally started stack. Twoincompatible strategies share the
containertag. Untouched here, reproduciblewithout this diff.