only run R2dbcOffsetStore delete task while the projection is running - #620
Open
pjfanning wants to merge 1 commit into
Open
only run R2dbcOffsetStore delete task while the projection is running#620pjfanning wants to merge 1 commit into
pjfanning wants to merge 1 commit into
Conversation
Motivation:
`R2dbcOffsetStore` scheduled `deleteOldTimestampOffsets` with a fixed delay in
its constructor and discarded the returned `Cancellable`. The store is created
eagerly by every `R2dbcProjection` factory method, so the task starts before the
projection runs and keeps firing every `delete-interval` (1 minute by default)
for the lifetime of the `ActorSystem`, also after the projection has been
stopped. It also keeps the offset store, its source provider and its
`R2dbcExecutor` reachable, so each stopped projection is retained. This adds up
when projections are stopped and started, such as on ShardedDaemonProcess
rebalance.
Modification:
Keep the `Cancellable` and expose `startDeleteTask()`/`stopDeleteTask()` on the
offset store. `R2dbcRunningProjection` starts the task when it is created and
cancels it in `stop()`, so the deletes run while the projection is running and a
projection that is stopped and started again gets the task back.
Result:
No scheduled task, and no retained offset store, for a projection that has been
stopped or that was never run. Note that the deletes are not scheduled for the
`ProjectionTestKit` path, which uses `mappedSource()` rather than a
`RunningProjection`.
Tests:
- New R2dbcTimestampOffsetStoreSpec case "stop the periodic delete of old records when the projection is stopped", fails when stopDeleteTask does not cancel
- sbt "r2dbc-int-test/Test/testOnly *R2dbcTimestampOffsetStoreSpec" - 21 passed
- sbt "r2dbc-int-test/test" - 116 passed; RuntimePluginConfigSpec aborts in this environment ("database database1 does not exist"), same on the unmodified main branch
- sbt "r2dbc/mimaReportBinaryIssues" - success
- sbt checkCodeStyle - success
References:
None - found during a resource-leak audit of the main sources
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
R2dbcOffsetStorescheduleddeleteOldTimestampOffsetswith a fixed delay in its constructor and discarded the returnedCancellable:system.scheduler.scheduleWithFixedDelay( settings.deleteInterval, settings.deleteInterval, () => deleteOldTimestampOffsets(), system.executionContext)The store is created eagerly by every
R2dbcProjectionfactory method (R2dbcProjectionImpl.createOffsetStore), so the task starts before the projection runs, and nothing ever cancels it. It keeps firing everydelete-interval(1 minuteby default) for the lifetime of theActorSystem, also after the projection has been stopped, and it keeps the offset store, its source provider and itsR2dbcExecutorreachable, so each stopped projection is retained. This adds up when projections are stopped and started, such as on ShardedDaemonProcess rebalance.Modification
Keep the
Cancellableand exposestartDeleteTask()/stopDeleteTask()on the offset store.R2dbcRunningProjectionstarts the task when it is created and cancels it instop(), so the deletes run while the projection is running, and a projection that is stopped and started again (ProjectionBehaviordoes this for the offset management commands) gets the task back.Result
No scheduled task, and no retained offset store, for a projection that has been stopped or that was never run.
One behaviour note: the deletes are not scheduled for the
ProjectionTestKitpath, which runsmappedSource()without aRunningProjection. Tests that exercise deletion call the offset store directly, andR2dbcTimestampOffsetStoreSpec"periodically delete old records" now starts the task explicitly.Tests
R2dbcTimestampOffsetStoreSpeccase "stop the periodic delete of old records when the projection is stopped": counts the scheduled delete ticks through a probe, asserts they stop afterstopDeleteTask()and resume after a laterstartDeleteTask(). It fails ifstopDeleteTask()does not cancel (verified by neutering the cancel).sbt "r2dbc-int-test/Test/testOnly *R2dbcTimestampOffsetStoreSpec"- 21 passedsbt "r2dbc-int-test/test"- 116 passed.RuntimePluginConfigSpecaborts in my environment withdatabase "database1" does not exist; it aborts identically on the unmodified main branch, so it is unrelated to this change.sbt "r2dbc/mimaReportBinaryIssues"- successsbt checkCodeStyle- successReferences
None - found during a resource-leak audit of the main sources