feat: add optional Glue retry with exponential backoff and HTTP-level metrics - #144
Open
jamespfaulkner wants to merge 5 commits into
Open
feat: add optional Glue retry with exponential backoff and HTTP-level metrics #144jamespfaulkner wants to merge 5 commits into
jamespfaulkner wants to merge 5 commits into
Conversation
… metrics Glue calls occasionally fail with ConcurrentModificationException or timeouts. Retries are off by default (safe for HMS threads) and enabled via GLUE_RETRY_ENABLED=true for Dronefly. A RequestHandler2-based GlueMetricRequestHandler records per-operation call duration and error counts via Micrometer for full observability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ation - Switch GlueMetricRequestHandler to beforeAttempt/afterAttempt hooks so metrics fire per HTTP attempt (including retries) rather than once per logical call - Clarify 'retries disabled' log: SDK default retries still apply when custom policy is off; the custom policy adds CME retry on top - Fix GlueSyncCli to share a single AWSGlue client across ApiaryGlueSync, GluePartitionService, and GlueDatabaseService so the 600s request timeout applies to all table operations - Rename maxAttempts -> maxRetries throughout to match SDK semantics (maxErrorRetry is a retry count, not total attempts) - Unify retry metric tag namespace: use getErrorCode() for AmazonServiceException in both retry condition branches Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…counter only Per-call duration and error total metrics via RequestHandler2 were removed in favour of the existing ApiaryGlueSync success/failure counters, which already provide sufficient operational coverage. The glue_listener_retry_attempt counter from GlueClientFactory is retained to surface CME and throttle retry activity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jamespfaulkner
force-pushed
the
feat/glue-retry-and-client-metrics
branch
from
August 5, 2026 14:41
124317f to
81abea5
Compare
Add missing CHANGELOG entry for the Glue retry feature, document the new GLUE_RETRY_ENABLED/GLUE_RETRY_MAX_ATTEMPTS env vars in the README, fix a javadoc typo (GLUE_RETRY_MAX_RETRIES -> GLUE_RETRY_MAX_ATTEMPTS), and log a warning when GLUE_RETRY_MAX_ATTEMPTS is zero/negative instead of silently falling back to the default. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
jamespfaulkner
marked this pull request as ready for review
August 5, 2026 16:15
javsanbel2
reviewed
Aug 6, 2026
Address PR review comment asking for the retried exception(s) to be configurable instead of hardcoded, while defaulting to ConcurrentModificationException when the env var is unset. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GlueClientFactoryto centraliseAWSGlueclient construction with optional exponential-backoff retry supportGLUE_RETRY_ENABLED=true. Retries onConcurrentModificationException(configurable viaGLUE_RETRY_EXCEPTIONS) and standard transient AWS errors usingPredefinedRetryPolicies.DEFAULT_BACKOFF_STRATEGYglue_listener_retry_attemptMicrometer counter tagged byexception_typeApiaryGlueSync's CLI constructor building its own separate Glue client instead of reusing the one passed to itNew env vars
GLUE_RETRY_ENABLEDtrueto enable retries (Dronefly only)GLUE_RETRY_MAX_ATTEMPTS3GLUE_RETRY_EXCEPTIONSConcurrentModificationExceptionNew metrics
glue_listener_retry_attemptexception_typeTest plan
GlueClientFactoryTest— retry condition,ConcurrentModificationExceptionhandling, metric recording, disabled-by-default, configurable exceptionsMetricServiceTest— newrecordGlueRetryAttemptmethod covered🤖 Generated with Claude Code