diff --git a/.gitignore b/.gitignore index 1d8a56543..3b3083cf6 100644 --- a/.gitignore +++ b/.gitignore @@ -19,3 +19,4 @@ hostkey.ser /eclipse-classes .vscode/ .factorypath +graphify-out diff --git a/README.md b/README.md index f0cd014ee..0c54ce41c 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,6 @@ Each job can be configured with one Gerrit server. ## Maintainers * Robert Sandell - - robert.sandell@cloudbees.com - sandell.robert@gmail.com * Tomas Westling @@ -52,6 +51,13 @@ Run checkstyle mvn checkstyle:checkstyle +# Distributed Event Management support + +The plugin supports an distributed event management for the memory that will track the events (for example using Hazelcast client). +See [README_DISTRIBUTED_EVENT_MANAGEMENT.md](docs/README_DISTRIBUTED_EVENT_MANAGEMENT.md) for configuration +properties and deployment examples. + + # License The MIT License diff --git a/docs/README_DISTRIBUTED_EVENT_MANAGEMENT.md b/docs/README_DISTRIBUTED_EVENT_MANAGEMENT.md new file mode 100644 index 000000000..a5d562a03 --- /dev/null +++ b/docs/README_DISTRIBUTED_EVENT_MANAGEMENT.md @@ -0,0 +1,107 @@ +# Distributed Event Management support + +The plugin supports Distributed Event Management support where two or more replicas or nodes of a logical Jenkins(*) instance +run in parallel (sharing the Gerrit memory of the plugin). When enabled, a Hazelcast +cluster coordinates the instances so that: + +- Each Gerrit event is processed by **exactly one** instance (event claiming) +- Build state is shared across instances (distributed build memory) +- Gerrit feedback (votes and comments) are sent **exactly once** per build event + +By default, the plugin runs in **local mode** and requires no additional configuration. +Local mode is fully backward-compatible with single-instance Jenkins deployments. + +Alternative coordination backends can be implemented by extending +[`CoordinationModeProvider`](../src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java) +— a Jenkins `ExtensionPoint` that wires together the storage, event-claiming, and +notification-claiming strategies for a given coordination mode. A higher `@Extension` +ordinal takes precedence over the built-in Hazelcast provider. + +## Hazelcast implementation + +Hazelcast mode is activated via a JVM system property. Jenkins connects as a lightweight +client to a Hazelcast sidecar container, reusing the cross-pod cluster the sidecar +maintains. + +### Configuration Properties + +All distributed storage settings are controlled by JVM system properties passed to Jenkins on startup. + +| Property | Default | Description | +|---|---|-------------------------------------------------------| +| `gerrit.trigger.coordination.mode` | `local` | Set to `hazelcast` to enable distributed coordination | +| `gerrit.trigger.coordination.hazelcast.client.addresses` | `localhost:5702` | Comma-separated `host:port` list of sidecar addresses | +| `gerrit.trigger.coordination.hazelcast.client.cluster.name` | `gerrit-trigger-cluster` | Cluster name to connect to | + +Port `5702` is used by default to avoid potential conflicts with other Hazelcast cluster, which could occupy port `5701`. + +Cluster name must be different for each logical instance. Multiple replicas or nodes of a logical instance may configure the same cluster name. Different logical instances require separate cluster names. + +### Configuration Example + +#### Kubernetes — Client Mode with Hazelcast Sidecar + +The plugin can connect to Hazelcast cluster as a lightweight client. For example, if we are running +K8s environment with the Jenkins instance inside a pod, we can have a side-container with Hazelcast +to set up the Hazelcast cluster. In this kind of cases, we would the a configuration setup similar +to the following one: + +Add the following JVM arguments to the Jenkins instance: + + -Dgerrit.trigger.coordination.mode=hazelcast + -Dgerrit.trigger.coordination.hazelcast.client.addresses=localhost:5702 + -Dgerrit.trigger.coordination.hazelcast.client.cluster.name=gerrit-trigger-cluster + +Add the sidecar container to the instance pod spec: + +```yaml +- name: hazelcast + image: hazelcast/hazelcast:5.3.8 + ports: + - containerPort: 5702 + name: hazelcast + env: + - name: JAVA_OPTS + value: >- + -Dhazelcast.config=/dev/stdin + -Dhazelcast.local.publicAddress=$(POD_IP):5702 + - name: HZ_CLUSTERNAME + value: gerrit-trigger-cluster + - name: HZ_NETWORK_PORT_PORT + value: "5702" +``` + +Grant the pod's service account read access to Kubernetes endpoints so Hazelcast can +discover its peers: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: hazelcast-gerrit-trigger +rules: + - apiGroups: [""] + resources: ["endpoints", "pods", "nodes", "services"] + verbs: ["get", "list"] + - apiGroups: ["discovery.k8s.io"] + resources: ["endpointslices"] + verbs: ["get", "list"] +``` + +#### Kubernetes — Client Mode with Separate Hazelcast Cluster + +For larger deployments or strict separation of concerns, you can decouple the coordination layer by running a standalone Hazelcast cluster. Jenkins still connects as a lightweight client, but routes traffic to the separate cluster via a Kubernetes service instead of a sidecar. + +Add the following JVM arguments to the Jenkins instance, updating the client address to point to your standalone Hazelcast Kubernetes service (replace hazelcast-service.default.svc.cluster.local with your actual service DNS and namespace, along with the cluster name for your logical instance): + + -Dgerrit.trigger.coordination.mode=hazelcast + -Dgerrit.trigger.coordination.hazelcast.client.addresses=hazelcast-service.default.svc.cluster.local:5702 + -Dgerrit.trigger.coordination.hazelcast.client.cluster.name=gerrit-trigger-cluster- + +In this topology: +- You do not need to add the sidecar container to the Jenkins pod spec. +- The Jenkins service account does not need RBAC permissions for peer discovery, as cluster management is handled entirely by the standalone Hazelcast nodes. +- You must deploy and manage the Hazelcast cluster independently (e.g. via the official Hazelcast Helm chart), ensuring you configure it to match your expected `HZ_CLUSTERNAME` and port (`5702`). + +(*) Jenkins does not support multiple replicas or nodes for a single logical instance, this feature is not tested with Jenkins. This feature is provided for CloudBees CI (Enterprise Jenkins). +This feature is provided as a community effort and is not endorsed or officially supported by CloudBees. diff --git a/pom.xml b/pom.xml index 0ae2c0a01..586191f65 100644 --- a/pom.xml +++ b/pom.xml @@ -67,6 +67,7 @@ 3 0.5C High + 5.3.8 @@ -107,6 +108,11 @@ + + com.hazelcast + hazelcast + ${hazelcast.version} + io.jenkins.plugins gson-api @@ -203,6 +209,12 @@ workflow-support test + + org.jenkins-ci.plugins.workflow + workflow-support + tests + test + org.jenkins-ci.plugins @@ -370,6 +382,42 @@ + + + + test-hazelcast + + + + maven-surefire-plugin + + false + 1 + + + hazelcast + + + + + + + + scm:git:https://github.com/${gitHubRepo}.git scm:git:git@github.com:${gitHubRepo}.git diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/PluginImpl.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/PluginImpl.java index ed6670dd0..9d5292bba 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/PluginImpl.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/PluginImpl.java @@ -24,8 +24,11 @@ */ package com.sonyericsson.hudson.plugins.gerrit.trigger; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.LocalCoordinationProvider; import com.sonyericsson.hudson.plugins.gerrit.trigger.dependency.DependencyQueueTaskDispatcher; import com.sonyericsson.hudson.plugins.gerrit.trigger.replication.ReplicationQueueTaskDispatcher; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider; import com.sonymobile.tools.gerrit.gerritevents.GerritHandler; import com.sonymobile.tools.gerrit.gerritevents.GerritSendCommandQueue; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; @@ -583,6 +586,18 @@ public void start() { logger.info("Starting Gerrit-Trigger Plugin"); logger.trace("Loading configs"); load(); + + // Initialize coordination providers early (before any code that might use CoordinationModeFactory) + // This must happen before BuildMemory, EventClaimStrategy, or NotificationClaimStrategy are used + // because provider.isAvailable() may check if resources are initialized + initializeCoordinationProviders(); + + // Eagerly initialize CoordinationModeFactory so discoverMode() runs now (during startup) + // rather than lazily on first event — deferred initialization can add several seconds of + // latency to the first build trigger when ExtensionList.lookup() is called from a + // background event-processing thread. + CoordinationModeFactory.get().getStorage(); + GerritSendCommandQueue.initialize(pluginConfig); gerritEventManager = new JenkinsAwareGerritHandler(pluginConfig.getNumberOfReceivingWorkerThreads()); for (GerritServer s : servers) { @@ -591,6 +606,70 @@ public void start() { active = true; } + /** + * Initialize the active coordination mode provider. + *

+ * This is called early in plugin startup, before any code that might use + * CoordinationModeFactory. Only initializes the provider that matches the configured + * coordination mode, making it more efficient than calling initialize() on all providers. + *

+ * Implementation Note: We cannot use {@code provider.isAvailable()} + * before initialization because isAvailable() checks if the provider is actually initialized. + * Instead, we check the configured mode directly and initialize the matching provider. + * After initialization, isAvailable() will return true. + *

+ * Fails gracefully - if a provider's initialization fails, it will not be available + * and the factory will fall back to the next highest-priority provider. + */ + private void initializeCoordinationProviders() { + ExtensionList providers = ExtensionList.lookup(CoordinationModeProvider.class); + String configuredMode = CoordinationModeProvider.getConfiguredMode(); + logger.debug("Configured coordination mode: {}", configuredMode); + + // Try to initialize the configured (non-local) provider first. + // Each provider's initialize() internally checks the configured mode and is a no-op + // if the mode doesn't match, so we try non-local providers in priority order. + // Local is handled separately below as the explicit fallback. + if (!"local".equalsIgnoreCase(configuredMode)) { + for (CoordinationModeProvider provider : providers) { + if (provider instanceof LocalCoordinationProvider) { + continue; + } + try { + logger.info("Initializing coordination provider: {}", provider.getModeName()); + provider.initialize(); + if (provider.isAvailable()) { + logger.info("Provider {} initialized successfully", provider.getModeName()); + return; + } + } catch (Exception e) { + logger.warn("Failed to initialize {} coordination provider. Falling back to Local.", + provider.getModeName(), e); + break; + } + } + } + + // Explicit fallback to LocalCoordinationProvider, which is always available. + for (CoordinationModeProvider provider : providers) { + if (provider instanceof LocalCoordinationProvider) { + if ("local".equalsIgnoreCase(configuredMode)) { + logger.info("Initializing LocalCoordinationProvider"); + } else { + logger.info("Initializing LocalCoordinationProvider (fallback after failed initialization)"); + } + try { + provider.initialize(); + } catch (Exception e) { + logger.error("Failed to initialize LocalCoordinationProvider - this should never happen", e); + } + return; + } + } + + logger.error("LocalCoordinationProvider not found - this should never happen"); + } + /** * Forces initialization of the Dispatchers. * @@ -674,6 +753,11 @@ protected static void doXStreamRegistrations() { */ public void stop() { active = false; + + // Shutdown coordination providers before stopping servers + // This ensures any coordination operations are cleaned up before servers disconnect + shutdownCoordinationProviders(); + for (GerritServer s : servers) { s.stop(); } @@ -686,6 +770,30 @@ public void stop() { servers.clear(); } + /** + * Shutdown all coordination mode providers. + *

+ * Called during plugin shutdown to clean up coordination resources. + * Fails gracefully - errors are logged but don't prevent plugin shutdown. + */ + private void shutdownCoordinationProviders() { + logger.debug("Shutting down coordination providers..."); + for (com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider provider + : hudson.ExtensionList.lookup( + com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider.class)) { + try { + logger.debug("Shutting down provider: {}", provider.getModeName()); + provider.shutdown(); + logger.debug("Provider {} shut down successfully", provider.getModeName()); + } catch (Exception e) { + logger.warn("Error shutting down coordination provider: {} (non-critical, continuing shutdown)", + provider.getModeName(), e); + // Continue with other providers even if one fails + } + } + logger.debug("Coordination provider shutdown complete"); + } + /** * Startup hook. */ diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/CoordinationModeFactory.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/CoordinationModeFactory.java index 56405df78..bf34362a0 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/CoordinationModeFactory.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/CoordinationModeFactory.java @@ -25,9 +25,13 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.BuildMemoryStorage; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.QueueCancellationStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.storage.LocalBuildMemoryStorage; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalEventClaimStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalNotificationClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalQueueCancellationStrategy; import hudson.Extension; import hudson.ExtensionList; import org.slf4j.Logger; @@ -83,6 +87,7 @@ * @see CoordinationModeProvider * @see BuildMemoryStorage * @see NotificationClaimStrategy + * @see EventClaimStrategy */ @Extension public class CoordinationModeFactory { @@ -107,6 +112,18 @@ public class CoordinationModeFactory { */ private volatile NotificationClaimStrategy claimStrategy; + /** + * The event claim strategy instance, lazily initialized. + * Instance field - managed by Jenkins lifecycle, not static. + */ + private volatile EventClaimStrategy eventClaimStrategy; + + /** + * The queue cancellation strategy instance, lazily initialized. + * Instance field - managed by Jenkins lifecycle, not static. + */ + private volatile QueueCancellationStrategy queueCancellationStrategy; + /** * Constructor - called by Jenkins once per Jenkins instance. * Public constructor allows Jenkins to instantiate via @Extension mechanism. @@ -182,6 +199,39 @@ public NotificationClaimStrategy getClaimStrategy() { return claimStrategy; } + /** + * Gets the EventClaimStrategy instance for the current coordination mode. + * + *

Uses double-checked locking for thread-safe lazy initialization. + * The mode is discovered and instances are created on first access.

+ * + *

The EventClaimStrategy prevents duplicate build processing when multiple Jenkins + * instances receive the same Gerrit event in distributed scenarios.

+ * + * @return the event claim strategy implementation + * @throws IllegalStateException if no available mode provider is found + */ + @NonNull + public EventClaimStrategy getEventClaimStrategy() { + ensureInitialized(); + return eventClaimStrategy; + } + + /** + * Gets the QueueCancellationStrategy instance for the current coordination mode. + * + *

Uses double-checked locking for thread-safe lazy initialization. + * The mode is discovered and instances are created on first access.

+ * + * @return the queue cancellation strategy implementation + * @throws IllegalStateException if no available mode provider is found + */ + @NonNull + public QueueCancellationStrategy getQueueCancellationStrategy() { + ensureInitialized(); + return queueCancellationStrategy; + } + /** * Ensures the factory is initialized by discovering the mode if needed. * Uses double-checked locking for thread safety. @@ -246,12 +296,16 @@ private void discoverMode() { selectedMode = selectedProvider; - // Create both implementations from the selected mode + // Create all three implementations from the selected mode storage = selectedProvider.createStorage(); claimStrategy = selectedProvider.createClaimStrategy(); + eventClaimStrategy = selectedProvider.createEventClaimStrategy(); + queueCancellationStrategy = selectedProvider.createQueueCancellationStrategy(); logger.info("Created BuildMemoryStorage: {}", storage.getClass().getSimpleName()); logger.info("Created NotificationClaimStrategy: {}", claimStrategy.getClass().getSimpleName()); + logger.info("Created EventClaimStrategy: {}", eventClaimStrategy.getClass().getSimpleName()); + logger.info("Created QueueCancellationStrategy: {}", queueCancellationStrategy.getClass().getSimpleName()); } catch (Exception e) { logger.warn("Failed to discover mode via ExtensionList, using fallback", e); @@ -267,6 +321,8 @@ private void createFallbackMode() { logger.info("Using fallback local mode (ExtensionList unavailable)"); storage = new LocalBuildMemoryStorage(); claimStrategy = new LocalNotificationClaimStrategy(); + eventClaimStrategy = new LocalEventClaimStrategy(); + queueCancellationStrategy = new LocalQueueCancellationStrategy(); selectedMode = null; // No provider in fallback mode } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/LocalCoordinationProvider.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/LocalCoordinationProvider.java index 9411b09fb..2a4a9d94d 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/LocalCoordinationProvider.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/LocalCoordinationProvider.java @@ -23,10 +23,14 @@ */ package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalEventClaimStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalNotificationClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.LocalQueueCancellationStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.BuildMemoryStorage; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.QueueCancellationStrategy; import com.sonyericsson.hudson.plugins.gerrit.trigger.storage.LocalBuildMemoryStorage; import hudson.Extension; @@ -45,11 +49,10 @@ * * @see LocalBuildMemoryStorage * @see LocalNotificationClaimStrategy + * @see LocalEventClaimStrategy * @see CoordinationModeFactory */ -// CHECKSTYLE:OFF MagicNumber - Ordinal must be literal in annotation, -1000 ensures fallback priority @Extension(ordinal = LocalCoordinationProvider.FALLBACK_PRIORITY) -// CHECKSTYLE:ON MagicNumber public class LocalCoordinationProvider extends CoordinationModeProvider { /** @@ -100,4 +103,46 @@ public BuildMemoryStorage createStorage() { public NotificationClaimStrategy createClaimStrategy() { return new LocalNotificationClaimStrategy(); } + + /** + * Creates a new local event claim strategy instance. + * Always succeeds and executes the action immediately - no coordination needed in standalone mode. + * + * @return a new LocalEventClaimStrategy + */ + @Override + public EventClaimStrategy createEventClaimStrategy() { + return new LocalEventClaimStrategy(); + } + + /** + * Creates a new local queue cancellation strategy instance. + * Always returns false - no distributed load balancer present in standalone mode. + * + * @return a new LocalQueueCancellationStrategy + */ + @Override + public QueueCancellationStrategy createQueueCancellationStrategy() { + return new LocalQueueCancellationStrategy(); + } + + /** + * Initializes local coordination mode. + *

+ * Local mode requires no initialization - no external resources to set up. + */ + @Override + public void initialize() { + // Local mode needs no initialization + } + + /** + * Shuts down local coordination mode. + *

+ * Local mode requires no shutdown - no external resources to release. + */ + @Override + public void shutdown() { + // Local mode needs no shutdown + } } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EntryDataSerializer.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EntryDataSerializer.java new file mode 100644 index 000000000..c06527847 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EntryDataSerializer.java @@ -0,0 +1,90 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.nio.serialization.compact.CompactReader; +import com.hazelcast.nio.serialization.compact.CompactSerializer; +import com.hazelcast.nio.serialization.compact.CompactWriter; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.EntryData; +import edu.umd.cs.findbugs.annotations.NonNull; + +/** + * Hazelcast Compact Serializer for {@link EntryData}. + *

+ * Serializes individual BuildMemory entries with fixed schema. + * + */ +public class EntryDataSerializer implements CompactSerializer { + + /** + * Type name for schema registration. + * Uses fully-qualified name to prevent conflicts in shared Hazelcast clusters. + */ + private static final String TYPE_NAME = "com.sonyericsson.gerrit.trigger.EntryData"; + + @Override + @NonNull + public EntryData read(@NonNull CompactReader reader) { + EntryData entry = new EntryData(); + entry.setProjectFullName(reader.readString("projectFullName")); + entry.setBuildId(reader.readString("buildId")); + entry.setBuildCompleted(reader.readBoolean("buildCompleted")); + entry.setCancelling(reader.readBoolean("cancelling")); + entry.setCancelled(reader.readBoolean("cancelled")); + entry.setQueueLeft(reader.readBoolean("queueLeft")); + entry.setCustomUrl(reader.readString("customUrl")); + entry.setUnsuccessfulMessage(reader.readString("unsuccessfulMessage")); + entry.setTriggeredTimestamp(reader.readInt64("triggeredTimestamp")); + entry.setCompletedTimestamp(reader.readNullableInt64("completedTimestamp")); + entry.setStartedTimestamp(reader.readNullableInt64("startedTimestamp")); + return entry; + } + + @Override + public void write(@NonNull CompactWriter writer, @NonNull EntryData entry) { + writer.writeString("projectFullName", entry.getProjectFullName()); + writer.writeString("buildId", entry.getBuildId()); + writer.writeBoolean("buildCompleted", entry.isBuildCompleted()); + writer.writeBoolean("cancelling", entry.isCancelling()); + writer.writeBoolean("cancelled", entry.isCancelled()); + writer.writeBoolean("queueLeft", entry.isQueueLeft()); + writer.writeString("customUrl", entry.getCustomUrl()); + writer.writeString("unsuccessfulMessage", entry.getUnsuccessfulMessage()); + writer.writeInt64("triggeredTimestamp", entry.getTriggeredTimestamp()); + writer.writeNullableInt64("completedTimestamp", entry.getCompletedTimestamp()); + writer.writeNullableInt64("startedTimestamp", entry.getStartedTimestamp()); + } + + @Override + @NonNull + public String getTypeName() { + return TYPE_NAME; + } + + @Override + @NonNull + public Class getCompactClass() { + return EntryData.class; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaim.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaim.java new file mode 100644 index 000000000..c4af702a4 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaim.java @@ -0,0 +1,144 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +/** + * Represents a claimed Gerrit event in the distributed cluster. + *

+ * In distributed environments with multiple replicas, each Gerrit event + * arrives at all replicas. To prevent duplicate builds, replicas use event claiming: + * the first replica to claim an event processes it, while others skip it. + *

+ * EventClaim objects are stored in a Hazelcast IMap to coordinate claims across replicas. + * Claims automatically expire via TTL to prevent memory leaks. + *

+ * Serialization: Uses Hazelcast Compact Serialization for cross-JVM + * compatibility with sidecar deployment. The sidecar Hazelcast cluster doesn't need + * this class in its classpath. + * + */ +public class EventClaim { + + /** + * Unique event identifier (generated by {@link EventIdGenerator}). + */ + private final String eventId; + + /** + * Jenkins instance (hostname/pod name) that claimed the event. + */ + private final String claimedBy; + + /** + * Timestamp when event was claimed (milliseconds since epoch). + */ + private final long claimedAt; + + /** + * Event type for logging and debugging (e.g., "patchset-created"). + */ + private final String eventType; + + /** + * Standard constructor. + * + * @param eventId unique event identifier + * @param claimedBy instance that claimed the event + * @param claimedAt timestamp when claimed + * @param eventType Gerrit event type + */ + public EventClaim(String eventId, String claimedBy, long claimedAt, String eventType) { + this.eventId = eventId; + this.claimedBy = claimedBy; + this.claimedAt = claimedAt; + this.eventType = eventType; + } + + /** + * Gets the unique event identifier. + * + * @return event ID + */ + public String getEventId() { + return eventId; + } + + /** + * Gets the instance that claimed the event. + * + * @return claimant instance identifier + */ + public String getClaimedBy() { + return claimedBy; + } + + /** + * Gets the timestamp when the event was claimed. + * + * @return claim timestamp (milliseconds since epoch) + */ + public long getClaimedAt() { + return claimedAt; + } + + /** + * Gets the Gerrit event type. + * + * @return event type + */ + public String getEventType() { + return eventType; + } + + @Override + public String toString() { + return String.format("EventClaim[id=%s, by=%s, at=%d, type=%s]", + eventId, claimedBy, claimedAt, eventType); + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + EventClaim that = (EventClaim)o; + if (eventId != null) { + return eventId.equals(that.eventId); + } else { + return that.eventId == null; + } + } + + @Override + public int hashCode() { + if (eventId != null) { + return eventId.hashCode(); + } else { + return 0; + } + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaimSerializer.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaimSerializer.java new file mode 100644 index 000000000..e1eb5897f --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaimSerializer.java @@ -0,0 +1,80 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.nio.serialization.compact.CompactReader; +import com.hazelcast.nio.serialization.compact.CompactSerializer; +import com.hazelcast.nio.serialization.compact.CompactWriter; +import edu.umd.cs.findbugs.annotations.NonNull; + +/** + * Hazelcast Compact Serializer for {@link EventClaim}. + *

+ * Compact Serialization is schema-based and doesn't require class definitions + * on the Hazelcast server (sidecar container). This enables cross-JVM serialization + * without classloading issues. + *

+ * The serializer writes a schema with field names and types, which the sidecar + * Hazelcast can process without needing the EventClaim class. + * + */ +public class EventClaimSerializer implements CompactSerializer { + + /** + * Type name for schema registration. + * Uses fully-qualified name to prevent conflicts in shared Hazelcast clusters. + */ + private static final String TYPE_NAME = "com.sonyericsson.gerrit.trigger.EventClaim"; + + @Override + @NonNull + public EventClaim read(@NonNull CompactReader reader) { + String eventId = reader.readString("eventId"); + String claimedBy = reader.readString("claimedBy"); + long claimedAt = reader.readInt64("claimedAt"); + String eventType = reader.readString("eventType"); + + return new EventClaim(eventId, claimedBy, claimedAt, eventType); + } + + @Override + public void write(@NonNull CompactWriter writer, @NonNull EventClaim claim) { + writer.writeString("eventId", claim.getEventId()); + writer.writeString("claimedBy", claim.getClaimedBy()); + writer.writeInt64("claimedAt", claim.getClaimedAt()); + writer.writeString("eventType", claim.getEventType()); + } + + @Override + @NonNull + public String getTypeName() { + return TYPE_NAME; + } + + @Override + @NonNull + public Class getCompactClass() { + return EventClaim.class; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventIdGenerator.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventIdGenerator.java new file mode 100644 index 000000000..42f3a58cc --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventIdGenerator.java @@ -0,0 +1,249 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.sonymobile.tools.gerrit.gerritevents.dto.attr.Change; +import com.sonymobile.tools.gerrit.gerritevents.dto.attr.PatchSet; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.ChangeBasedEvent; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.RefUpdated; + +/** + * Utility class for generating unique, consistent event identifiers. + *

+ * Event IDs are used for distributed event claiming in distributed scenarios. + * The same Gerrit event arriving at different replicas must produce the same event ID + * to enable proper claim coordination. + *

+ * Event IDs are deterministic and based on immutable event properties such as: + *

    + *
  • Change number and patchset number (for change-based events)
    • + *
  • Project and ref name (for ref-updated events)
    • + *
  • Event type
    • + *
  • Server-side timestamp (eventCreatedOn from Gerrit server)
    • + *
+ *

+ * Important: Uses {@code eventCreatedOn} (server timestamp) rather than {@code receivedOn} + * (replica timestamp) to ensure identical event IDs across all replicas receiving the same event. + * + */ +public final class EventIdGenerator { + + /** + * Length of short Git revision hash (first 8 characters). + */ + private static final int SHORT_REVISION_LENGTH = 8; + + /** + * Private constructor to prevent instantiation. + */ + private EventIdGenerator() { + // Utility class + } + + /** + * Generates a unique identifier for a Gerrit event. + *

+ * The ID is deterministic - the same event on different replicas produces the same ID. + * This is critical for distributed event claiming to work correctly. + * + * @param event the Gerrit event + * @return unique event identifier + */ + public static String generateEventId(GerritTriggeredEvent event) { + if (event instanceof ChangeBasedEvent) { + return generateChangeBasedEventId((ChangeBasedEvent)event); + } + + if (event instanceof RefUpdated) { + return generateRefUpdatedEventId((RefUpdated)event); + } + + // Fallback for other event types + return generateFallbackEventId(event); + } + + /** + * Generates ID for change-based events (patchset-created, comment-added, etc.). + *

+ * Prefers {@code changeId} over deprecated {@code change.getNumber()}. + * Includes branch since changeId is not unique across branches. + * + * @param event the change-based event + * @return event ID in format: change-{project}-{changeId}-{branch}-{patchset}-{type}-{timestamp} + */ + private static String generateChangeBasedEventId(ChangeBasedEvent event) { + Change change = event.getChange(); + PatchSet patchSet = event.getPatchSet(); + + if (change == null || patchSet == null) { + return generateFallbackEventId(event); + } + + // Use server-side timestamp (eventCreatedOn) for consistency across replicas + // Fall back to receivedOn if eventCreatedOn is not available + long timestamp = getEventTimestamp(event); + + // Prefer changeId (I...) over deprecated change number + // Note: changeId is not unique across branches, so branch must be included + String changeIdentifier; + if (change.getId() != null && !change.getId().isEmpty()) { + // Use Change-Id (format: I1234567890abcdef...) + changeIdentifier = sanitize(change.getId()); + } else { + // Fallback to change number if changeId not available (old Gerrit versions) + changeIdentifier = "num-" + change.getNumber(); + } + + // Include branch for uniqueness (changeId can be reused across branches) + String branch = "unknown"; + if (change.getBranch() != null && !change.getBranch().isEmpty()) { + branch = sanitize(change.getBranch()); + } + + // Format: change-{project}-{changeId}-{branch}-{patchset}-{type}-{timestamp} + return String.format("change-%s-%s-%s-%s-%s-%d", + sanitize(change.getProject()), + changeIdentifier, + branch, + patchSet.getNumber(), + sanitizeEventType(event.getEventType().getTypeValue()), + timestamp); + } + + /** + * Generates ID for ref-updated events. + * + * @param event the ref-updated event + * @return event ID in format: ref-{project}-{refName}-{shortRev}-{timestamp} + */ + private static String generateRefUpdatedEventId(RefUpdated event) { + if (event.getRefUpdate() == null) { + return generateFallbackEventId(event); + } + + String project = sanitize(event.getRefUpdate().getProject()); + String refName = sanitize(event.getRefUpdate().getRefName()); + String newRev = event.getRefUpdate().getNewRev(); + + // Use first 8 chars of revision (short hash) + String shortRev; + if (newRev != null && newRev.length() >= SHORT_REVISION_LENGTH) { + shortRev = newRev.substring(0, SHORT_REVISION_LENGTH); + } else { + shortRev = "unknown"; + } + + // Use server-side timestamp (eventCreatedOn) for consistency across replicas + // Fall back to receivedOn if eventCreatedOn is not available + long timestamp = getEventTimestamp(event); + + // Format: ref---- + return String.format("ref-%s-%s-%s-%d", + project, + refName, + shortRev, + timestamp); + } + + /** + * Generates fallback ID for events that don't match known patterns. + *

+ * Uses {@code event.hashCode()} for uniqueness. All events in the gerrit-events library + * implement their own content-based {@code hashCode()}, so this is stable across replicas. + * If a specific event type has a weak {@code hashCode()}, the fix belongs in the + * gerrit-events library. + * + * @param event the event + * @return event ID in format: event-{type}-{server}-{timestamp}-{hash} + */ + private static String generateFallbackEventId(GerritTriggeredEvent event) { + // Use server-side timestamp (eventCreatedOn) for consistency across replicas + // Fall back to receivedOn if eventCreatedOn is not available + long timestamp = getEventTimestamp(event); + + // Get server name for additional uniqueness + String serverName = "unknown"; + if (event.getProvider() != null && event.getProvider().getName() != null) { + serverName = sanitize(event.getProvider().getName()); + } + + // Format: event---- + return String.format("event-%s-%s-%d-%08x", + sanitizeEventType(event.getEventType().getTypeValue()), + serverName, + timestamp, + event.hashCode()); + } + + /** + * Gets the event timestamp, preferring server-side eventCreatedOn over replica-local receivedOn. + *

+ * This ensures that the same Gerrit event produces the same event ID across all Jenkins replicas, + * which is critical for distributed event claiming to work correctly. + *

+ * Falls back to receivedOn if eventCreatedOn is null (defensive programming for older Gerrit + * versions or events that don't populate this field). + * + * @param event the Gerrit event + * @return timestamp in milliseconds since epoch + */ + private static long getEventTimestamp(GerritTriggeredEvent event) { + if (event.getEventCreatedOn() != null) { + // Prefer server-side timestamp (same across all replicas) + return event.getEventCreatedOn().getTime(); + } + // Fallback to replica-local timestamp (may differ between replicas) + return event.getReceivedOn(); + } + + /** + * Sanitizes a string to be safe for use in event ID. + * Replaces special characters with underscores. + * + * @param input the input string + * @return sanitized string safe for use in identifiers + */ + private static String sanitize(String input) { + if (input == null) { + return "null"; + } + // Replace non-alphanumeric characters (except dash and underscore) with underscore + return input.replaceAll("[^a-zA-Z0-9_-]", "_"); + } + + /** + * Sanitizes event type string. + * Converts to lowercase and replaces spaces/special chars with dashes. + * + * @param eventType the event type + * @return sanitized event type + */ + private static String sanitizeEventType(String eventType) { + if (eventType == null) { + return "unknown"; + } + return eventType.toLowerCase().replaceAll("[^a-z0-9-]", "-"); + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastBuildMemoryStorage.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastBuildMemoryStorage.java new file mode 100644 index 000000000..bd0b32a0f --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastBuildMemoryStorage.java @@ -0,0 +1,1309 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Copyright 2010 Sony Ericsson Mobile Communications. All rights reserved. + * Copyright 2012 Sony Mobile Communications AB. All rights reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import com.hazelcast.map.IMap; +import com.hazelcast.map.listener.EntryAddedListener; +import com.sonyericsson.hudson.plugins.gerrit.trigger.diagnostics.BuildMemoryReport; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.ToGerritRunListener; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildsStartedStats; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.EntryData; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.MemoryImprintData; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.AbandonedPatchsetInterruption; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritCause; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.NewPatchSetInterruption; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.PipelineAbortHelper; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.BuildMemoryStorage; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import hudson.model.Computer; +import hudson.model.Executor; +import hudson.model.Job; +import hudson.model.Result; +import hudson.model.Run; +import hudson.model.TaskListener; +import hudson.security.ACL; +import jenkins.model.CauseOfInterruption; +import hudson.security.ACLContext; +import jenkins.model.Jenkins; +import jenkins.util.Timer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import edu.umd.cs.findbugs.annotations.CheckForNull; +import edu.umd.cs.findbugs.annotations.NonNull; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * Hazelcast-backed implementation of BuildMemoryStorage for distributed scenarios. + *

+ * Uses distributed IMap for storing build memory across multiple Jenkins replicas. + * All operations use distributed locks ({@code map.tryLock(key, timeout, unit)}) to prevent race conditions. + *

+ * Note on EntryProcessors: EntryProcessors are intentionally not used. In Hazelcast + * client mode the processor class executes on the Hazelcast sidecar JVM, which does not have + * Jenkins or plugin classes on its classpath, causing {@link ClassNotFoundException} at runtime. + * Distributed locks provide equivalent atomicity guarantees without this constraint. + *

+ * This implementation is automatically selected when: + *

    + *
  • Coordination mode is set to 'hazelcast' via system property
    • + *
  • Hazelcast instance is available and running
    • + *
+ *

+ * Serialization Strategy (MemoryImprint ↔ MemoryImprintData): + *

+ * The API type + * ({@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint}) + * knows how to convert itself to and from its plain data form + * ({@link MemoryImprintData}) via + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint#toData()} + * and + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint#fromData(MemoryImprintData)}: + *

    + *
  • Write Path: Business logic → MemoryImprint → {@code toData()} → MemoryImprintData → Hazelcast IMap
    • + *
  • Read Path: Hazelcast IMap → MemoryImprintData → {@code fromData()} → MemoryImprint → Business logic
    • + *
+ *

+ * The {@link MemoryImprintData} DTO carries the event as a live object; turning it into a JSON + * string on the wire (with polymorphic type preservation) is handled by + * {@link MemoryImprintDataSerializer} — the only Hazelcast-specific serialization boundary. This + * keeps both the API type and the DTO unaware of storage concerns. + * + * @see HazelcastCoordinationProvider + * @see MemoryImprintData + * @see MemoryImprintDataSerializer + * @see PolymorphicEventTypeAdapter + */ +public class HazelcastBuildMemoryStorage extends BuildMemoryStorage { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastBuildMemoryStorage.class); + + /** + * Hazelcast map name for distributed build memory. + */ + private static final String MAP_NAME = "gerrit-trigger-build-memory"; + + /** + * Hazelcast map name for cross-replica build abort requests. + *

+ * When a replica needs to abort a build running on another replica, it puts an entry + * {@code "jobFullName:buildNumber"} into this map. Each Jenkins replica has a listener + * on this map and will abort any matching build running on its local executors. + *

+ * This approach works in Hazelcast CLIENT mode where {@code executeOnMember()} would + * target Hazelcast server sidecars (not Jenkins replicas) and fail with + * {@code ClassNotFoundException} because Jenkins classes are absent from the sidecar JVM. + */ + private static final String ABORT_INBOX_MAP_NAME = "gerrit-trigger-abort-inbox"; + + /** + * TTL for abort inbox entries in seconds (1 minute). + * Entries expire automatically to prevent memory leaks in case the target build + * has already finished or is not found on any replica. + */ + private static final long ABORT_INBOX_TTL_SECONDS = 60; + + /** + * Abort inbox value indicating the build was superseded by a new patchset. + * The receiving replica will use {@link NewPatchSetInterruption} as the cause. + */ + private static final String CAUSE_NEW_PATCHSET = "NEW_PATCHSET"; + + /** + * Abort inbox value indicating the patchset was abandoned. + * The receiving replica will use {@link AbandonedPatchsetInterruption} as the cause. + */ + private static final String CAUSE_ABANDONED = "ABANDONED"; + + /** + * Delay in seconds before writing a deferred cross-replica abort inbox entry. + *

+ * When {@code started()} detects that an entry is already marked {@code isCancelling=true} + * (race condition: build started after the abort decision was made), the abort inbox entry + * is written after this delay so the CPS engine has had time to attach a + * FlowExecution before the interrupt arrives. + */ + private static final long DEFERRED_ABORT_DELAY_SECONDS = 3L; + + /** + * Grace period in seconds before treating an ambiguous queue-item cancellation as final. + *

+ * {@link HazelcastQueueCancellationStrategy#isLoadBalancedCancellation} cannot reliably tell + * a genuine Gerrit-triggered cancellation apart from a benign cross-replica queue-item + * relocation - Jenkins preserves no marker on {@code LeftItem} either way (see that method's + * own doc comment). Rather than finalize immediately whenever {@code isCancelling} happens to + * already be set, {@link #cancelled} waits this long for a possible {@link #started} call to + * arrive from whichever replica the item actually landed on before concluding the build was + * genuinely, permanently cancelled. + *

+ * This narrows the race rather than closing it: observed real-world delay between a + * cancellation decision and the relocated build's own started() call has been as long as ~6s + * in production logs, so this is set comfortably above that - but a sufficiently slow + * relocation can still, in principle, outlast this window. + */ + private static final long CANCEL_FINALIZE_GRACE_SECONDS = 10L; + + /** + * Poll interval in milliseconds used by {@link #handleAbortRequest} while waiting for a + * Pipeline build's CPS execution to start before delivering the interrupt. + */ + private static final long ABORT_RETRY_POLL_MS = 250L; + + /** + * Maximum number of poll attempts in {@link #handleAbortRequest} before interrupting + * regardless of CPS execution state. + *

+ * Total maximum wait = {@link #ABORT_RETRY_POLL_MS} * {@code ABORT_MAX_RETRIES} + * = 250 ms * 12 = 3 seconds. + */ + private static final int ABORT_MAX_RETRIES = 12; + + /** + * Maximum time in seconds to wait when acquiring a distributed lock. + *

+ * Using {@link IMap#tryLock(Object, long, TimeUnit)} instead of {@link IMap#lock(Object)} + * prevents threads from blocking indefinitely when a lock is stuck (e.g. after an + * interrupted lock acquisition that left the Hazelcast server holding the lock). + */ + private static final int LOCK_ACQUIRE_TIMEOUT_SECONDS = 10; + + /** + * The Hazelcast instance to use for distributed storage. + */ + private final HazelcastInstance hazelcastInstance; + + /** + * Distributed mode storage (coordination mode). + * Lazy-initialized when first accessed. + * Marked volatile for thread-safe double-checked locking pattern. + * Uses String keys (event IDs) instead of BuildMemoryKey to avoid Hazelcast classloader + * issues when deserializing plugin-specific key classes via Java serialization. + */ + private transient volatile IMap distributedMemory = null; + + /** + * Constructor. + * + * @param hazelcastInstance the Hazelcast instance to use + */ + public HazelcastBuildMemoryStorage(@NonNull HazelcastInstance hazelcastInstance) { + this.hazelcastInstance = hazelcastInstance; + registerAbortInboxListener(); + } + + /** + * Registers a listener on the abort inbox IMap so this replica can receive and process + * cross-replica build abort requests submitted by other Jenkins replicas. + *

+ * The listener fires in this JVM (which has Jenkins on its classpath), so it can safely + * look up jobs and interrupt executors. This is the correct approach for Hazelcast CLIENT + * mode where {@code executeOnMember()} would instead run code on the Hazelcast sidecar. + */ + private void registerAbortInboxListener() { + if (hazelcastInstance == null) { + return; + } + IMap abortInbox = hazelcastInstance.getMap(ABORT_INBOX_MAP_NAME); + abortInbox.addEntryListener((EntryAddedListener)event -> { + String abortKey = event.getKey(); + int lastColon = abortKey.lastIndexOf(':'); + if (lastColon < 0) { + logger.warn("Abort-inbox: invalid key (no colon separator): {}", abortKey); + return; + } + String jobName = abortKey.substring(0, lastColon); + String buildId = abortKey.substring(lastColon + 1); + String causeType = event.getValue(); + handleAbortRequest(jobName, buildId, causeType); + }, true); // includeValue=true: the cause type string is needed by handleAbortRequest + logger.debug("Registered abort-inbox listener on map: {}", ABORT_INBOX_MAP_NAME); + } + + /** + * Handles an abort request received via the abort inbox IMap. + * Looks up the build on this replica and interrupts it if still running. + *

+ * The {@code causeType} string determines which {@link CauseOfInterruption} subclass + * is used so that the aborted build is annotated with the correct reason: + *

    + *
  • {@value #CAUSE_ABANDONED} → {@link AbandonedPatchsetInterruption}
    • + *
  • {@value #CAUSE_NEW_PATCHSET} (or any other value) → {@link NewPatchSetInterruption}
    • + *
+ * + * @param jobName full name of the job + * @param buildId build number as string + * @param causeType cause type string ({@value #CAUSE_ABANDONED} or {@value #CAUSE_NEW_PATCHSET}) + */ + private static void handleAbortRequest(String jobName, String buildId, String causeType) { + handleAbortRequest(jobName, buildId, causeType, ABORT_MAX_RETRIES); + } + + /** + * @param jobName full name of the Jenkins job + * @param buildId build number as a string + * @param causeType abort cause type ({@link #CAUSE_NEW_PATCHSET} or {@link #CAUSE_ABANDONED}) + * @param retriesLeft remaining poll attempts before interrupting regardless of CPS state; + * decremented on each reschedule until it reaches zero + */ + private static void handleAbortRequest(String jobName, String buildId, + String causeType, int retriesLeft) { + try (ACLContext ignored = ACL.as(ACL.SYSTEM)) { + Jenkins jenkins = Jenkins.getInstanceOrNull(); + if (jenkins == null) { + return; + } + Job job = jenkins.getItemByFullName(jobName, Job.class); + if (job == null) { + return; + } + Run build = job.getBuildByNumber(Integer.parseInt(buildId)); + if (build == null || !build.isBuilding()) { + return; + } + + // For Pipeline builds, wait until the CPS execution has started (i.e. + // FlowExecution is attached to its FlowExecutionOwner) before delivering the + // interrupt. Interrupting during CPS initialisation has no effect — the interrupt + // flag is set before FlowExecution exists, so it is silently lost. See + // PipelineAbortHelper for why waiting for FlowExecution alone is sufficient. + // We poll every ABORT_RETRY_POLL_MS for up to ABORT_MAX_RETRIES attempts + // (ABORT_RETRY_POLL_MS * ABORT_MAX_RETRIES = 3 s total maximum wait). + boolean notYetStarted; + try { + notYetStarted = PipelineAbortHelper.isPipelineNotYetStarted(build); + } catch (NoClassDefFoundError e) { + // workflow-api not installed — not a pipeline, safe to interrupt now + notYetStarted = false; + } + if (notYetStarted) { + if (retriesLeft > 0) { + logger.debug("Abort-inbox: build={}/{} CPS not yet started, retrying in {}ms ({} attempts left)", + jobName, buildId, ABORT_RETRY_POLL_MS, retriesLeft); + Timer.get().schedule( + () -> handleAbortRequest(jobName, buildId, causeType, retriesLeft - 1), + ABORT_RETRY_POLL_MS, TimeUnit.MILLISECONDS); + return; + } + logger.info("Abort-inbox: build={}/{} CPS still not started after {} attempts, interrupting anyway", + jobName, buildId, ABORT_MAX_RETRIES); + } + + CauseOfInterruption cause; + if (CAUSE_ABANDONED.equals(causeType)) { + cause = new AbandonedPatchsetInterruption(); + } else { + cause = new NewPatchSetInterruption(); + } + // Iterate all executors to find the one currently running this build. + // Using build.getExecutor() is unreliable for Pipeline jobs: the flyweight + // executor (controller-side CPS orchestrator) may be parked/suspended while + // the actual work runs on an agent executor. Iterating computers mirrors the + // same approach used in BuildMemory.cancelMatchingJobs() for local cancellation. + boolean interrupted = false; + for (Computer c : jenkins.getComputers()) { + for (Executor e : c.getAllExecutors()) { + if (build.equals(e.getCurrentExecutable())) { + e.interrupt(Result.ABORTED, cause); + interrupted = true; + } + } + } + if (interrupted) { + logger.info("Abort-inbox: interrupted job={} build={} cause={}", jobName, buildId, causeType); + } else { + logger.debug("Abort-inbox: no executor found for job={} build={} (may not be running locally)", + jobName, buildId); + } + } catch (Exception e) { + logger.error("Abort-inbox: failed to abort job={} build={}", jobName, buildId, e); + } + } + + /** + * Gets or initializes the distributed memory map using thread-safe double-checked locking. + *

+ * Uses volatile field and synchronized block to ensure only one thread initializes + * the map while avoiding synchronization overhead on subsequent accesses. + * + * @return distributed memory map, or null if Hazelcast unavailable + */ + private IMap getDistributedMemory() { + // First check (no locking) - fast path for already-initialized case + if (distributedMemory == null) { + synchronized (this) { + // Second check (with locking) - ensures only one thread initializes + if (distributedMemory == null) { + if (hazelcastInstance != null) { + distributedMemory = hazelcastInstance.getMap(MAP_NAME); + logger.debug("Initialized distributed BuildMemory map: {} (size: {})", + MAP_NAME, distributedMemory.size()); + } else { + logger.warn("Hazelcast unavailable, distributed memory not available"); + } + } + } + } + return distributedMemory; + } + + /** + * Attempts to acquire a distributed lock with a bounded timeout. + *

+ * Unlike {@link IMap#lock(Object)}, this method will not block indefinitely. + * If the lock cannot be acquired within {@link #LOCK_ACQUIRE_TIMEOUT_SECONDS} seconds + * (e.g. a previous holder was interrupted mid-operation and left the lock unreleased), + * this method returns {@code false} so the caller can skip the operation rather than + * deadlocking a Gerrit event worker thread. + *

+ * If interrupted while waiting, the thread's interrupt status is restored before returning. + * + * @param map the distributed map that owns the lock + * @param key the key to lock + * @return {@code true} if the lock was acquired, {@code false} otherwise + */ + private static boolean tryLockWithTimeout(IMap map, String key) { + try { + return map.tryLock(key, LOCK_ACQUIRE_TIMEOUT_SECONDS, TimeUnit.SECONDS); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + logger.warn("Interrupted while waiting for distributed lock on key: {}", key); + return false; + } + } + + /** + * Code to run while a distributed lock is held, passed to {@link #withLock}. + */ + @FunctionalInterface + private interface LockedAction { + void run(); + } + + /** + * Outcome of a {@link #withLock} call, letting the caller react if the lock was never + * acquired (in which case the {@link LockedAction} did not run at all). + */ + private static final class LockOutcome { + private final boolean acquired; + + private LockOutcome(boolean acquired) { + this.acquired = acquired; + } + + /** + * Runs {@code action} only if the lock could not be acquired. + * + * @param action ran when the lock was not acquired within {@link #LOCK_ACQUIRE_TIMEOUT_SECONDS} + */ + void onFailure(Runnable action) { + if (!acquired) { + action.run(); + } + } + } + + /** + * Runs {@code action} while holding the distributed lock on {@code key}, always releasing + * the lock afterwards. If the lock cannot be acquired within + * {@link #LOCK_ACQUIRE_TIMEOUT_SECONDS}, {@code action} is not run at all; use the returned + * {@link LockOutcome#onFailure} to react to that case (e.g. logging and skipping the + * operation). + *

+ * {@code action} is responsible for catching and logging its own exceptions - this method + * only manages lock acquisition and release, not business-logic error handling. + * + * @param map the distributed map that owns the lock + * @param key the key to lock + * @param action the code to run while the lock is held + * @return a {@link LockOutcome} - chain {@link LockOutcome#onFailure} to react to lock failure + */ + private LockOutcome withLock(IMap map, String key, LockedAction action) { + if (!tryLockWithTimeout(map, key)) { + return new LockOutcome(false); + } + try { + action.run(); + } finally { + map.unlock(key); + } + return new LockOutcome(true); + } + + // ===== Implement BuildMemoryStorage abstract methods ===== + + @Override + @CheckForNull + public synchronized MemoryImprint getMemoryImprint(@NonNull GerritTriggeredEvent event) { + IMap map = getDistributedMemory(); + if (map == null) { + return null; + } + + String key = EventIdGenerator.generateEventId(event); + MemoryImprintData data = map.get(key); + if (data != null) { + return MemoryImprint.fromData(data); + } + return null; + } + + @Override + public synchronized void triggered(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot record triggered - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = project.getFullName(); + + // ATOMIC OPERATION - Distributed lock ensures only one replica modifies this entry at a time. + // This is critical when multiple projects are triggered by the same event simultaneously. + // Without atomic operations, concurrent threads can overwrite each other's entries, + // causing some project entries to be lost from BuildMemory (which breaks cancellation logic). + // Note: EntryProcessor is NOT used here because in client mode the processor class would need + // to exist on the Hazelcast sidecar member's classpath, causing ClassNotFoundException. + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null) { + data = new MemoryImprintData(); + data.setEvent(event); + } + boolean found = false; + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + found = true; + break; + } + } + } + if (!found) { + EntryData newEntry = new EntryData(); + newEntry.setProjectFullName(projectFullName); + data.addEntry(newEntry); + } + map.put(key, data); + if (!found) { + logger.trace("Triggered event stored in distributed memory: {} for project: {}", + key, projectFullName); + } else { + logger.trace("Project {} already triggered for event: {}", projectFullName, key); + } + } catch (Exception e) { + logger.error("Failed to store triggered event in distributed memory for project: {} event: {}", + projectFullName, key, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping triggered()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + public synchronized void started(@NonNull GerritTriggeredEvent event, @NonNull Run build) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot mark started - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = build.getParent().getFullName(); + String buildId = build.getId(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + long startedTimestamp = System.currentTimeMillis(); + // Track whether a deferred cross-replica abort needs to be sent after the lock is released. + // This handles the race condition where requestCrossReplicaAbort() was called before this + // build's buildId was written to Hazelcast (buildId was null at that time, so no abort + // inbox entry was written). We detect this by checking isCancelling on the entry after + // writing the buildId, and then write the abort inbox entry here. + AtomicBoolean pendingCrossReplicaAbort = new AtomicBoolean(false); + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null) { + data = new MemoryImprintData(); + } + boolean found = false; + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + entryData.setBuildId(buildId); + entryData.setStartedTimestamp(startedTimestamp); + // The build actually started on a replica — clear any queueLeft flag that + // was set when the queue item was moved by a load balancer. The entry is + // now actively building and must be visible to cancelOutdatedBuilds again. + entryData.setQueueLeft(false); + found = true; + // If this entry is already marked for cancellation, we need to trigger + // cross-replica abort now that buildId is known. + if (entryData.isCancelling()) { + pendingCrossReplicaAbort.set(true); + } + break; + } + } + } + if (!found) { + EntryData newEntry = new EntryData(); + newEntry.setProjectFullName(projectFullName); + newEntry.setBuildId(buildId); + newEntry.setStartedTimestamp(startedTimestamp); + data.setEvent(event); + data.addEntry(newEntry); + } + map.put(key, data); + if (!found) { + logger.warn("Build started without being registered first (distributed mode)."); + } + logger.trace("Build started event stored in distributed memory: {}", key); + } catch (Exception e) { + logger.error("Failed to mark build started in distributed memory: project={}, build={}, event={}", + projectFullName, buildId, key, e); + } + }).onFailure(() -> logger.error("Could not acquire distributed lock for key {} within {}s - skipping started()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + + // Deferred cross-replica abort: if the entry was already marked for cancellation when this + // build started, requestCrossReplicaAbort() previously found buildId=null and could not + // write the abort inbox entry. Now that buildId is known, schedule the abort signal with + // a short delay so the CPS pipeline has time to complete initialization before the + // interrupt is delivered. Firing the interrupt too early (during CPS init, before any + // step begins) has no effect — the interrupt flag is set on the wrong thread context. + if (pendingCrossReplicaAbort.get() && hazelcastInstance != null) { + final HazelcastInstance hz = hazelcastInstance; + final String abortKey = projectFullName + ":" + buildId; + logger.info("Scheduling deferred cross-replica abort in {}s (started race): job={} build={}", + DEFERRED_ABORT_DELAY_SECONDS, projectFullName, buildId); + Timer.get().schedule(() -> { + try { + IMap abortInbox = hz.getMap(ABORT_INBOX_MAP_NAME); + abortInbox.put(abortKey, CAUSE_NEW_PATCHSET, ABORT_INBOX_TTL_SECONDS, TimeUnit.SECONDS); + logger.info("Queued deferred cross-replica abort (started race): job={} build={} cause={}", + projectFullName, buildId, CAUSE_NEW_PATCHSET); + } catch (Exception ex) { + logger.warn("Failed to write deferred abort inbox entry: key={}", abortKey, ex); + } + }, DEFERRED_ABORT_DELAY_SECONDS, TimeUnit.SECONDS); + } + } + + @Override + public synchronized void completed(@NonNull GerritTriggeredEvent event, @NonNull Run build) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot mark completed - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = build.getParent().getFullName(); + String buildId = build.getId(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + long completedTimestamp = System.currentTimeMillis(); + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null) { + data = new MemoryImprintData(); + } + boolean found = false; + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + if (entryData.getBuildId() == null) { + entryData.setBuildId(buildId); + } + entryData.setCompletedTimestamp(completedTimestamp); + entryData.setBuildCompleted(true); + found = true; + break; + } + } + } + if (!found) { + EntryData newEntry = new EntryData(); + newEntry.setProjectFullName(projectFullName); + newEntry.setBuildId(buildId); + newEntry.setCompletedTimestamp(completedTimestamp); + newEntry.setBuildCompleted(true); + data.setEvent(event); + data.addEntry(newEntry); + } + map.put(key, data); + if (!found) { + logger.debug("Build completed without being registered first (distributed mode)."); + } + logger.trace("Build completed event stored in distributed memory: {}", key); + } catch (Exception e) { + logger.error("Failed to mark build completed in distributed memory: project={}, build={}, event={}", + projectFullName, buildId, key, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping completed()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + public synchronized void retriggered(@NonNull GerritTriggeredEvent event, @NonNull Job project, + @CheckForNull List otherBuilds) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot record retriggered - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = project.getFullName(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null) { + data = new MemoryImprintData(); + data.setEvent(event); + if (otherBuilds != null) { + for (Run otherBuild : otherBuilds) { + EntryData entryData = new EntryData(); + entryData.setProjectFullName(otherBuild.getParent().getFullName()); + entryData.setBuildId(otherBuild.getId()); + entryData.setBuildCompleted(!otherBuild.isBuilding()); + data.addEntry(entryData); + } + } + } + boolean found = false; + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + entryData.setBuildId(null); + entryData.setBuildCompleted(false); + entryData.setStartedTimestamp(null); + entryData.setCompletedTimestamp(null); + found = true; + break; + } + } + } + if (!found) { + EntryData newEntry = new EntryData(); + newEntry.setProjectFullName(projectFullName); + data.addEntry(newEntry); + } + map.put(key, data); + logger.trace("Retriggered event stored in distributed memory: {}", key); + } catch (Exception e) { + logger.error("Failed to record retriggered in distributed memory: project={}, event={}", + projectFullName, key, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping retriggered()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + public synchronized void cancelled(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot mark cancelled - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = project.getFullName(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + // Set when this queue exit is ambiguous (isCancelling was already true) and needs a + // deferred re-check after the lock is released - see CANCEL_FINALIZE_GRACE_SECONDS. + AtomicBoolean scheduleFinalizeCheck = new AtomicBoolean(false); + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null) { + data = new MemoryImprintData(); + } + boolean found = false; + boolean modified = false; + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + found = true; + if (!entryData.isBuildCompleted()) { + if (entryData.isCancelling()) { + // Ambiguous: the intent was set via setCancelling() first (e.g. by + // cancelOutdatedBuilds), but this queue exit itself could equally be + // a genuine Gerrit-triggered cancellation OR a benign cross-replica + // queue-item relocation - isLoadBalancedCancellation() can't tell + // them apart (see its own doc comment; Jenkins preserves no marker + // on LeftItem either way). Don't finalize yet: mark queueLeft (not + // completed) and leave isCancelling=true so started()'s own + // deferred-abort compensator still fires correctly if this really is + // a relocation and the build reports in from wherever it landed. + // scheduleDeferredCancelFinalize (below, after the lock is released) + // decides the real outcome after a grace period. + entryData.setQueueLeft(true); + scheduleFinalizeCheck.set(true); + } else if (entryData.getBuildId() == null) { + // No prior cancellation intent AND build has not started anywhere. + // This is a potential load-balanced queue move (build will restart + // on another instance) or a direct Queue.doCancelItem before start. + // Mark queueLeft=true but do NOT set buildCompleted=true so the + // IMap entry is preserved for cross-instance PS2-aborts-PS1 scenarios. + // NOTE: if buildId IS already set, started() already ran on another + // instance — leave the entry completely untouched so it stays visible + // to cancelOutdatedBuilds on that instance. + entryData.setQueueLeft(true); + logger.info("Marking queueLeft (pre-cancellation-intent queue exit, no " + + "buildId yet) for project={} event={} - possible load-balanced " + + "relocation with no started() call yet", projectFullName, key); + } else { + logger.debug("cancelled() called after started() for project={} event={}: " + + "build already running (buildId={}), ignoring late onLeft.", + projectFullName, key, entryData.getBuildId()); + } + modified = true; + } else { + logger.debug("Skipping cancelled() for project={} event={}: " + + "already completed, buildId={}.", + projectFullName, key, entryData.getBuildId()); + } + break; + } + } + } + if (!found) { + logger.debug("cancelled() called for untracked project={} event={}: skipping.", + projectFullName, key); + } + if (modified) { + map.put(key, data); + } + logger.trace("Cancelled event stored in distributed memory: {}", key); + } catch (Exception e) { + logger.error("Failed to mark cancelled in distributed memory: project={}, event={}", + projectFullName, key, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping cancelled()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + + if (scheduleFinalizeCheck.get()) { + scheduleDeferredCancelFinalize(event, project, key, projectFullName); + } + } + + /** + * Re-checks a provisionally-cancelled entry after {@link #CANCEL_FINALIZE_GRACE_SECONDS} and + * finalizes it as a genuine cancellation only if no {@link #started} call arrived for it in + * the meantime. + *

+ * Exists because {@link HazelcastQueueCancellationStrategy#isLoadBalancedCancellation} cannot + * distinguish a genuine Gerrit-triggered cancellation from a benign cross-replica queue-item + * relocation - without this grace period, a relocated-but-still-alive build was getting marked + * completed/forgotten (triggering premature Gerrit "No Builds Executed" feedback) before it + * had a chance to call {@code started()} on whichever replica it landed on, silently skipping + * the deferred cross-replica abort this class already schedules for the reverse race (see + * {@link #started}). + *

+ * Deliberately narrows the race rather than closing it - see {@link #CANCEL_FINALIZE_GRACE_SECONDS}. + * + * @param event the event whose entry may need finalizing + * @param project the project/job the entry is for + * @param key the distributed map key for event + * @param projectFullName project's full name + */ + private void scheduleDeferredCancelFinalize( + @NonNull GerritTriggeredEvent event, @NonNull Job project, String key, String projectFullName) { + IMap map = getDistributedMemory(); + if (map == null) { + return; + } + Timer.get().schedule(() -> { + AtomicBoolean finalized = new AtomicBoolean(false); + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null || data.getEntries() == null) { + return; + } + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + if (entryData.isCancelling() && !entryData.isBuildCompleted() + && entryData.getBuildId() == null) { + logger.info("Finalizing deferred cancellation for project={} event={}: " + + "no started() call arrived within {}s grace period - genuine cancel.", + projectFullName, key, CANCEL_FINALIZE_GRACE_SECONDS); + entryData.setCancelled(true); + entryData.setCancelling(false); + entryData.setCompletedTimestamp(System.currentTimeMillis()); + entryData.setBuildCompleted(true); + map.put(key, data); + finalized.set(true); + } else { + logger.debug("Deferred cancel-finalize check for project={} event={}: " + + "buildId={} isCancelling={} isBuildCompleted={} - a build " + + "reported in elsewhere or already completed; treating as a " + + "relocation, not a genuine cancellation.", + projectFullName, key, entryData.getBuildId(), entryData.isCancelling(), + entryData.isBuildCompleted()); + } + break; + } + } + } catch (Exception e) { + logger.error("Failed deferred cancel-finalize check: project={}, event={}", + projectFullName, key, e); + } + }).onFailure(() -> logger.error("Could not acquire distributed lock for key {} within {}s - skipping " + + "deferred cancel-finalize check", key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + if (finalized.get()) { + // The original GerritQueueListener#onLeft call that led here checked + // isAllBuildsCompleted() synchronously, before this deferred finalize ran, so it + // saw "not yet complete" and skipped the Gerrit-feedback/forget step. Re-invoke it + // now that the entry is actually finalized, exactly as it would have run + // synchronously had this been unambiguous from the start. + ToGerritRunListener runListener = ToGerritRunListener.getInstance(); + if (runListener != null) { + runListener.allBuildsCompleted(event, new GerritCause(event, false), TaskListener.NULL); + } + } + }, CANCEL_FINALIZE_GRACE_SECONDS, TimeUnit.SECONDS); + } + + @Override + public synchronized void setCancelling(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot mark cancelling - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = project.getFullName(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data != null && data.getEntries() != null) { + boolean updated = false; + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + // Not gated on !isQueueLeft(): an ambiguous queue exit (possible + // load-balanced relocation) is not proof this entry is done, and must + // remain eligible to be marked cancelling so a relocated-then-started + // build still gets cross-replica-aborted (see BuildMemory's identical + // reasoning in cancelOutdatedEvents()). + if (!entryData.isBuildCompleted() && !entryData.isCancelling() + && !entryData.isCancelled()) { + entryData.setCancelling(true); + updated = true; + } + } + } + if (updated) { + map.put(key, data); + } + } + logger.trace("Cancelling flag set in distributed memory for event: {}", key); + } catch (Exception e) { + logger.error("Failed to set cancelling flag in distributed memory: project={}, event={}", + projectFullName, key, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping setCancelling()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + public synchronized void requestCrossReplicaAbort(@NonNull GerritTriggeredEvent event, @NonNull Job project, + @NonNull CauseOfInterruption cause) { + if (hazelcastInstance == null) { + return; + } + + IMap map = getDistributedMemory(); + if (map == null) { + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = project.getFullName(); + String causeType; + if (cause instanceof AbandonedPatchsetInterruption) { + causeType = CAUSE_ABANDONED; + } else { + causeType = CAUSE_NEW_PATCHSET; + } + + // Collect build IDs that were marked as cancelling for this project/event. + // setCancelling() has already set isCancelling=true in the distributed map. + List buildIdsToAbort = new ArrayList<>(); + MemoryImprintData data = map.get(key); + if (data != null && data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName()) + && entryData.isCancelling() + && entryData.getBuildId() != null) { + buildIdsToAbort.add(entryData.getBuildId()); + } + } + } + + // Put abort requests into the distributed abort inbox for cross-replica cancellation. + // Each Jenkins replica has a listener on this map (registered in the constructor) and will + // abort any matching build running on its local executors. + // NOTE: executeOnMember() cannot be used in Hazelcast CLIENT mode because it runs the task + // on the Hazelcast sidecar JVM, which does not have Jenkins classes on its classpath. + if (!buildIdsToAbort.isEmpty()) { + IMap abortInbox = hazelcastInstance.getMap(ABORT_INBOX_MAP_NAME); + for (String buildId : buildIdsToAbort) { + String abortKey = projectFullName + ":" + buildId; + abortInbox.put(abortKey, causeType, ABORT_INBOX_TTL_SECONDS, TimeUnit.SECONDS); + logger.info("Queued cross-replica abort: job={} build={} cause={}", projectFullName, buildId, causeType); + } + } + } + + @Override + public synchronized void forget(@NonNull GerritTriggeredEvent event) { + IMap map = getDistributedMemory(); + if (map == null) { + return; + } + + String key = EventIdGenerator.generateEventId(event); + map.remove(key); + logger.trace("Forgot event from distributed memory: {}", key); + } + + @Override + public synchronized void removeProject(@NonNull Job project) { + String projectFullName = project.getFullName(); + + IMap map = getDistributedMemory(); + if (map == null) { + return; + } + + // ATOMIC OPERATION - Distributed lock per key. EntryProcessor not used (ClassNotFoundException in client mode). + // Collect keys first to avoid ConcurrentModificationException + java.util.Set keys = new java.util.HashSet<>(map.keySet()); + + for (String key : keys) { + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null || data.getEntries() == null) { + return; + } + boolean removed = data.getEntries().removeIf( + entryData -> projectFullName.equals(entryData.getProjectFullName())); + if (removed) { + if (data.getEntries().isEmpty()) { + map.delete(key); + logger.trace("Removed empty entry for project {} from distributed memory: {}", + projectFullName, key); + } else { + map.put(key, data); + logger.trace("Removed project {} from distributed memory entry: {}", + projectFullName, key); + } + } + } catch (Exception e) { + logger.error("Failed to remove project from distributed memory entry: project={}, key={}", + projectFullName, key, e); + // Continue processing other keys + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping removeProject() entry", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + } + + @Override + public synchronized boolean isAllBuildsCompleted(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + return imprint != null && imprint.isAllBuildsCompleted(); + } + + @Override + public synchronized boolean isAllBuildsStarted(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + return imprint != null && imprint.isAllBuildsSet(); + } + + @Override + @CheckForNull + public synchronized BuildsStartedStats getBuildsStartedStats(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint != null) { + return imprint.getBuildsStartedStats(); + } + return null; + } + + @Override + @CheckForNull + public synchronized String getStatusReport(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint != null) { + return imprint.getStatusReport(); + } + return null; + } + + @Override + public synchronized boolean isTriggered(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint == null) { + return false; + } + String fullName = project.getFullName(); + for (MemoryImprint.Entry entry : imprint.getEntries()) { + if (entry.isProject(fullName)) { + return true; + } + } + return false; + } + + @Override + public synchronized boolean isBuilding(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint == null) { + return false; + } + String fullName = project.getFullName(); + for (MemoryImprint.Entry entry : imprint.getEntries()) { + if (entry.isProject(fullName)) { + if (entry.getBuild() != null) { + return !entry.isBuildCompleted(); + } else { + return !entry.isCancelling() && !entry.isCancelled() && !entry.isQueueLeft(); + } + } + } + return false; + } + + @Override + public synchronized boolean isBuilding(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint == null) { + return false; + } + // An event is still "building" if at least one entry is not yet in a terminal state. + // queueLeft entries (load-balanced moves or direct doCancelItem) are treated as + // inactive — they are not running on this replica. This allows isBuilding(event) + // to return false for the unit-test case (direct doCancelItem) while preserving + // the IMap key for cross-replica PS2-aborts-PS1 scenarios. + for (MemoryImprint.Entry entry : imprint.getEntries()) { + if (!entry.isBuildCompleted() && !entry.isQueueLeft()) { + return true; + } + } + return false; + } + + @Override + @CheckForNull + public synchronized List getBuilds(@NonNull GerritTriggeredEvent event) { + MemoryImprint imprint = getMemoryImprint(event); + if (imprint != null) { + List list = new LinkedList<>(); + for (MemoryImprint.Entry entry : imprint.getEntries()) { + if (entry.getBuild() != null) { + list.add(entry.getBuild()); + } + } + return list; + } + return null; + } + + @Override + public void setEntryCustomUrl(@NonNull GerritTriggeredEvent event, @NonNull Run r, + @CheckForNull String customUrl) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot set custom URL - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = r.getParent().getFullName(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null || data.getEntries() == null) { + logger.warn("Could not set custom URL - event not found: {}", event); + return; + } + boolean found = false; + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + entryData.setCustomUrl(customUrl); + found = true; + break; + } + } + if (found) { + map.put(key, data); + logger.trace("Recording custom URL for {}: {}", event, customUrl); + } else { + logger.warn("Could not set custom URL - event not found: {}", event); + } + } catch (Exception e) { + logger.error("Failed to set custom URL in distributed memory: project={}, event={}, url={}", + projectFullName, key, customUrl, e); + } + }).onFailure(() -> logger.error( + "Could not acquire distributed lock for key {} within {}s - skipping setEntryCustomUrl()", + key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + public void setEntryUnsuccessfulMessage(@NonNull GerritTriggeredEvent event, @NonNull Run r, + @CheckForNull String unsuccessfulMessage) { + IMap map = getDistributedMemory(); + if (map == null) { + logger.warn("Cannot set unsuccessful message - Hazelcast unavailable"); + return; + } + + String key = EventIdGenerator.generateEventId(event); + String projectFullName = r.getParent().getFullName(); + + // ATOMIC OPERATION - Distributed lock. EntryProcessor not used (ClassNotFoundException in client mode). + withLock(map, key, () -> { + try { + MemoryImprintData data = map.get(key); + if (data == null || data.getEntries() == null) { + logger.warn("Could not set unsuccessful message - event not found: {}", event); + return; + } + boolean found = false; + for (EntryData entryData : data.getEntries()) { + if (projectFullName.equals(entryData.getProjectFullName())) { + entryData.setUnsuccessfulMessage(unsuccessfulMessage); + found = true; + break; + } + } + if (found) { + map.put(key, data); + logger.trace("Recording unsuccessful message for {}: {}", event, unsuccessfulMessage); + } else { + logger.warn("Could not set unsuccessful message - event not found: {}", event); + } + } catch (Exception e) { + logger.error("Failed to set unsuccessful message in distributed memory:" + + " project={}, event={}, message={}", + projectFullName, key, unsuccessfulMessage, e); + } + }).onFailure(() -> logger.error("Could not acquire distributed lock for key {} within {}s" + + " - skipping setEntryUnsuccessfulMessage()", key, LOCK_ACQUIRE_TIMEOUT_SECONDS)); + } + + @Override + @NonNull + public synchronized BuildMemoryReport report() { + BuildMemoryReport report = new BuildMemoryReport(); + + IMap map = getDistributedMemory(); + if (map == null) { + return report; + } + + // Read all entries from distributed memory + for (Map.Entry mapEntry : map.entrySet()) { + MemoryImprintData data = mapEntry.getValue(); + GerritTriggeredEvent event = data.getEvent(); + + if (event != null) { + MemoryImprint imprint = MemoryImprint.fromData(data); + List triggered = new LinkedList(); + for (MemoryImprint.Entry tr : imprint.getEntries()) { + triggered.add(tr.clone()); + } + report.put(event, triggered); + } + } + return report; + } + + @Override + @NonNull + public synchronized Map getAllEvents() { + Map result = new HashMap<>(); + + IMap map = getDistributedMemory(); + if (map == null) { + return result; + } + + // Convert all entries + for (Map.Entry entry : map.entrySet()) { + MemoryImprintData data = entry.getValue(); + if (data != null) { + GerritTriggeredEvent event = data.getEvent(); + if (event != null) { + MemoryImprint imprint = MemoryImprint.fromData(data); + result.put(event, imprint); + } + } + } + + logger.trace("Returning {} events from distributed memory", result.size()); + return result; + } + + @Override + public boolean eventsMatch(@NonNull GerritTriggeredEvent event1, @NonNull GerritTriggeredEvent event2) { + // In distributed mode, use logical comparison via EventIdGenerator + // because events may be deserialized from Hazelcast, creating new object instances + String id1 = EventIdGenerator.generateEventId(event1); + String id2 = EventIdGenerator.generateEventId(event2); + return id1.equals(id2); + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastConfig.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastConfig.java new file mode 100644 index 000000000..c0c979490 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastConfig.java @@ -0,0 +1,150 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.client.config.ClientConfig; +import jenkins.model.Jenkins; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Configuration builder for Hazelcast cluster. + * Creates appropriate configuration based on deployment environment (Kubernetes, TCP/IP, etc.). + * + */ +public final class HazelcastConfig { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastConfig.class); + + /** + * Default cluster name for Gerrit Trigger plugin Hazelcast cluster. + */ + public static final String DEFAULT_CLUSTER_NAME = "gerrit-trigger-cluster"; + + /** + * System property to specify addresses for Hazelcast client (comma-separated host:port). + * Default: "localhost:5702" — assumes a Hazelcast sidecar listening on port 5702 in the same pod. + * Example: "localhost:5702" + */ + public static final String CLIENT_ADDRESSES_PROPERTY = + "gerrit.trigger.coordination.hazelcast.client.addresses"; + + /** + * System property to specify the cluster name to connect to. + * Must match the cluster name of the target Hazelcast cluster. + * Default: {@link #DEFAULT_CLUSTER_NAME}. + */ + public static final String CLIENT_CLUSTER_NAME_PROPERTY = + "gerrit.trigger.coordination.hazelcast.client.cluster.name"; + + /** + * Default address for Hazelcast client: local sidecar on port 5702. + */ + public static final String DEFAULT_CLIENT_ADDRESS = "localhost:5702"; + + /** + * Private constructor to prevent instantiation. + */ + private HazelcastConfig() { + // Utility class + } + + /** + * Creates a Hazelcast client configuration to connect to an existing cluster. + *

+ * The client connects to the addresses specified by {@link #CLIENT_ADDRESSES_PROPERTY} + * (default: {@link #DEFAULT_CLIENT_ADDRESS}) and joins the cluster whose name matches + * {@link #CLIENT_CLUSTER_NAME_PROPERTY}. + * + * @return configured Hazelcast ClientConfig + */ + public static ClientConfig createClientConfig() { + ClientConfig config = new ClientConfig(); + + String clusterName = System.getProperty(CLIENT_CLUSTER_NAME_PROPERTY, DEFAULT_CLUSTER_NAME); + config.setClusterName(clusterName); + logger.info("Hazelcast client cluster name: {}", clusterName); + + String addressesProperty = System.getProperty(CLIENT_ADDRESSES_PROPERTY, DEFAULT_CLIENT_ADDRESS); + logger.info("Hazelcast client addresses: {}", addressesProperty); + + for (String address : addressesProperty.split(",")) { + String trimmed = address.trim(); + if (!trimmed.isEmpty()) { + config.getNetworkConfig().addAddress(trimmed); + } + } + + config.setProperty("hazelcast.logging.type", "slf4j"); + + // Register Compact Serializers — must match member config for cross-JVM deserialization + config.getSerializationConfig() + .getCompactSerializationConfig() + .addSerializer(new EventClaimSerializer()) + .addSerializer(new EntryDataSerializer()) + .addSerializer(new MemoryImprintDataSerializer()); + logger.debug("Registered Compact Serializers for EventClaim, EntryData, and MemoryImprintData"); + + logger.info("Hazelcast client configuration created for cluster: {}", clusterName); + return config; + } + + /** + * Generates a unique instance name for this Hazelcast client. + * Includes Jenkins URL and hostname for identification. + * + * @return instance name + */ + private static String generateInstanceName() { + StringBuilder name = new StringBuilder("gerrit-trigger"); + + // Add Jenkins root URL if available + Jenkins jenkins = Jenkins.getInstanceOrNull(); + if (jenkins != null) { + String rootUrl = jenkins.getRootUrl(); + if (rootUrl != null && !rootUrl.isEmpty()) { + // Extract hostname from URL + try { + java.net.URI uri = new java.net.URI(rootUrl); + String host = uri.getHost(); + if (host != null) { + name.append("-").append(host.replace('.', '-')); + } + } catch (Exception e) { + logger.debug("Could not parse Jenkins root URL: {}", rootUrl, e); + } + } + } + + // Add hostname + try { + String hostname = java.net.InetAddress.getLocalHost().getHostName(); + name.append("-").append(hostname.replace('.', '-')); + } catch (Exception e) { + logger.debug("Could not determine hostname", e); + } + + return name.toString(); + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationProvider.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationProvider.java new file mode 100644 index 000000000..e81005ce5 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationProvider.java @@ -0,0 +1,333 @@ +/* + * The MIT License + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.BuildMemoryStorage; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.QueueCancellationStrategy; +import hudson.Extension; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.concurrent.TimeUnit; + +/** + * Coordination provider for Hazelcast distributed mode. + *

+ * This provider is automatically discovered via Jenkins Extension Points mechanism. + * It has higher priority than LocalCoordinationProvider (100 vs -1000), so it will be + * selected when available. + *

+ * Availability Criteria: + *

    + *
  • Coordination mode set to 'hazelcast' via system property: + * {@code -Dgerrit.trigger.coordination.mode=hazelcast}
    • + *
  • Hazelcast instance is initialized and running
    • + *
+ *

+ * When selected, provides: + *

    + *
  • {@link HazelcastBuildMemoryStorage} - Distributed build tracking across replicas
    • + *
  • {@link HazelcastNotificationClaimStrategy} - Notification coordination to prevent duplicates
    • + *
  • {@link HazelcastEventClaimStrategy} - Event processing coordination to prevent duplicates
    • + *
+ *

+ * Architecture Note: This single class replaces ALL the + * {@code if (ClusterModeProvider.isClusterModeEnabled())} checks throughout the codebase! + * All three coordination concerns (build state storage, notification rights, event processing rights) + * now use the same Extension Points pattern consistently. + * + * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory + * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.LocalCoordinationProvider (fallback) + */ +@Extension(ordinal = HazelcastCoordinationProvider.HAZELCAST_PRIORITY) +public class HazelcastCoordinationProvider extends CoordinationModeProvider { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastCoordinationProvider.class); + + /** + * Extension ordinal priority for Hazelcast coordination provider. + * Higher value than LocalCoordinationProvider (-1000) ensures this is selected first when available. + */ + static final int HAZELCAST_PRIORITY = 100; + + /** + * The mode name that enables this provider. + */ + private static final String HAZELCAST_MODE = "hazelcast"; + + /** + * System property: minimum number of Hazelcast cluster members expected before connecting to Gerrit. + * Default 1 disables the wait (single-instance or local mode). + *

+ * Only relevant for an externally-managed Hazelcast cluster whose member + * discovery/formation is decoupled from this Jenkins replica's own startup - e.g. a shared + * cluster whose membership can still be changing (scaling, rebalancing, network delays) + * independently of when this replica boots. In that topology, formation time isn't bounded + * by anything Jenkins controls, so it can plausibly exceed Jenkins' own (slow) startup time. + *

+ * Does not apply to a per-replica Hazelcast sidecar (one server co-located + * with each Jenkins pod, joining only its Jenkins-managed peers): there, sidecar formation + * and Jenkins startup share the same pod lifecycle, and in practice Jenkins' own boot time + * (JVM start, CasC, plugin/extension loading - tens of seconds) dwarfs sidecar discovery time + * (single-digit seconds even from a cold multi-pod restart), so the client always observes + * the fully-formed cluster on its first connection regardless of this setting. + */ + public static final String HAZELCAST_EXPECTED_MEMBERS_PROPERTY = + "gerrit.trigger.coordination.hazelcast.expected.members"; + + /** + * System property: maximum seconds to wait for Hazelcast cluster formation. + * Default: 30 seconds. + */ + public static final String HAZELCAST_CLUSTER_WAIT_TIMEOUT_PROPERTY = + "gerrit.trigger.coordination.hazelcast.cluster.wait.timeout.seconds"; + + private static final int DEFAULT_EXPECTED_CLUSTER_MEMBERS = 1; + + private static final int DEFAULT_CLUSTER_WAIT_TIMEOUT_SECONDS = 30; + + private static final long CLUSTER_WAIT_POLL_INTERVAL_MS = 500L; + + /** + * The Hazelcast instance for this provider. + * Set during initialization, used to create strategies. + */ + private HazelcastInstance hazelcastInstance; + + /** + * Checks if this provider is available. + *

+ * Returns true only if: + *

    + *
  • Coordination mode is configured as 'hazelcast' (via system property)
    • + *
  • Hazelcast instance is initialized and running
    • + *
+ *

+ * Uses the {@link CoordinationModeProvider#getConfiguredMode()} helper method to check + * the coordination mode. This is future-proof - when we add UI configuration for coordination + * modes, only that one helper method needs to be updated. + *

+ * The initialization check is necessary because + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory} + * may call this method before + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl} + * has initialized providers. Without this check, the factory would select Hazelcast provider + * before Hazelcast is actually running, causing builds to not trigger. + * + * @return true if Hazelcast coordination mode is available, false otherwise + */ + @Override + public boolean isAvailable() { + // Check coordination mode using helper method (future-proof for UI config) + String configuredMode = getConfiguredMode(); + if (!HAZELCAST_MODE.equalsIgnoreCase(configuredMode)) { + logger.trace("Coordination mode is '{}', not '{}'", configuredMode, HAZELCAST_MODE); + return false; + } + + // Check Hazelcast availability + if (!HazelcastInstanceProvider.isInitialized()) { + logger.debug("Coordination mode is '{}' but Hazelcast not initialized yet. " + + "Provider will become available after initialization.", HAZELCAST_MODE); + return false; + } + + logger.debug("Hazelcast coordination mode active"); + return true; + } + + /** + * Returns the human-readable name of this coordination mode. + * + * @return "Hazelcast (Distributed)" + */ + @Override + public String getModeName() { + return "Hazelcast (Distributed)"; + } + + /** + * Creates Hazelcast-backed build memory storage. + *

+ * Uses distributed IMap to share build tracking state across all Jenkins replicas. + * All build lifecycle events (triggered, started, completed) are stored in Hazelcast, + * allowing any replica to see what builds other replicas are processing. + * + * @return HazelcastBuildMemoryStorage instance + */ + @Override + public BuildMemoryStorage createStorage() { + // Fetch instance from provider (multiple Extension instances may exist) + HazelcastInstance instance = HazelcastInstanceProvider.getInstanceOrThrow(); + logger.info("Creating HazelcastBuildMemoryStorage with instance: {}", instance.getName()); + return new HazelcastBuildMemoryStorage(instance); + } + + /** + * Creates Hazelcast notification claim strategy. + *

+ * Uses distributed atomic operations to ensure only one replica sends feedback + * to Gerrit for each event. Prevents duplicate comments/votes on Gerrit reviews. + * + * @return HazelcastNotificationClaimStrategy instance + */ + @Override + public NotificationClaimStrategy createClaimStrategy() { + // Fetch instance from provider (multiple Extension instances may exist) + HazelcastInstance instance = HazelcastInstanceProvider.getInstanceOrThrow(); + logger.info("Creating HazelcastNotificationClaimStrategy with instance: {}", instance.getName()); + return new HazelcastNotificationClaimStrategy(instance); + } + + /** + * Creates Hazelcast event claim strategy. + *

+ * Uses distributed IMap with atomic {@code putIfAbsent} to ensure only one replica + * processes each Gerrit event. The first replica to claim an event processes it, + * while other replicas skip it. This prevents duplicate builds in distributed scenarios. + *

+ * Replica-level claiming: Once a replica claims an event, ALL jobs + * on that replica can process it. This allows multiple jobs on the same replica to be + * triggered by the same event while preventing duplicate processing across replicas. + * + * @return HazelcastEventClaimStrategy instance + */ + @Override + public EventClaimStrategy createEventClaimStrategy() { + // Fetch instance from provider (multiple Extension instances may exist) + HazelcastInstance instance = HazelcastInstanceProvider.getInstanceOrThrow(); + logger.info("Creating HazelcastEventClaimStrategy with instance: {}", instance.getName()); + return new HazelcastEventClaimStrategy(instance); + } + + /** + * Creates Hazelcast queue cancellation strategy. + *

+ * Detects cancellations triggered by the potential distributed load balancer so that + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritQueueListener} + * can skip them and avoid sending premature Gerrit feedback. + * + * @return HazelcastQueueCancellationStrategy instance + */ + @Override + public QueueCancellationStrategy createQueueCancellationStrategy() { + return new HazelcastQueueCancellationStrategy(); + } + + /** + * Initializes Hazelcast coordination mode. + *

+ * Only initializes if coordination mode is configured as 'hazelcast'. + * This ensures Hazelcast is not started when using local mode. + *

+ * Also waits for the Hazelcast cluster to reach the expected member count (see + * {@link #HAZELCAST_EXPECTED_MEMBERS_PROPERTY}) before returning, so that + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl#start()} does not open + * Gerrit server connections until the distributed claim map is shared across replicas. + *

+ * If initialization fails, an exception is thrown and the provider will + * not be available (isAvailable() will return false). + * + * @throws Exception if Hazelcast initialization fails + */ + @Override + public void initialize() throws Exception { + // Check coordination mode - only initialize if this provider should be used + String configuredMode = getConfiguredMode(); + if (!HAZELCAST_MODE.equalsIgnoreCase(configuredMode)) { + logger.trace("Coordination mode is '{}', skipping Hazelcast initialization", configuredMode); + return; + } + + logger.info("Initializing Hazelcast coordination mode..."); + this.hazelcastInstance = HazelcastManager.initialize(); + logger.info("Hazelcast initialized successfully"); + + waitForClusterFormation(this.hazelcastInstance); + } + + /** + * Waits for the Hazelcast cluster to reach the expected number of members. + *

+ * In distributed scenarios each replica has its own SSH connection to Gerrit and therefore + * receives every event independently. Without this guard, a replica that starts while + * the Hazelcast cluster is still forming will process events against its own single-member + * IMap, making the distributed claim invisible to other replicas and causing duplicate builds. + * See {@link #HAZELCAST_EXPECTED_MEMBERS_PROPERTY} for when this scenario actually applies - + * in short, an externally-managed cluster, not a per-replica sidecar. + *

+ * The wait is skipped when {@link #HAZELCAST_EXPECTED_MEMBERS_PROPERTY} is 1 (the default). + * + * @param hz the Hazelcast instance whose cluster membership should be observed + */ + private void waitForClusterFormation(HazelcastInstance hz) { + int expectedMembers = Integer.getInteger(HAZELCAST_EXPECTED_MEMBERS_PROPERTY, + DEFAULT_EXPECTED_CLUSTER_MEMBERS); + if (expectedMembers <= DEFAULT_EXPECTED_CLUSTER_MEMBERS) { + return; + } + + int timeoutSeconds = Integer.getInteger(HAZELCAST_CLUSTER_WAIT_TIMEOUT_PROPERTY, + DEFAULT_CLUSTER_WAIT_TIMEOUT_SECONDS); + logger.info("Waiting for Hazelcast cluster to form ({} expected members, timeout: {}s)...", + expectedMembers, timeoutSeconds); + + long deadline = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(timeoutSeconds); + int currentSize = hz.getCluster().getMembers().size(); + while (currentSize < expectedMembers && System.currentTimeMillis() < deadline) { + logger.debug("Hazelcast cluster has {} of {} expected members, waiting...", + currentSize, expectedMembers); + try { + Thread.sleep(CLUSTER_WAIT_POLL_INTERVAL_MS); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + logger.warn("Interrupted while waiting for Hazelcast cluster formation"); + return; + } + currentSize = hz.getCluster().getMembers().size(); + } + + if (currentSize >= expectedMembers) { + logger.info("Hazelcast cluster ready: {} member(s)", currentSize); + } else { + logger.warn("Timed out waiting for Hazelcast cluster ({}/{} members). " + + "Proceeding anyway - duplicate builds may occur.", currentSize, expectedMembers); + } + } + + /** + * Shuts down Hazelcast coordination mode. + *

+ * This gracefully shuts down the Hazelcast instance, leaving the cluster + * and releasing all resources. + */ + @Override + public void shutdown() { + logger.info("Shutting down Hazelcast coordination mode..."); + HazelcastManager.shutdown(); + logger.info("Hazelcast shut down complete"); + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastEventClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastEventClaimStrategy.java new file mode 100644 index 000000000..f86d4ae22 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastEventClaimStrategy.java @@ -0,0 +1,269 @@ +/* + * The MIT License + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import com.hazelcast.map.IMap; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResult; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResults; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.net.InetAddress; +import java.util.concurrent.TimeUnit; + +/** + * Hazelcast-backed implementation of EventClaimStrategy for distributed scenarios. + *

+ * In distributed scenarios with multiple replicas, each Gerrit event + * arrives at all replicas via SSH event stream. To prevent duplicate builds, + * replicas use distributed event claiming: + *

    + *
  • First replica to claim an event processes it
    • + *
  • Other replicas skip the event (already claimed)
    • + *
+ *

+ * Claims are stored in a Hazelcast IMap with atomic {@code putIfAbsent} operations + * to prevent race conditions. Claims automatically expire via TTL to prevent memory leaks. + *

+ * Replica-level claiming: Once a replica claims an event, ALL jobs + * on that replica can process it. This allows multiple jobs on the same replica to be + * triggered by the same event while preventing duplicate processing across replicas. + *

+ * Fail-open behavior: If Hazelcast is unavailable, this strategy + * allows event processing to continue (better to risk duplicate builds than drop events). + * + */ +public class HazelcastEventClaimStrategy extends EventClaimStrategy { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastEventClaimStrategy.class); + + /** + * The Hazelcast instance to use for event claiming. + */ + private final HazelcastInstance hazelcastInstance; + + /** + * Hazelcast map name for event claims. + */ + private static final String CLAIMS_MAP_NAME = "gerrit-trigger-event-claims"; + + /** + * Constructor. + * + * @param hazelcastInstance the Hazelcast instance to use + */ + public HazelcastEventClaimStrategy(@NonNull HazelcastInstance hazelcastInstance) { + this.hazelcastInstance = hazelcastInstance; + } + + /** + * Default TTL for event claims in seconds (5 minutes). + * Claims expire automatically to prevent memory leaks. + */ + private static final long DEFAULT_CLAIM_TTL_SECONDS = 300; + + /** + * System property to override claim TTL. + * Example: -Dgerrit.trigger.coordination.hazelcast.claim.ttl.seconds=600 + */ + private static final String CLAIM_TTL_PROPERTY = "gerrit.trigger.coordination.hazelcast.claim.ttl.seconds"; + + /** + * Cached claim TTL in seconds. + * Parsed once at class initialization from system property or default. + */ + private static final long CLAIM_TTL_SECONDS = parseClaimTtlSeconds(); + + /** + * Cached instance identifier (hostname or pod name). + */ + private static volatile String instanceId = null; + + @Override + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, @NonNull Runnable claimed) { + // Check Hazelcast instance availability + if (hazelcastInstance == null) { + logger.error("Hazelcast not available, cannot claim event. Allowing event processing (fail-open)."); + // Fail-open: execute the action even without claiming + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception e) { + return ClaimResults.failed(e); + } + } + + // Generate event ID + String eventId = EventIdGenerator.generateEventId(event); + String thisInstanceId = getInstanceId(); + + try { + // Get claims map + IMap claimsMap = hazelcastInstance.getMap(CLAIMS_MAP_NAME); + + // Check if event is already claimed + EventClaim existingClaim = claimsMap.get(eventId); + + if (existingClaim != null) { + // Event already claimed - check who claimed it + if (existingClaim.getClaimedBy().equals(thisInstanceId)) { + // Claimed by THIS replica (another job already processed it) + // Allow this job to also process the event + logger.trace("Event already claimed by this replica, allowing: {} (job processing)", + eventId); + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception actionException) { + logger.error("Error executing action after claim: {}", eventId, actionException); + return ClaimResults.failed(actionException); + } + } else { + // Claimed by ANOTHER replica - skip processing + logger.debug("Event already claimed by {}: {} (type: {})", + existingClaim.getClaimedBy(), eventId, event.getEventType().getTypeValue()); + return ClaimResults.notClaimed(); + } + } + + // Event not yet claimed - attempt to claim it + EventClaim claim = new EventClaim( + eventId, + thisInstanceId, + System.currentTimeMillis(), + event.getEventType().getTypeValue() + ); + + // Attempt atomic claim with TTL + EventClaim previousClaim = claimsMap.putIfAbsent( + eventId, + claim, + CLAIM_TTL_SECONDS, + TimeUnit.SECONDS + ); + + if (previousClaim == null) { + // Successfully claimed by this replica + logger.debug("Successfully claimed event: {} (type: {})", + eventId, event.getEventType().getTypeValue()); + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception actionException) { + logger.error("Error executing action after successful claim: {}", eventId, actionException); + return ClaimResults.failed(actionException); + } + } else { + // Race condition: another replica claimed it between our get() and putIfAbsent() + // Check if it was claimed by this replica or another + if (previousClaim.getClaimedBy().equals(thisInstanceId)) { + // Claimed by THIS replica (race between jobs on same replica) + logger.trace("Event claimed by this replica during race condition: {}", eventId); + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception actionException) { + logger.error("Error executing action in race condition: {}", eventId, actionException); + return ClaimResults.failed(actionException); + } + } else { + // Claimed by ANOTHER replica + logger.debug("Event claimed by {} during race condition: {} (type: {})", + previousClaim.getClaimedBy(), eventId, event.getEventType().getTypeValue()); + return ClaimResults.notClaimed(); + } + } + } catch (Exception e) { + // Hazelcast operation failed + logger.error("Failed to claim event, allowing processing to continue (fail-open): " + eventId, e); + // Fail-open: execute the action even on error + try { + claimed.run(); + } catch (Exception innerException) { + logger.error("Error executing claimed action after claim failure", innerException); + return ClaimResults.failed(innerException); + } + return ClaimResults.success(); + } + } + + /** + * Gets the current instance identifier. + *

+ * Uses hostname or pod name to identify this Jenkins instance. + * Cached after first retrieval for performance. + * + * @return instance ID (hostname or pod name) + */ + private static String getInstanceId() { + if (instanceId == null) { + synchronized (HazelcastEventClaimStrategy.class) { + if (instanceId == null) { + try { + instanceId = System.getenv("HOSTNAME"); + if (instanceId == null || "".equals(instanceId.trim())) { + instanceId = InetAddress.getLocalHost().getHostName(); + } + } catch (Exception e) { + logger.warn("Could not determine hostname, using fallback", e); + instanceId = "unknown-" + System.currentTimeMillis(); + } + } + } + } + return instanceId; + } + + /** + * Parses the configured claim TTL in seconds from system property. + *

+ * Called once at class initialization to parse and cache the TTL value. + * Can be overridden via system property {@link #CLAIM_TTL_PROPERTY}. + * Default is {@link #DEFAULT_CLAIM_TTL_SECONDS} (5 minutes). + * + * @return TTL in seconds + */ + private static long parseClaimTtlSeconds() { + String ttlProperty = System.getProperty(CLAIM_TTL_PROPERTY); + if (ttlProperty != null) { + try { + long ttl = Long.parseLong(ttlProperty); + if (ttl > 0) { + logger.info("Using custom claim TTL: {} seconds (from system property)", ttl); + return ttl; + } else { + logger.warn("Invalid claim TTL property (must be > 0): {}, using default: {}", + ttlProperty, DEFAULT_CLAIM_TTL_SECONDS); + } + } catch (NumberFormatException e) { + logger.warn("Invalid claim TTL property (not a number): {}, using default: {}", + ttlProperty, DEFAULT_CLAIM_TTL_SECONDS); + } + } + return DEFAULT_CLAIM_TTL_SECONDS; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastInstanceProvider.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastInstanceProvider.java new file mode 100644 index 000000000..c13acedda --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastInstanceProvider.java @@ -0,0 +1,160 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import edu.umd.cs.findbugs.annotations.CheckForNull; +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Singleton provider for the Hazelcast instance. + *

+ * Provides thread-safe access to the Hazelcast embedded member instance. + * The instance is set by {@link HazelcastManager} during initialization. + * + */ +public final class HazelcastInstanceProvider { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastInstanceProvider.class); + + private static volatile HazelcastInstance instance; + private static final Object LOCK = new Object(); + + /** + * Private constructor to prevent instantiation. + */ + private HazelcastInstanceProvider() { + // Singleton + } + + /** + * Sets the Hazelcast instance. + * Should only be called by HazelcastManager during initialization. + * + * @param hazelcastInstance the Hazelcast instance to set + * @throws IllegalStateException if instance is already set + */ + static void setInstance(@NonNull HazelcastInstance hazelcastInstance) { + synchronized (LOCK) { + if (instance != null) { + throw new IllegalStateException("Hazelcast instance is already set. " + + "Call clearInstance() before setting a new instance."); + } + instance = hazelcastInstance; + logger.info("Hazelcast instance set: {}", hazelcastInstance.getName()); + } + } + + /** + * Clears the Hazelcast instance. + * Should only be called by HazelcastManager during shutdown. + */ + static void clearInstance() { + synchronized (LOCK) { + if (instance != null) { + logger.info("Clearing Hazelcast instance: {}", instance.getName()); + instance = null; + } + } + } + + /** + * Gets the Hazelcast instance. + * + * @return the Hazelcast instance, or null if not initialized + */ + @CheckForNull + public static HazelcastInstance getInstance() { + return instance; + } + + /** + * Gets the Hazelcast instance, throwing an exception if not initialized. + * + * @return the Hazelcast instance + * @throws IllegalStateException if Hazelcast is not initialized + */ + @NonNull + public static HazelcastInstance getInstanceOrThrow() { + HazelcastInstance hz = instance; + if (hz == null) { + throw new IllegalStateException("Hazelcast instance is not initialized. " + + "Ensure coordination mode is 'hazelcast' and Hazelcast has been started."); + } + return hz; + } + + /** + * Checks if the Hazelcast instance is initialized. + * + * @return true if instance is initialized and running + */ + public static boolean isInitialized() { + HazelcastInstance hz = instance; + return hz != null && hz.getLifecycleService().isRunning(); + } + + /** + * Gets the cluster name if Hazelcast is initialized. + * + * @return cluster name, or null if not initialized + */ + @CheckForNull + public static String getClusterName() { + HazelcastInstance hz = instance; + if (hz != null) { + return hz.getConfig().getClusterName(); + } + return null; + } + + /** + * Gets the instance name if Hazelcast is initialized. + * + * @return instance name, or null if not initialized + */ + @CheckForNull + public static String getInstanceName() { + HazelcastInstance hz = instance; + if (hz != null) { + return hz.getName(); + } + return null; + } + + /** + * Gets the number of members in the cluster. + * + * @return member count, or 0 if not initialized + */ + public static int getClusterSize() { + HazelcastInstance hz = instance; + if (hz != null) { + return hz.getCluster().getMembers().size(); + } + return 0; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastManager.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastManager.java new file mode 100644 index 000000000..1f5a1de55 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastManager.java @@ -0,0 +1,207 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.client.HazelcastClient; +import com.hazelcast.core.HazelcastInstance; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Manages the lifecycle of the Hazelcast client. + *

+ * This manager handles initialization and shutdown of the Hazelcast instance. + * Whether to initialize is determined by {@link HazelcastCoordinationProvider#isAvailable()}, + * not by this class. + * + */ +public final class HazelcastManager { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastManager.class); + + // TODO: This is still a bit dangerous to do and should try to be avoided. + // But hazelcast itself is already a static field, so perhaps nothing can be done? + private static volatile boolean initialized = false; + private static final Object INIT_LOCK = new Object(); + + /** + * Private constructor to prevent instantiation. + */ + private HazelcastManager() { + // Utility class + } + + /** + * Initializes the Hazelcast client. + *

+ * The client connects to an existing cluster (e.g. a Hazelcast sidecar on the same pod) + * and accesses its distributed maps. No cluster member is created inside Jenkins, so there + * is no port conflict with the sidecar and no need for cross-pod member discovery. + *

+ * This method is idempotent — calling it multiple times returns the existing instance. + * + * @return the Hazelcast client instance + * @throws RuntimeException if initialization fails + */ + public static HazelcastInstance initialize() { + synchronized (INIT_LOCK) { + if (initialized) { + logger.debug("Hazelcast is already initialized"); + HazelcastInstance existing = HazelcastInstanceProvider.getInstance(); + if (existing != null) { + return existing; + } + } + + try { + HazelcastInstance hazelcastInstance = initializeClient(); + HazelcastInstanceProvider.setInstance(hazelcastInstance); + initialized = true; + return hazelcastInstance; + + } catch (Exception e) { + logger.error("Failed to initialize Hazelcast", e); + initialized = false; + throw new RuntimeException("Failed to initialize Hazelcast", e); + } + } + } + + /** + * Creates a Hazelcast client using {@link HazelcastConfig#createClientConfig()}. + * + * @return the initialized Hazelcast client instance + */ + private static HazelcastInstance initializeClient() { + logger.info("Initializing Hazelcast client (connecting to existing cluster)..."); + com.hazelcast.client.config.ClientConfig config = HazelcastConfig.createClientConfig(); + HazelcastInstance hz = HazelcastClient.newHazelcastClient(config); + logger.info("Hazelcast client initialized. Cluster: {}, Members: {}", + config.getClusterName(), hz.getCluster().getMembers().size()); + return hz; + } + + /** + * Shuts down the Hazelcast client gracefully. + *

+ * This method is idempotent - calling it multiple times has no effect if already shut down. + */ + public static void shutdown() { + synchronized (INIT_LOCK) { + if (!initialized) { + logger.debug("Hazelcast is not initialized, nothing to shut down"); + return; + } + + try { + logger.info("Shutting down Hazelcast client..."); + + HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + if (instance != null) { + String instanceName = instance.getName(); + + // Shutdown the instance + instance.shutdown(); + + logger.info("Hazelcast client shut down: {}", instanceName); + } + + // Clear the provider + HazelcastInstanceProvider.clearInstance(); + + initialized = false; + + logger.info("Hazelcast shutdown complete"); + + } catch (Exception e) { + logger.error("Error during Hazelcast shutdown", e); + // Continue with cleanup even if error occurred + HazelcastInstanceProvider.clearInstance(); + initialized = false; + } + } + } + + /** + * Checks if Hazelcast is currently initialized. + * + * @return true if initialized + */ + public static boolean isInitialized() { + return initialized && HazelcastInstanceProvider.isInitialized(); + } + + /** + * Reinitializes Hazelcast. + * This will shutdown the existing instance and create a new one. + * Used when configuration has changed. + * + * @return the new Hazelcast instance + */ + public static HazelcastInstance reinitialize() { + logger.info("Reinitializing Hazelcast..."); + + synchronized (INIT_LOCK) { + // Shutdown existing instance + if (initialized) { + shutdown(); + } + + // Initialize new instance + return initialize(); + } + } + + /** + * Gets status information about Hazelcast cluster. + * + * @return status string with cluster information + */ + public static String getStatus() { + if (!initialized) { + return "Hazelcast: Not initialized"; + } + + HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + if (instance == null) { + return "Hazelcast: Error - initialized flag is true but instance is null"; + } + + if (!instance.getLifecycleService().isRunning()) { + return "Hazelcast: Not running"; + } + + try { + int clusterSize = instance.getCluster().getMembers().size(); + // getConfig() is not supported on Hazelcast clients — read the property directly + String clusterName = System.getProperty( + HazelcastConfig.CLIENT_CLUSTER_NAME_PROPERTY, + HazelcastConfig.DEFAULT_CLUSTER_NAME); + return String.format("Hazelcast Client: Running | Cluster: %s | Members: %d", + clusterName, clusterSize); + } catch (Exception e) { + return String.format("Hazelcast: Error getting status: %s", e.getMessage()); + } + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastNotificationClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastNotificationClaimStrategy.java new file mode 100644 index 000000000..076f3e807 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastNotificationClaimStrategy.java @@ -0,0 +1,201 @@ +/* + * The MIT License + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import com.hazelcast.map.IMap; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResult; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResults; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.concurrent.TimeUnit; + +/** + * Hazelcast-backed implementation of NotificationClaimStrategy for distributed scenarios. + *

+ * In distributed scenarios with multiple replicas, each replica tracks build + * completions independently. To prevent duplicate notifications to Gerrit, + * replicas use distributed notification claiming: + *

    + *
  • First replica to claim notification right sends feedback to Gerrit
    • + *
  • Other replicas skip notification (already sent)
    • + *
+ *

+ * Claims are stored in a Hazelcast IMap with atomic {@code putIfAbsent} operations + * to prevent race conditions. Claims automatically expire via TTL to prevent memory leaks. + *

+ * Fail-open behavior: If Hazelcast is unavailable, this strategy + * allows notification sending to continue (better to risk duplicate notifications than + * lose feedback entirely). + * + */ +public class HazelcastNotificationClaimStrategy extends NotificationClaimStrategy { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastNotificationClaimStrategy.class); + + /** + * The Hazelcast instance to use for notification claiming. + */ + private final HazelcastInstance hazelcastInstance; + + /** + * Hazelcast map name for notification claim flags. + */ + private static final String NOTIFICATION_FLAGS_MAP = "gerrit-trigger-notification-flags"; + + /** + * Constructor. + * + * @param hazelcastInstance the Hazelcast instance to use + */ + public HazelcastNotificationClaimStrategy(@NonNull HazelcastInstance hazelcastInstance) { + this.hazelcastInstance = hazelcastInstance; + } + + /** + * Default notification claim TTL in minutes (10 minutes). + * Claims expire automatically to prevent memory leaks. + */ + private static final int DEFAULT_NOTIFICATION_CLAIM_TTL_MINUTES = 10; + + /** + * System property to override notification claim TTL. + * Example: -Dgerrit.trigger.coordination.hazelcast.notification.ttl.minutes=20 + */ + private static final String NOTIFICATION_TTL_PROPERTY = + "gerrit.trigger.coordination.hazelcast.notification.ttl.minutes"; + + /** + * Cached claim TTL in minutes. + * Parsed once at class initialization from system property or default. + */ + private static final int NOTIFICATION_TTL_MINUTES = parseNotificationTtlMinutes(); + + + @Override + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, + @NonNull String notificationType, + String jobIdentifier, + @NonNull Runnable claimed) { + logger.debug("Claiming notification for event: {} (type: {}, job: {})", + event, notificationType, jobIdentifier); + + // Check Hazelcast instance availability + if (hazelcastInstance == null) { + logger.warn("Hazelcast not available for notification claim, proceeding with local mode (fail-open)"); + // Fail-open: execute the notification action even without claiming + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception e) { + return ClaimResults.failed(e); + } + } + + try { + IMap notificationFlags = hazelcastInstance.getMap(NOTIFICATION_FLAGS_MAP); + String eventId = EventIdGenerator.generateEventId(event); + + // Build claim key: + // - With job identifier: per-job claim (e.g., build-started notifications) + // - Without job identifier: per-event claim (e.g., build-completed notifications) + String flagKey; + if (jobIdentifier != null && !jobIdentifier.isEmpty()) { + flagKey = "notified-" + notificationType + "-" + eventId + "-" + jobIdentifier; + } else { + flagKey = "notified-" + notificationType + "-" + eventId; + } + + logger.debug("Notification claim key: {}", flagKey); + + // Atomic operation: set flag if not already set + Boolean previousValue = notificationFlags.putIfAbsent( + flagKey, + Boolean.TRUE, + NOTIFICATION_TTL_MINUTES, + TimeUnit.MINUTES + ); + + if (previousValue == null) { + // Successfully claimed notification right + logger.debug("Successfully claimed notification right for event: {} (type: {}, job: {})", + eventId, notificationType, jobIdentifier); + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception actionException) { + logger.error("Error executing notification action after successful claim: {} (type: {}, job: {})", + eventId, notificationType, jobIdentifier, actionException); + return ClaimResults.failed(actionException); + } + } else { + // Another replica already claimed notification + logger.debug("Another replica already claimed notification for event: {} (type: {}, job: {})", + eventId, notificationType, jobIdentifier); + return ClaimResults.notClaimed(); + } + } catch (Exception e) { + // Hazelcast operation failed + logger.error("Error claiming notification right, proceeding with send to avoid notification loss", e); + // Fail-open: execute the notification action even on error + try { + claimed.run(); + } catch (Exception innerException) { + logger.error("Error executing notification action after claim failure", innerException); + return ClaimResults.failed(innerException); + } + return ClaimResults.success(); + } + } + + /** + * Parse the configured notification claim TTL in minutes. + *

+ * Called once at class initialization to parse and cache the TTL value. + * Can be overridden via system property {@link #NOTIFICATION_TTL_PROPERTY}. + * Default is {@link #DEFAULT_NOTIFICATION_CLAIM_TTL_MINUTES} (10 minutes). + * + * @return TTL in minutes + */ + private static int parseNotificationTtlMinutes() { + String ttlProperty = System.getProperty(NOTIFICATION_TTL_PROPERTY); + if (ttlProperty != null) { + try { + int ttl = Integer.parseInt(ttlProperty); + if (ttl > 0) { + logger.info("Using custom notification TTL: {} seconds (from system property)", ttl); + return ttl; + } else { + logger.warn("Invalid notification TTL property (must be > 0): {}, using default", ttlProperty); + } + } catch (NumberFormatException e) { + logger.warn("Invalid notification TTL property (not a number): {}, using default", ttlProperty); + } + } + return DEFAULT_NOTIFICATION_CLAIM_TTL_MINUTES; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastQueueCancellationStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastQueueCancellationStrategy.java new file mode 100644 index 000000000..b45c0d38a --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastQueueCancellationStrategy.java @@ -0,0 +1,58 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.QueueCancellationStrategy; +import edu.umd.cs.findbugs.annotations.NonNull; +import hudson.model.Queue.LeftItem; + +/** + * Hazelcast (distributed) implementation of QueueCancellationStrategy. + * + * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastCoordinationProvider + * @see QueueCancellationStrategy + */ +public class HazelcastQueueCancellationStrategy extends QueueCancellationStrategy { + + /** + * Returns false unconditionally. + * + *

Potential {@code CancelQueueItem} calls {@code Queue.cancel(item)} with no markers + * attached to the resulting {@code LeftItem}:

+ *
    + *
  • {@code QueueLoadBalancerAction} is attached to the NEW item on the target instance + * (inside {@code QueueRequest} executed remotely), never to the item being cancelled.
    • + *
  • {@code LoadBalancedCauseOfBlockage} is a {@code BlockedItem.causeOfBlockage} that + * Jenkins does not copy into {@code LeftItem} — {@code LeftItem.getCauseOfBlockage()} + * returns null for load-balanced cancellations.
    • + *
+ * + * @param item the queue item that left the queue as cancelled + * @return always false + */ + @Override + public boolean isLoadBalancedCancellation(@NonNull LeftItem item) { + return false; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/MemoryImprintDataSerializer.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/MemoryImprintDataSerializer.java new file mode 100644 index 000000000..a58ecc0c0 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/MemoryImprintDataSerializer.java @@ -0,0 +1,152 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.hazelcast.nio.serialization.compact.CompactReader; +import com.hazelcast.nio.serialization.compact.CompactSerializer; +import com.hazelcast.nio.serialization.compact.CompactWriter; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.EntryData; +import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.MemoryImprintData; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.ArrayList; +import java.util.List; + +/** + * Hazelcast Compact Serializer for {@link MemoryImprintData}. + *

+ * Compact Serialization is schema-based and doesn't require class definitions + * on the Hazelcast server (sidecar container). This enables cross-JVM serialization + * without classloading issues. + *

+ * The {@link GerritTriggeredEvent} held by {@link MemoryImprintData} is a live object; this + * serializer is the boundary where it is turned into (and restored from) a JSON string on the + * wire, using {@link PolymorphicEventTypeAdapter} to preserve the concrete event subtype. Keeping + * this concern here means {@link MemoryImprintData} stays a plain, storage-agnostic DTO. + * + */ +public class MemoryImprintDataSerializer implements CompactSerializer { + + private static final Logger logger = LoggerFactory.getLogger(MemoryImprintDataSerializer.class); + + /** + * Type name for schema registration. + * Uses fully-qualified name to prevent conflicts in shared Hazelcast clusters. + */ + private static final String TYPE_NAME = "com.sonyericsson.gerrit.trigger.MemoryImprintData"; + + /** + * Gson instance for JSON serialization of events. + * Configured to handle polymorphic event types by including runtime type information. + */ + private static final Gson GSON = new GsonBuilder() + .registerTypeAdapter(GerritTriggeredEvent.class, new PolymorphicEventTypeAdapter()) + .create(); + + @Override + @NonNull + public MemoryImprintData read(@NonNull CompactReader reader) { + String eventJson = reader.readString("eventJson"); + GerritTriggeredEvent event = deserializeEvent(eventJson); + + // Read entries array using Compact Serialization array support + EntryData[] entriesArray = reader.readArrayOfCompact("entries", EntryData.class); + List entries = new ArrayList<>(); + if (entriesArray != null) { + for (EntryData entry : entriesArray) { + entries.add(entry); + } + } + + return new MemoryImprintData(event, entries); + } + + @Override + public void write(@NonNull CompactWriter writer, @NonNull MemoryImprintData data) { + writer.writeString("eventJson", serializeEvent(data.getEvent())); + + // Write entries array using Compact Serialization array support + List entries = data.getEntries(); + EntryData[] entriesArray = null; + if (entries != null && !entries.isEmpty()) { + entriesArray = entries.toArray(new EntryData[0]); + } + writer.writeArrayOfCompact("entries", entriesArray); + } + + /** + * Serializes a GerritTriggeredEvent to JSON. + * + * @param event the event to serialize, may be null + * @return JSON string, or null if the event is null or serialization fails + */ + private static String serializeEvent(GerritTriggeredEvent event) { + if (event == null) { + return null; + } + try { + // IMPORTANT: Must explicitly specify GerritTriggeredEvent.class to ensure + // the PolymorphicEventTypeAdapter is used, even when event is a concrete subclass. + return GSON.toJson(event, GerritTriggeredEvent.class); + } catch (Exception e) { + logger.error("Failed to serialize event to JSON: " + event, e); + return null; + } + } + + /** + * Deserializes a GerritTriggeredEvent from JSON. + * + * @param eventJson the JSON string, may be null + * @return deserialized event, or null if the JSON is null or deserialization fails + */ + private static GerritTriggeredEvent deserializeEvent(String eventJson) { + if (eventJson == null) { + return null; + } + try { + return GSON.fromJson(eventJson, GerritTriggeredEvent.class); + } catch (Exception e) { + logger.error("Failed to deserialize event from JSON (length: " + eventJson.length() + ")", e); + return null; + } + } + + @Override + @NonNull + public String getTypeName() { + return TYPE_NAME; + } + + @Override + @NonNull + public Class getCompactClass() { + return MemoryImprintData.class; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/PolymorphicEventTypeAdapter.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/PolymorphicEventTypeAdapter.java new file mode 100644 index 000000000..641c02271 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/PolymorphicEventTypeAdapter.java @@ -0,0 +1,86 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.google.gson.JsonDeserializationContext; +import com.google.gson.JsonDeserializer; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import com.google.gson.JsonSerializationContext; +import com.google.gson.JsonSerializer; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; + +import java.lang.reflect.Type; + +/** + * Custom Gson type adapter for polymorphic GerritTriggeredEvent serialization. + *

+ * Handles serialization and deserialization of GerritTriggeredEvent subclasses + * by including type information in the JSON. This allows proper reconstruction + * of the correct concrete event type when deserializing from Hazelcast. + *

+ * JSON format: + * 

+ * {
+ *   "@type": "com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated",
+ *   "data": { ... actual event properties ... }
+ * }
+ *
+ * + */ +public class PolymorphicEventTypeAdapter + implements JsonSerializer, JsonDeserializer { + + private static final String TYPE_FIELD = "@type"; + private static final String DATA_FIELD = "data"; + + @Override + public JsonElement serialize(GerritTriggeredEvent src, Type typeOfSrc, JsonSerializationContext context) { + JsonObject result = new JsonObject(); + result.addProperty(TYPE_FIELD, src.getClass().getName()); + result.add(DATA_FIELD, context.serialize(src, src.getClass())); + return result; + } + + @Override + public GerritTriggeredEvent deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) + throws JsonParseException { + JsonObject jsonObject = json.getAsJsonObject(); + + if (!jsonObject.has(TYPE_FIELD)) { + throw new JsonParseException("Missing type field '" + TYPE_FIELD + "' in JSON"); + } + + String className = jsonObject.get(TYPE_FIELD).getAsString(); + JsonElement data = jsonObject.get(DATA_FIELD); + + try { + Class clazz = Class.forName(className); + return context.deserialize(data, clazz); + } catch (ClassNotFoundException e) { + throw new JsonParseException("Unknown event type: " + className, e); + } + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/GerritNotifierFactory.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/GerritNotifierFactory.java index 1cdfe42b6..7de3e5a7b 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/GerritNotifierFactory.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/GerritNotifierFactory.java @@ -33,12 +33,14 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.IGerritHudsonTriggerConfig; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.job.ssh.BuildCompletedCommandJob; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.job.rest.BuildCompletedRestCommandJob; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.job.ssh.BuildStartedCommandJob; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.job.rest.BuildStartedRestCommandJob; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildsStartedStats; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; import hudson.model.Run; import hudson.model.TaskListener; import org.slf4j.Logger; @@ -118,14 +120,22 @@ public GerritNotifier createGerritNotifier(IGerritHudsonTriggerConfig config, Ge public void queueBuildCompleted(BuildMemory.MemoryImprint memoryImprint, TaskListener listener) { String serverName = getServerName(memoryImprint); if (serverName != null) { + GerritServer server = PluginImpl.getServer_(serverName); IGerritHudsonTriggerConfig config = getConfig(serverName); if (config != null) { - if (config.isUseRestApi() - && memoryImprint.getEvent() instanceof ChangeBasedEvent) { - GerritSendCommandQueue.queue(new BuildCompletedRestCommandJob(config, memoryImprint, listener)); - } else { - GerritSendCommandQueue.queue(new BuildCompletedCommandJob(config, memoryImprint, listener)); - } + GerritTriggeredEvent event = memoryImprint.getEvent(); + + // Claim notification for sending (prevents duplicate notifications in distributed scenarios) + NotificationClaimStrategy notificationClaimStrategy = + CoordinationModeFactory.get().getClaimStrategy(); + notificationClaimStrategy.withClaim(event, "build-completed", () -> { + if (config.isUseRestApi() + && event instanceof ChangeBasedEvent) { + GerritSendCommandQueue.queue(new BuildCompletedRestCommandJob(config, memoryImprint, listener)); + } else { + GerritSendCommandQueue.queue(new BuildCompletedCommandJob(config, memoryImprint, listener)); + } + }); } else { logger.warn("Nothing queued since there is no configuration for serverName: {}", serverName); } @@ -196,12 +206,19 @@ public void queueBuildStarted(Run build, TaskListener listener, if (serverName != null) { IGerritHudsonTriggerConfig config = getConfig(serverName); if (config != null) { - if (config.isUseRestApi() && event instanceof ChangeBasedEvent) { - GerritSendCommandQueue.queue(new BuildStartedRestCommandJob(config, build, listener, - (ChangeBasedEvent)event, stats)); - } else { - GerritSendCommandQueue.queue(new BuildStartedCommandJob(config, build, listener, event, stats)); - } + // Claim notification for sending (prevents duplicate notifications in distributed scenarios) + // Build-started uses per-job claim (each job sends its own notification) + String jobName = build.getParent().getFullName(); + NotificationClaimStrategy notificationClaimStrategy = + CoordinationModeFactory.get().getClaimStrategy(); + notificationClaimStrategy.withClaim(event, "build-started", jobName, () -> { + if (config.isUseRestApi() && event instanceof ChangeBasedEvent) { + GerritSendCommandQueue.queue(new BuildStartedRestCommandJob(config, build, listener, + (ChangeBasedEvent)event, stats)); + } else { + GerritSendCommandQueue.queue(new BuildStartedCommandJob(config, build, listener, event, stats)); + } + }); } else { logger.warn("Nothing queued since there is no configuration for serverName: {}", serverName); } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalEventClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalEventClaimStrategy.java new file mode 100644 index 000000000..41e7d4179 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalEventClaimStrategy.java @@ -0,0 +1,70 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier; + +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResult; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResults; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Local (standalone) implementation of EventClaimStrategy. + * Always succeeds since there is no coordination needed in single-instance mode. + * Executes the claimed action immediately. + * + *

This is the fallback implementation used when no higher-priority coordination + * mode (like Hazelcast) is available. In standalone Jenkins deployments, there's + * only one instance, so it always processes all events without coordination.

+ * + * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.LocalCoordinationProvider + * @see EventClaimStrategy + */ +public class LocalEventClaimStrategy extends EventClaimStrategy { + + private static final Logger logger = LoggerFactory.getLogger(LocalEventClaimStrategy.class); + + /** + * Claims the event and executes the action immediately. + * Always succeeds in local mode since there's no contention. + * + * @param event the Gerrit event to claim + * @param claimed action to execute (always runs in local mode) + * @return ClaimResult indicating success or error + */ + @Override + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, @NonNull Runnable claimed) { + // Local mode: always claim and execute immediately + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception e) { + logger.error("Error processing event in local mode", e); + return ClaimResults.failed(e); + } + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalNotificationClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalNotificationClaimStrategy.java index 6f630e435..5b5bb10dc 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalNotificationClaimStrategy.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalNotificationClaimStrategy.java @@ -23,13 +23,17 @@ */ package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResult; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.ClaimResults; import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.NotificationClaimStrategy; import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * Local (non-cluster) implementation of notification claiming. - * Always returns true since there's no need for coordination in standalone mode. + * Always executes the notification action since there's no need for coordination in standalone mode. * *

This is the default/fallback implementation used when cluster mode is not enabled. * In standalone Jenkins deployments, there's only one instance, so it always has the @@ -39,14 +43,22 @@ */ public class LocalNotificationClaimStrategy extends NotificationClaimStrategy { - @Override - public boolean tryClaimNotificationRight(@NonNull GerritTriggeredEvent event) { - // In local mode, always send notifications - no coordination needed - return true; - } + private static final Logger logger = LoggerFactory.getLogger(LocalNotificationClaimStrategy.class); @Override - public void releaseNotificationRight(@NonNull GerritTriggeredEvent event) { - // No-op in local mode - nothing to release + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, + @NonNull String notificationType, + String jobIdentifier, + @NonNull Runnable claimed) { + // In local mode, always allow notification - no coordination needed + // jobIdentifier is ignored since there's only one instance + try { + claimed.run(); + return ClaimResults.success(); + } catch (Exception e) { + logger.error("Error executing notification action", e); + return ClaimResults.failed(e); + } } } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalQueueCancellationStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalQueueCancellationStrategy.java new file mode 100644 index 000000000..38a853f05 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalQueueCancellationStrategy.java @@ -0,0 +1,49 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier; + +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.QueueCancellationStrategy; +import edu.umd.cs.findbugs.annotations.NonNull; +import hudson.model.Queue.LeftItem; + +/** + * Local (standalone) implementation of QueueCancellationStrategy. + * Always returns false since there is no distributed load balancer in single-instance mode. + * + * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.LocalCoordinationProvider + * @see QueueCancellationStrategy + */ +public class LocalQueueCancellationStrategy extends QueueCancellationStrategy { + + /** + * Always returns false in standalone mode — no load balancer is present. + * + * @param item the queue item that was cancelled + * @return false + */ + @Override + public boolean isLoadBalancedCancellation(@NonNull LeftItem item) { + return false; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java index 7c2ee8846..611e692f2 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java @@ -353,6 +353,12 @@ public void cancelOutdatedEvents( List outdatedEvents = new ArrayList<>(); CauseOfInterruption cause = new NewPatchSetInterruption(); + // True if newEvent is itself already outdated relative to some already-registered + // event that's a strictly newer patchset of the same change and still has an active + // build for this job. Set inside the loop below - see its own comment at the + // shouldIgnoreEvent call for why this direction needs a separate check. + boolean newEventIsOutdated = false; + synchronized (storage) { Map allEvents = storage.getAllEvents(); logger.info("BuildMemory has {} events in memory", allEvents.size()); @@ -369,7 +375,30 @@ public void cancelOutdatedEvents( ChangeBasedEvent runningChangeBasedEvent = (ChangeBasedEvent)runningEvent; logger.debug("Checking running event: {}", runningChangeBasedEvent); + // Never cancel an event against itself (self-cancellation). + // This can happen when isAbortNewPatchsets=true and the same event is + // processed by multiple jobs — the second job finds the first job's entry + // in memory and considers the event "outdated" against itself, poisoning + // the isCancelling flag before the build is even scheduled. + if (storage.eventsMatch(newEvent, runningChangeBasedEvent)) { + logger.debug("Skipping self-cancellation: running event matches new event"); + continue; + } + if (shouldIgnoreEvent(newEvent, policy, runningChangeBasedEvent, trigger)) { + // shouldIgnoreEvent can return true because runningChangeBasedEvent is + // actually the newer patchset (see its own isOldPatch check) - i.e. + // newEvent itself is the outdated one, not the other way around. This + // loop only ever asks "should I cancel the OTHER, already-registered + // event?" - it never asks the reverse. Left unguarded, a late-arriving + // older patchset's own build runs to completion fully unsuppressed + // whenever cross-replica event delivery reorders patchset arrival. + // Detect that case here so newEvent gets cancelled too, + // via the same isCancelling + deferred-abort machinery already used for + // the normal direction. + if (isNewEventOutdatedByRunningEvent(newEvent, runningChangeBasedEvent, jobName, entry.getValue())) { + newEventIsOutdated = true; + } logger.debug("Ignoring event based on policy"); continue; } @@ -382,6 +411,13 @@ public void cancelOutdatedEvents( logger.debug("Checking entry: project={}, completed={}, cancelling={}, cancelled={}", imprintEntry.getProject(), imprintEntry.isBuildCompleted(), imprintEntry.isCancelling(), imprintEntry.isCancelled()); + // Deliberately does NOT check !imprintEntry.isQueueLeft(): queueLeft means + // "left the queue for an ambiguous reason (possibly a load-balanced + // relocation to another replica), not yet confirmed as a genuine + // cancel" - it is not itself proof the entry is done. Excluding it here + // let relocated-but-not-yet-restarted entries dodge cancellation entirely + // whenever a newer patchset arrived during the relocation window + // (a cross-replica race). if (imprintEntry.isProject(jobName) && !imprintEntry.isBuildCompleted() && !imprintEntry.isCancelling() @@ -403,19 +439,8 @@ public void cancelOutdatedEvents( // in future cancellation checks (prevents state accumulation issues). // The actual "cancelled" flag will be set later by GerritQueueListener when Jenkins confirms. // - // IMPORTANT: We need to get the imprint from storage again to modify the real one, - // not the copy from getAllEvents() - MemoryImprint storageImprint = storage.getMemoryImprint(runningEvent); - if (storageImprint != null) { - for (Entry imprintEntry : storageImprint.getEntries()) { - if (imprintEntry.isProject(jobName) - && !imprintEntry.isBuildCompleted() - && !imprintEntry.isCancelling() - && !imprintEntry.isCancelled()) { - imprintEntry.setCancelling(true); - } - } - } + // Use storage.setCancelling() to persist the flag atomically + storage.setCancelling(runningEvent, job); } } @@ -431,6 +456,19 @@ public void cancelOutdatedEvents( // Add event so it can be found and cancelled by future events // This is critical for silent mode where onTriggered() isn't called triggered(newEvent, job); + + // newEvent turned out to be outdated relative to an already-active newer + // patchset for this exact job (see the loop above) - cancel it too, now + // that it's registered, via the same post-loop cancelMatchingJobs path + // used for the normal direction, so started()'s existing deferred-abort + // compensator can still catch it if its own build has already started or + // starts shortly on some replica. + if (newEventIsOutdated) { + logger.info("New event {} is itself outdated relative to an already-active " + + "newer patchset for job {} - cancelling it too", newEvent, jobName); + storage.setCancelling(newEvent, job); + outdatedEvents.add(newEvent); + } } } } @@ -466,7 +504,9 @@ private boolean shouldIgnoreEvent( if (!abortBecauseOfTopic) { Change change = runningChangeBasedEvent.getChange(); - if (!change.equals(event.getChange())) { + Change newChange = event.getChange(); + boolean changesEqual = change != null && change.equals(newChange); + if (!changesEqual) { return true; } @@ -499,6 +539,72 @@ private boolean shouldIgnoreEvent( return false; } + /** + * True if {@code runningEvent} is a strictly newer patchset of the same change as + * {@code newEvent} and still has an active (non-completed, non-cancelling, non-cancelled, + * non-queueLeft) build entry for {@code jobName} - i.e. {@code newEvent} itself is the + * outdated one here, not {@code runningEvent}. + *

+ * This is the mirror image of the {@code isOldPatch} check in {@link #shouldIgnoreEvent}. + * That check only ever decides whether {@code runningEvent} should be cancelled by + * {@code newEvent} - it correctly refuses to do so when {@code runningEvent} is actually + * newer, but nothing then cancels {@code newEvent} itself in that case. Cross-replica event + * delivery can deliver a newer patchset's event to some replica before an older one reaches + * any replica at all, which is exactly when this matters: without this check, the late-arriving, + * actually-outdated {@code newEvent} would never recognize itself as such and would run to + * completion alongside the newer patchset that's already building. + *

+ * Independently re-checks the same-change condition ({@link #shouldIgnoreEvent} can return + * {@code true} for several unrelated reasons - topic mismatch, different change, manual + * patchset policy - so its return value alone doesn't confirm this is a patchset-order + * situation on the same change). + * + * @param newEvent the event that just arrived + * @param runningEvent an already-registered event to compare against + * @param jobName the job to check for an active build + * @param imprint runningEvent's current memory imprint + * @return true if newEvent should be treated as already outdated relative to runningEvent + */ + private boolean isNewEventOutdatedByRunningEvent( + ChangeBasedEvent newEvent, ChangeBasedEvent runningEvent, String jobName, MemoryImprint imprint) { + + Change change = runningEvent.getChange(); + Change newChange = newEvent.getChange(); + if (change == null || !change.equals(newChange)) { + return false; + } + + if (newEvent.getPatchSet() == null || runningEvent.getPatchSet() == null + || newEvent.getPatchSet().getNumber() == null || runningEvent.getPatchSet().getNumber() == null) { + return false; + } + + int newEventNum; + int runningEventNum; + try { + newEventNum = Integer.parseInt(newEvent.getPatchSet().getNumber()); + runningEventNum = Integer.parseInt(runningEvent.getPatchSet().getNumber()); + } catch (NumberFormatException e) { + return false; + } + + if (runningEventNum <= newEventNum) { + return false; + } + + for (Entry imprintEntry : imprint.getEntries()) { + // See the identical comment in cancelOutdatedEvents() above: queueLeft is not + // proof this entry is done, so it must not exclude it from being "active". + if (imprintEntry.isProject(jobName) + && !imprintEntry.isBuildCompleted() + && !imprintEntry.isCancelling() + && !imprintEntry.isCancelled()) { + return true; + } + } + return false; + } + /** * Cancels any jobs that were triggered by the given event. * Ported from RunningJobs.cancelMatchingJobs(). @@ -553,6 +659,10 @@ private void cancelMatchingJobs( e.interrupt(Result.ABORTED, cause); } } + + // Ask the storage implementation to notify other replicas, if any, to + // abort matching builds on their local executors. + storage.requestCrossReplicaAbort(event, job, cause); } catch (Exception e) { logger.error("Error canceling job", e); } @@ -561,16 +671,27 @@ private void cancelMatchingJobs( /** * Checks if any of the given causes references the given event. * Ported from RunningJobs.checkCausedByGerrit(). + *

+ * Important: Event comparison is delegated to the storage implementation + * via {@link BuildMemoryStorage#eventsMatch(GerritTriggeredEvent, GerritTriggeredEvent)}, + * which always uses logical equality rather than instance identity ({@code ==}) - events + * can be deserialized (e.g. {@code GerritCause}'s event loaded from disk, or from Hazelcast + * in distributed mode), so two logically-equal events are not guaranteed to be the same + * instance: + *

    + *
  • Local mode: Uses {@link Object#equals(Object)}
    • + *
  • Distributed mode: Uses logical comparison via EventIdGenerator
    • + *
* - * @param event the event to check for (checks for identity, not equality) + * @param event the event to check for * @param causes the list of causes - * @return true if the list contains a GerritCause with this event + * @return true if the list contains a GerritCause with an equivalent event */ private boolean checkCausedByGerrit(GerritTriggeredEvent event, Collection causes) { for (Cause c : causes) { if (c instanceof GerritCause) { GerritCause gc = (GerritCause)c; - if (gc.getEvent() == event) { + if (storage.eventsMatch(event, gc.getEvent())) { return true; } } @@ -761,6 +882,41 @@ public List getEntriesList() { return list; } + /** + * Converts this imprint to its serialization-friendly {@link MemoryImprintData} form. + *

+ * The event is carried as-is; each entry is mapped via {@link Entry#toEntryData()}. + * No Jenkins lookups or serialization happen here — turning the event into a wire + * representation is left to the storage layer. + * + * @return the data representation of this imprint. + * @see #fromData(MemoryImprintData) + */ + public synchronized MemoryImprintData toData() { + List entries = new ArrayList(); + for (Entry entry : list) { + entries.add(entry.toEntryData()); + } + return new MemoryImprintData(event, entries); + } + + /** + * Restores a {@link MemoryImprint} from its {@link MemoryImprintData} form. + * + * @param data the data to restore from. + * @return the reconstructed imprint. + * @see #toData() + */ + public static MemoryImprint fromData(@NonNull MemoryImprintData data) { + MemoryImprint imprint = new MemoryImprint(data.getEvent()); + if (data.getEntries() != null) { + for (EntryData entryData : data.getEntries()) { + imprint.list.add(Entry.fromEntryData(entryData)); + } + } + return imprint; + } + /** * Sets the build to a project or adds the project to the list. * @@ -1046,7 +1202,7 @@ public synchronized boolean wereAllBuildsNotBuilt() { if (entry == null) { continue; } - if (entry.isCancelling() || entry.isCancelled()) { + if (entry.isCancelling() || entry.isCancelled() || entry.isQueueLeft()) { continue; } Run build = entry.getBuild(); @@ -1094,6 +1250,7 @@ public static class Entry implements Cloneable { private boolean buildCompleted; private boolean cancelling; private boolean cancelled; + private boolean queueLeft; private String customUrl; private String unsuccessfulMessage; private final long triggeredTimestamp; @@ -1144,6 +1301,7 @@ public Entry(Entry copy) { this.customUrl = copy.customUrl; this.cancelling = copy.cancelling; this.cancelled = copy.cancelled; + this.queueLeft = copy.queueLeft; } @Override @@ -1151,6 +1309,67 @@ public Entry clone() { return new Entry(this); } + /** + * Constructor that restores an entry from its {@link EntryData} form. + *

+ * All fields are copied verbatim, including the timestamps. Unlike the + * {@link #setBuild(Run)}/{@link #setBuildCompleted(boolean)} setters, this does not + * re-stamp {@code startedTimestamp}/{@code completedTimestamp} with the current time, + * so the original moments are preserved across a store/restore round-trip. + * + * @param data the data to restore from. + * @see #fromEntryData(EntryData) + */ + private Entry(EntryData data) { + this.project = data.getProjectFullName(); + this.build = data.getBuildId(); + this.buildCompleted = data.isBuildCompleted(); + this.cancelling = data.isCancelling(); + this.cancelled = data.isCancelled(); + this.queueLeft = data.isQueueLeft(); + this.customUrl = data.getCustomUrl(); + this.unsuccessfulMessage = data.getUnsuccessfulMessage(); + this.triggeredTimestamp = data.getTriggeredTimestamp(); + this.completedTimestamp = data.getCompletedTimestamp(); + this.startedTimestamp = data.getStartedTimestamp(); + } + + /** + * Converts this entry to its serialization-friendly {@link EntryData} form. + *

+ * This is a straight field copy with no Jenkins lookups: the entry already holds the + * project and build as {@code String} identifiers. + * + * @return the data representation of this entry. + * @see #fromEntryData(EntryData) + */ + public EntryData toEntryData() { + EntryData data = new EntryData(); + data.setProjectFullName(project); + data.setBuildId(build); + data.setBuildCompleted(buildCompleted); + data.setCancelling(cancelling); + data.setCancelled(cancelled); + data.setQueueLeft(queueLeft); + data.setCustomUrl(customUrl); + data.setUnsuccessfulMessage(unsuccessfulMessage); + data.setTriggeredTimestamp(triggeredTimestamp); + data.setCompletedTimestamp(completedTimestamp); + data.setStartedTimestamp(startedTimestamp); + return data; + } + + /** + * Restores an {@link Entry} from its {@link EntryData} form. + * + * @param data the data to restore from. + * @return the reconstructed entry. + * @see #toEntryData() + */ + public static Entry fromEntryData(@NonNull EntryData data) { + return new Entry(data); + } + /** * The Project. * @@ -1301,6 +1520,35 @@ public void setCancelled(boolean cancelled) { this.cancelled = cancelled; } + /** + * Whether the queue item left the queue without a prior Gerrit-triggered cancellation intent. + *

+ * This flag covers two cases that are indistinguishable at {@code QueueListener.onLeft} time: + *

    + *
  • Potential load-balanced move — the item was moved to another instance; + * the build will reappear via {@code onStarted} on that instance.
    • + *
  • Direct {@code Queue.doCancelItem} without a preceding {@code setCancelling} — + * truly removed but not through the normal Gerrit cancellation path.
    • + *
+ * Unlike {@link #isCancelled()}, setting this flag does NOT also set + * {@link #setBuildCompleted(boolean)}, preserving the IMap entry for cross-instance + * new-patchset abort scenarios. + * + * @return true if the queue item left without a prior cancelling intent + */ + public boolean isQueueLeft() { + return queueLeft; + } + + /** + * Sets the queueLeft flag. + * + * @param queueLeft true if the queue item left without a prior cancelling intent + */ + public void setQueueLeft(boolean queueLeft) { + this.queueLeft = queueLeft; + } + /** * The timestamp when {@link #setBuildCompleted(boolean)} was set to true. * null indicates not completed yet. diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/EntryData.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/EntryData.java new file mode 100644 index 000000000..8008bbfbf --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/EntryData.java @@ -0,0 +1,300 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model; + +import edu.umd.cs.findbugs.annotations.CheckForNull; + +/** + * Plain data transfer object mirroring the state of a + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint.Entry}. + *

+ * It carries only primitives and strings, so it can be serialized and shipped across JVMs + * (for distributed storage backends) without dragging in Jenkins object references. + * Note that {@code Entry} itself already stores the project and build as {@code String} + * identifiers and resolves the live {@link hudson.model.Job}/{@link hudson.model.Run} + * lazily, so the conversion is a straight field copy — see + * {@link BuildMemory.MemoryImprint.Entry#toEntryData()} and + * {@link BuildMemory.MemoryImprint.Entry#fromEntryData(EntryData)}. + *

+ * This class is intentionally free of any storage-technology dependency. Concerns such as + * wire serialization live in the storage layer (e.g. the Hazelcast compact serializer), + * keeping this type reusable and easy to extract. + * + * @see MemoryImprintData + * @see BuildMemory.MemoryImprint.Entry + */ +public class EntryData { + + private String projectFullName; + private String buildId; + private boolean buildCompleted; + private boolean cancelling; + private boolean cancelled; + private boolean queueLeft; + private String customUrl; + private String unsuccessfulMessage; + private long triggeredTimestamp; + private Long completedTimestamp; + private Long startedTimestamp; + + /** + * Default constructor. + */ + public EntryData() { + this.triggeredTimestamp = System.currentTimeMillis(); + } + + /** + * Constructor with parameters. + * + * @param projectFullName full job name + * @param buildId build identifier + * @param buildCompleted whether build is completed + */ + public EntryData(String projectFullName, String buildId, boolean buildCompleted) { + this.projectFullName = projectFullName; + this.buildId = buildId; + this.buildCompleted = buildCompleted; + this.triggeredTimestamp = System.currentTimeMillis(); + } + + /** + * Gets the project full name. + * + * @return project full name + */ + public String getProjectFullName() { + return projectFullName; + } + + /** + * Sets the project full name. + * + * @param projectFullName project full name + */ + public void setProjectFullName(String projectFullName) { + this.projectFullName = projectFullName; + } + + /** + * Gets the build ID. + * + * @return build ID + */ + @CheckForNull + public String getBuildId() { + return buildId; + } + + /** + * Sets the build ID. + *

+ * Note: Does not automatically set startedTimestamp. Callers should explicitly set + * the timestamp using {@link #setStartedTimestamp(Long)} when appropriate. + * + * @param buildId build ID + */ + public void setBuildId(String buildId) { + this.buildId = buildId; + } + + /** + * Checks if build is completed. + * + * @return true if completed + */ + public boolean isBuildCompleted() { + return buildCompleted; + } + + /** + * Sets build completed status. + *

+ * Note: Does not automatically set completedTimestamp. Callers should explicitly set + * the timestamp using {@link #setCompletedTimestamp(Long)} when appropriate. + * + * @param buildCompleted completed status + */ + public void setBuildCompleted(boolean buildCompleted) { + this.buildCompleted = buildCompleted; + } + + /** + * Checks if build is being cancelled (cancellation intent). + * + * @return true if cancellation initiated + */ + public boolean isCancelling() { + return cancelling; + } + + /** + * Sets cancelling status (cancellation intent). + * + * @param cancelling cancelling status + */ + public void setCancelling(boolean cancelling) { + this.cancelling = cancelling; + } + + /** + * Checks if build was cancelled. + * + * @return true if cancelled + */ + public boolean isCancelled() { + return cancelled; + } + + /** + * Sets cancelled status. + * + * @param cancelled cancelled status + */ + public void setCancelled(boolean cancelled) { + this.cancelled = cancelled; + } + + /** + * Checks if the queue item left the queue without a prior Gerrit-triggered cancellation intent. + *

+ * This flag covers two cases that look identical at {@code QueueListener.onLeft} time: + *

    + *
  • Potential load-balanced move — item moved to another instance; the build will appear + * again on that instance via {@code onStarted}.
    • + *
  • Direct queue item cancellation without a preceding {@code setCancelling} — + * item truly removed but without going through the normal Gerrit cancellation path.
    • + *
+ * Unlike {@link #isCancelled()}, this flag does NOT set {@link #isBuildCompleted()}, so the + * IMap entry is preserved for cross-replica PS2-aborts-PS1 scenarios. + * + * @return true if the queue item left without a prior cancelling intent + */ + public boolean isQueueLeft() { + return queueLeft; + } + + /** + * Sets the queueLeft flag. + * + * @param queueLeft true if the queue item left without a prior cancelling intent + */ + public void setQueueLeft(boolean queueLeft) { + this.queueLeft = queueLeft; + } + + /** + * Gets custom URL. + * + * @return custom URL + */ + @CheckForNull + public String getCustomUrl() { + return customUrl; + } + + /** + * Sets custom URL. + * + * @param customUrl custom URL + */ + public void setCustomUrl(String customUrl) { + this.customUrl = customUrl; + } + + /** + * Gets unsuccessful message. + * + * @return unsuccessful message + */ + @CheckForNull + public String getUnsuccessfulMessage() { + return unsuccessfulMessage; + } + + /** + * Sets unsuccessful message. + * + * @param unsuccessfulMessage unsuccessful message + */ + public void setUnsuccessfulMessage(String unsuccessfulMessage) { + this.unsuccessfulMessage = unsuccessfulMessage; + } + + /** + * Gets triggered timestamp. + * + * @return triggered timestamp + */ + public long getTriggeredTimestamp() { + return triggeredTimestamp; + } + + /** + * Sets triggered timestamp. + * + * @param triggeredTimestamp triggered timestamp + */ + public void setTriggeredTimestamp(long triggeredTimestamp) { + this.triggeredTimestamp = triggeredTimestamp; + } + + /** + * Gets completed timestamp. + * + * @return completed timestamp + */ + @CheckForNull + public Long getCompletedTimestamp() { + return completedTimestamp; + } + + /** + * Sets completed timestamp. + * + * @param completedTimestamp completed timestamp + */ + public void setCompletedTimestamp(Long completedTimestamp) { + this.completedTimestamp = completedTimestamp; + } + + /** + * Gets started timestamp. + * + * @return started timestamp + */ + @CheckForNull + public Long getStartedTimestamp() { + return startedTimestamp; + } + + /** + * Sets started timestamp. + * + * @param startedTimestamp started timestamp + */ + public void setStartedTimestamp(Long startedTimestamp) { + this.startedTimestamp = startedTimestamp; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintData.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintData.java new file mode 100644 index 000000000..a47d1611f --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintData.java @@ -0,0 +1,119 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model; + +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; + +import java.util.ArrayList; +import java.util.List; + +/** + * Plain data transfer object mirroring the state of a + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory.MemoryImprint}. + *

+ * It holds the triggering {@link GerritTriggeredEvent} together with a list of {@link EntryData}. + * Conversion to and from {@code MemoryImprint} is a straight structural copy provided by + * {@link BuildMemory.MemoryImprint#toData()} and {@link BuildMemory.MemoryImprint#fromData(MemoryImprintData)}; + * neither performs any Jenkins lookups. + *

+ * Serialization is not this type's concern. The event is kept as a live object here. + * Turning it into a storage/wire representation (for example JSON, for distributed backends) + * is the responsibility of the storage layer's serializer, so this DTO stays free of any + * storage-technology dependency and is easy to reuse or extract. + * + * @see EntryData + * @see BuildMemory.MemoryImprint + */ +public class MemoryImprintData { + + private GerritTriggeredEvent event; + private List entries; + + /** + * Default constructor. + */ + public MemoryImprintData() { + this.entries = new ArrayList<>(); + } + + /** + * Constructor with parameters. + * + * @param event the triggering event + * @param entries list of entry data + */ + public MemoryImprintData(GerritTriggeredEvent event, List entries) { + this.event = event; + if (entries != null) { + this.entries = entries; + } else { + this.entries = new ArrayList<>(); + } + } + + /** + * Gets the triggering event. + * + * @return the event + */ + public GerritTriggeredEvent getEvent() { + return event; + } + + /** + * Sets the triggering event. + * + * @param event the event + */ + public void setEvent(GerritTriggeredEvent event) { + this.event = event; + } + + /** + * Gets the list of entries. + * + * @return list of entry data + */ + public List getEntries() { + return entries; + } + + /** + * Sets the list of entries. + * + * @param entries list of entry data + */ + public void setEntries(List entries) { + this.entries = entries; + } + + /** + * Adds an entry to the list. + * + * @param entry the entry to add + */ + public void addEntry(EntryData entry) { + this.entries.add(entry); + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java index e719aec2e..13c22bdb2 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java @@ -24,11 +24,14 @@ package com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.IGerritHudsonTriggerConfig; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.BuildCancellationPolicy; import com.sonyericsson.hudson.plugins.gerrit.trigger.events.ManualPatchsetCreated; import com.sonyericsson.hudson.plugins.gerrit.trigger.events.lifecycle.GerritEventLifecycle; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.ToGerritRunListener; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.actions.RetriggerAction; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.actions.RetriggerAllAction; +import com.sonyericsson.hudson.plugins.gerrit.trigger.spi.EventClaimStrategy; import com.sonymobile.tools.gerrit.gerritevents.GerritEventListener; import com.sonymobile.tools.gerrit.gerritevents.dto.GerritEvent; import com.sonymobile.tools.gerrit.gerritevents.dto.events.ChangeBasedEvent; @@ -123,18 +126,23 @@ public void gerritEvent(GerritEvent event) { } if (event instanceof GerritTriggeredEvent) { GerritTriggeredEvent triggeredEvent = (GerritTriggeredEvent)event; - synchronized (this) { - if (t.isInteresting(triggeredEvent)) { - logger.trace("The event is interesting."); - abortBuild(t, triggeredEvent); - if (t.isOnlyAbortRunningBuild(triggeredEvent)) { - logger.trace("Just aborting build based on event not scheduling new one."); - return; + + // Claim event for processing (prevents duplicate builds in distributed scenarios) + EventClaimStrategy eventClaimStrategy = CoordinationModeFactory.get().getEventClaimStrategy(); + eventClaimStrategy.withClaim(triggeredEvent, () -> { + synchronized (EventListener.this) { + if (t.isInteresting(triggeredEvent)) { + logger.trace("The event is interesting."); + abortBuild(t, triggeredEvent); + if (t.isOnlyAbortRunningBuild(triggeredEvent)) { + logger.trace("Just aborting build based on event not scheduling new one."); + return; + } + notifyOnTriggered(t, triggeredEvent); + schedule(t, new GerritCause(triggeredEvent, t.isSilentMode()), triggeredEvent); } - notifyOnTriggered(t, triggeredEvent); - schedule(t, new GerritCause(triggeredEvent, t.isSilentMode()), triggeredEvent); } - } + }); } } @@ -163,18 +171,23 @@ public void gerritEvent(ManualPatchsetCreated event) { // to just return now without processing the event. return; } - synchronized (this) { - if (t.isInteresting(event)) { - logger.trace("The event is interesting."); - abortBuild(t, event); - if (t.isOnlyAbortRunningBuild(event)) { - logger.trace("Just aborting build based on event not scheduling new one."); - return; + + // Claim event for processing (prevents duplicate builds in distributed scenarios) + EventClaimStrategy eventClaimStrategy = CoordinationModeFactory.get().getEventClaimStrategy(); + eventClaimStrategy.withClaim(event, () -> { + synchronized (EventListener.this) { + if (t.isInteresting(event)) { + logger.trace("The event is interesting."); + abortBuild(t, event); + if (t.isOnlyAbortRunningBuild(event)) { + logger.trace("Just aborting build based on event not scheduling new one."); + return; + } + notifyOnTriggered(t, event); + schedule(t, new GerritManualCause(event, t.isSilentMode()), event); } - notifyOnTriggered(t, event); - schedule(t, new GerritManualCause(event, t.isSilentMode()), event); } - } + }); } /** @@ -209,18 +222,23 @@ public void gerritEvent(CommentAdded event) { // to just return now without processing the event. return; } - synchronized (this) { - if (t.isInteresting(event) && t.commentAddedMatch(event)) { - logger.trace("The event is interesting."); - abortBuild(t, event); - if (t.isOnlyAbortRunningBuild(event)) { - logger.trace("Just aborting build based on event not scheduling new one."); - return; + + // Claim event for processing (prevents duplicate builds in distributed scenarios) + EventClaimStrategy eventClaimStrategy = CoordinationModeFactory.get().getEventClaimStrategy(); + eventClaimStrategy.withClaim(event, () -> { + synchronized (EventListener.this) { + if (t.isInteresting(event) && t.commentAddedMatch(event)) { + logger.trace("The event is interesting."); + abortBuild(t, event); + if (t.isOnlyAbortRunningBuild(event)) { + logger.trace("Just aborting build based on event not scheduling new one."); + return; + } + notifyOnTriggered(t, event); + schedule(t, new GerritCause(event, t.isSilentMode()), event); } - notifyOnTriggered(t, event); - schedule(t, new GerritCause(event, t.isSilentMode()), event); } - } + }); } /** @@ -242,12 +260,12 @@ private void abortBuild(GerritTrigger t, GerritTriggeredEvent event) { return; } + BuildCancellationPolicy policy = t.getBuildCancellationPolicy(); // Per-trigger cancellation policy - if (t.getBuildCancellationPolicy() != null && t.getBuildCancellationPolicy().isEnabled()) { - logger.debug("Cancelling builds for event {} using trigger policy", changeBasedEvent); + if (policy != null && policy.isEnabled()) { listener.getMemory().cancelTriggeredJob( changeBasedEvent, - t.getBuildCancellationPolicy(), + policy, t, t.getJob()); } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritQueueListener.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritQueueListener.java index 9c021911c..01aa9f0cf 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritQueueListener.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritQueueListener.java @@ -11,6 +11,7 @@ import java.util.logging.Level; import java.util.logging.Logger; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; import com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.ToGerritRunListener; import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; @@ -25,6 +26,9 @@ public class GerritQueueListener extends QueueListener { @Override public void onLeft(LeftItem item) { if (item.isCancelled() && item.task instanceof Job) { + if (CoordinationModeFactory.get().getQueueCancellationStrategy().isLoadBalancedCancellation(item)) { + return; + } for (Cause cause : item.getCauses()) { if (cause instanceof GerritCause gerritCause && !gerritCause.isSilentMode()) { GerritTriggeredEvent event = gerritCause.getEvent(); diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelper.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelper.java new file mode 100644 index 000000000..539959216 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelper.java @@ -0,0 +1,76 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger; + +import hudson.model.Run; +import org.jenkinsci.plugins.workflow.flow.FlowExecution; +import org.jenkinsci.plugins.workflow.flow.FlowExecutionOwner; + +/** + * Optional helper that checks whether a Pipeline build's CPS execution has started. + *

+ * Isolated in its own class so that {@code workflow-api} classes are only loaded when + * the workflow plugin is present. Callers must guard with a {@code try/catch} for + * {@link NoClassDefFoundError} or check plugin availability before calling. + *

+ * Interrupting a Pipeline build during CPS initialisation (before {@link FlowExecution} is + * attached to its {@link FlowExecutionOwner}) has no effect — the interrupt flag is silently + * lost. This helper detects that window by checking whether {@link FlowExecutionOwner#getOrNull()} + * is still {@code null}. + *

+ * Why not also wait for {@code FlowExecution.getCurrentHeads()} to be non-empty: + * an earlier version of this check additionally required at least one {@link + * org.jenkinsci.plugins.workflow.graph.FlowNode} to exist. Repeated local trials (interrupting + * a build at the earliest possible moment {@code getOrNull()} became non-null, i.e. strictly + * before any head existed) showed the interrupt was honored (build result {@code ABORTED}) in + * every case, with the heads-non-empty moment consistently arriving several milliseconds later. + * So once {@code FlowExecution} is attached, the interrupt is already deliverable - checking + * heads added no observed protection, only extra polling delay. + */ +public final class PipelineAbortHelper { + + private PipelineAbortHelper() { } + + /** + * Returns {@code true} if {@code build} is a Pipeline build whose CPS execution has + * not yet started (i.e. it is still initialising and cannot yet receive an interrupt). + *

+ * Returns {@code false} for non-Pipeline builds or when the flow execution is + * unavailable, both of which are safe to interrupt immediately. + * + * @param build the build to check + * @return true if the build is a pipeline still initialising + */ + public static boolean isPipelineNotYetStarted(Run build) { + if (!(build instanceof FlowExecutionOwner.Executable)) { + return false; + } + FlowExecutionOwner owner = ((FlowExecutionOwner.Executable)build).asFlowExecutionOwner(); + if (owner == null) { + return false; + } + // Execution not yet attached — CPS is still initialising + return owner.getOrNull() == null; + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/BuildMemoryStorage.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/BuildMemoryStorage.java index 4df3cc722..6125b0359 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/BuildMemoryStorage.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/BuildMemoryStorage.java @@ -29,6 +29,7 @@ import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; import hudson.model.Job; import hudson.model.Run; +import jenkins.model.CauseOfInterruption; import edu.umd.cs.findbugs.annotations.CheckForNull; import edu.umd.cs.findbugs.annotations.NonNull; @@ -47,10 +48,6 @@ *

* Implementations are discovered via Jenkins Extension Points pattern using * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider}. - *

- * Design Note: This is an abstract class (not an interface) to allow - * adding concrete helper methods in the future without breaking existing implementations. - * This follows Jenkins plugin development best practices. * * @see com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider * @see com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory @@ -124,6 +121,23 @@ public abstract void retriggered(@NonNull GerritTriggeredEvent event, @NonNull J */ public abstract void cancelled(@NonNull GerritTriggeredEvent event, @NonNull Job project); + /** + * Marks a build as "cancelling" (cancellation intent) for an event. + *

+ * This is called when the cancellation policy decides a build should be cancelled, + * but before Jenkins actually processes the cancellation. The "cancelling" flag + * prevents the same build from being considered for cancellation again in future + * policy checks (avoids state accumulation). + *

+ * The actual "cancelled" flag is set later by {@link #cancelled} when Jenkins + * confirms the build/queue item was actually cancelled. + * + * @param event the event + * @param project the project being marked for cancellation + * @see #cancelled(GerritTriggeredEvent, Job) + */ + public abstract void setCancelling(@NonNull GerritTriggeredEvent event, @NonNull Job project); + /** * Removes the memory for an event. *

@@ -269,4 +283,59 @@ public abstract void setEntryUnsuccessfulMessage(@NonNull GerritTriggeredEvent e */ @NonNull public abstract Map getAllEvents(); + + /** + * Requests cross-replica abort for builds tracked for this event and project. + *

+ * In distributed deployments, this notifies other replicas to abort any + * matching builds running on their local executors. The cause of interruption + * is forwarded so the aborted build is annotated with the correct reason + * (e.g. {@link com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.AbandonedPatchsetInterruption} + * vs {@link com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.NewPatchSetInterruption}). + *

+ * This method is called from + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory#cancelOutdatedEvents} + * after {@link #setCancelling} has already marked the entry, so the cause is known at this point. + *

+ * The default implementation is a no-op — standalone Jenkins has no other replicas to notify. + * + * @param event the event whose builds should be aborted on remote replicas + * @param project the project being cancelled + * @param cause the cause of interruption + */ + public void requestCrossReplicaAbort(@NonNull GerritTriggeredEvent event, @NonNull Job project, + @NonNull CauseOfInterruption cause) { + // Default no-op: standalone mode has no other replicas to notify + } + + /** + * Checks if two events are logically equivalent for cancellation purposes. + *

+ * This method allows each storage implementation to define its own event equality + * semantics. Both modes use logical comparison, but differ in their approach: + *

    + *
  • Local mode: Delegates to {@code ChangeBasedEvent.equals()}, + * which compares {@code eventType}, {@code change}, and {@code patchSet} fields. + * This deliberately ignores timestamp so that deserialized event instances + * (e.g. {@code GerritCause.tEvent} loaded from disk) correctly match against + * in-memory events representing the same logical change.
    • + *
  • Distributed mode: Uses {@link com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.EventIdGenerator#generateEventId}, + * which produces a deterministic string key including the server-side timestamp. + * The timestamp is needed because the IMap key must uniquely identify each + * event reception across replicas; without it, two different events on the + * same patchset could collide.
    • + *
+ *

+ * Design rationale: Event equality semantics belong in the storage + * layer, not in business logic + * ({@link com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model.BuildMemory}). + * This respects the abstraction boundary and allows future coordination modes to + * define their own comparison strategy without modifying BuildMemory. + * + * @param event1 the first event + * @param event2 the second event + * @return true if the events are logically equivalent according to this storage + */ + public abstract boolean eventsMatch(@NonNull GerritTriggeredEvent event1, + @NonNull GerritTriggeredEvent event2); } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResult.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResult.java new file mode 100644 index 000000000..3cda285b4 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResult.java @@ -0,0 +1,75 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.spi; + +import edu.umd.cs.findbugs.annotations.NonNull; +import java.util.function.Consumer; + +/** + * Result of a claim attempt, allows chaining handlers for not-claimed and error cases. + *

+ * This interface supports a fluent API pattern for handling different outcomes + * of claim attempts in both event and notification claiming strategies. + *

+ * Usage Example: + * 

+ * strategy.withClaim(event, () -> {
+ *     processEvent(event);
+ * })
+ * .notClaimed(() -> {
+ *     logger.debug("Event already claimed, skipping");
+ * })
+ * .onError((ex) -> {
+ *     logger.error("Error processing event", ex);
+ * });
+ *
+ * + * @see EventClaimStrategy + * @see NotificationClaimStrategy + * @see ClaimResults + */ +public interface ClaimResult { + /** + * Handler called if the claim was not acquired (another instance already processing). + *

+ * This is optional - if not specified, nothing happens when the claim fails. + * + * @param notClaimed action to execute if claim failed + * @return this for chaining + */ + @NonNull + ClaimResult notClaimed(@NonNull Runnable notClaimed); + + /** + * Handler called if an exception occurs during claim processing. + *

+ * This is optional - if not specified, exceptions are silently ignored + * (though they may be logged by the implementation). + * + * @param onError action to execute on error (receives the exception) + * @return this for chaining + */ + @NonNull + ClaimResult onError(@NonNull Consumer onError); +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResults.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResults.java new file mode 100644 index 000000000..6adee2e99 --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/ClaimResults.java @@ -0,0 +1,177 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.spi; + +import edu.umd.cs.findbugs.annotations.NonNull; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.function.Consumer; + +/** + * Shared implementations of ClaimResult for use by all claim strategy implementations. + *

+ * This utility class eliminates code duplication across local and distributed claim strategies + * by providing reusable result implementations. All strategies (NotificationClaimStrategy and + * EventClaimStrategy, both local and Hazelcast modes) can use these shared implementations. + *

+ * Usage: + * 

+ * // In any claim strategy implementation:
+ * return ClaimResults.success();
+ * return ClaimResults.notClaimed();
+ * return ClaimResults.failed(exception);
+ *
+ * + * @see ClaimResult + */ +public final class ClaimResults { + + private static final Logger logger = LoggerFactory.getLogger(ClaimResults.class); + + /** + * Private constructor - utility class with only static methods. + */ + private ClaimResults() { + // Prevent instantiation + } + + /** + * Creates a successful claim result. + * Use when the claim was acquired and the action executed successfully. + * + * @return a ClaimResult indicating success + */ + @NonNull + public static ClaimResult success() { + return SuccessfulClaim.INSTANCE; + } + + /** + * Creates a not-claimed result. + * Use when the claim was not acquired (another instance already claimed it). + * + * @return a ClaimResult indicating not claimed + */ + @NonNull + public static ClaimResult notClaimed() { + return new NotClaimedResult(); + } + + /** + * Creates a failed claim result. + * Use when an exception occurred during claim processing. + * + * @param exception the exception that occurred + * @return a ClaimResult indicating failure + */ + @NonNull + public static ClaimResult failed(@NonNull Exception exception) { + return new FailedClaim(exception); + } + + /** + * Successful claim result - the claim was acquired and action executed. + * Singleton pattern for memory efficiency. + */ + private static class SuccessfulClaim implements ClaimResult { + + static final SuccessfulClaim INSTANCE = new SuccessfulClaim(); + + @Override + @NonNull + public ClaimResult notClaimed(@NonNull Runnable notClaimed) { + // Claim was successful, don't run notClaimed handler + return this; + } + + @Override + @NonNull + public ClaimResult onError(@NonNull Consumer onError) { + // No error occurred + return this; + } + } + + /** + * Not claimed result - another instance already claimed it. + */ + private static class NotClaimedResult implements ClaimResult { + + @Override + @NonNull + public ClaimResult notClaimed(@NonNull Runnable notClaimed) { + // Run the notClaimed handler + try { + notClaimed.run(); + } catch (Exception e) { + logger.error("Error in notClaimed handler", e); + } + return this; + } + + @Override + @NonNull + public ClaimResult onError(@NonNull Consumer onError) { + // No error occurred (just not claimed) + return this; + } + } + + /** + * Failed claim result - an exception occurred during processing. + */ + private static class FailedClaim implements ClaimResult { + + private final Exception exception; + + /** + * Constructor. + * + * @param exception the exception that occurred + */ + FailedClaim(@NonNull Exception exception) { + this.exception = exception; + } + + @Override + @NonNull + public ClaimResult notClaimed(@NonNull Runnable notClaimed) { + // Don't run notClaimed - this was an error, not a "not claimed" situation + return this; + } + + @Override + @NonNull + public ClaimResult onError(@NonNull Consumer onError) { + // Run the error handler + try { + onError.accept(exception); + } catch (Exception e) { + logger.error("Error in error handler", e); + } + return this; + } + } +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java index 893aa9c1d..e69b59a99 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java @@ -63,10 +63,35 @@ * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.LocalCoordinationProvider * @see BuildMemoryStorage * @see NotificationClaimStrategy + * @see EventClaimStrategy * @see com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory */ public abstract class CoordinationModeProvider implements ExtensionPoint { + /** + * System property to specify coordination mode. + */ + private static final String COORDINATION_MODE_PROPERTY = "gerrit.trigger.coordination.mode"; + + /** + * Gets the configured coordination mode. + *

+ * Helper method for providers: Centralizes the logic for determining + * which coordination mode is configured. This allows future evolution from system + * properties to UI configuration without changing provider implementations. + *

+ * Today: Reads from system property + * {@code gerrit.trigger.coordination.mode} (default: "local") + *

+ * Future: Can check UI configuration when providers add config pages + * (e.g., Redis/JDBC providers with connection settings in Jenkins UI) + * + * @return the configured mode name (e.g., "local", "hazelcast", "redis", "jdbc") + */ + public static String getConfiguredMode() { + return System.getProperty(COORDINATION_MODE_PROPERTY, "local"); + } + /** * Checks if this mode provider can create implementations in the current environment. * @@ -118,4 +143,86 @@ public abstract class CoordinationModeProvider implements ExtensionPoint { * @return a new NotificationClaimStrategy instance (non-null) */ public abstract NotificationClaimStrategy createClaimStrategy(); + + /** + * Creates a new EventClaimStrategy instance for this mode. + * + *

Called once during factory initialization after this provider is selected + * as the highest-priority available provider.

+ * + *

The EventClaimStrategy prevents duplicate build processing when multiple Jenkins + * instances receive the same Gerrit event in distributed scenarios. In local mode, this + * is a NO-OP (always claims). In distributed mode (e.g., Hazelcast), this uses + * distributed coordination to ensure only one instance processes each event.

+ * + *

Thread Safety: This method may be called from multiple threads during + * factory initialization (double-checked locking). Implementations should be stateless + * or properly synchronized.

+ * + * @return a new EventClaimStrategy instance (non-null) + */ + public abstract EventClaimStrategy createEventClaimStrategy(); + + /** + * Creates a new QueueCancellationStrategy instance for this mode. + * + *

Called once during factory initialization after this provider is selected + * as the highest-priority available provider.

+ * + *

The QueueCancellationStrategy determines whether a cancelled Jenkins queue item + * should be ignored because it was moved by the distributed load balancer rather than being + * cancelled by a user or a new patchset event. In local mode, this is a NO-OP (always + * returns false). In distributed mode (e.g., Hazelcast), this inspects the item for + * load-balancer markers.

+ * + *

Thread Safety: This method may be called from multiple threads during + * factory initialization (double-checked locking). Implementations should be stateless + * or properly synchronized.

+ * + * @return a new QueueCancellationStrategy instance (non-null) + */ + public abstract QueueCancellationStrategy createQueueCancellationStrategy(); + + /** + * Initializes this coordination mode provider. + * + *

Called during plugin startup (PluginImpl.start()) to initialize any resources + * needed by this provider. For example:

+ *
    + *
  • Local mode: no-op (no initialization needed)
    • + *
  • Hazelcast mode: initializes Hazelcast instance and cluster membership
    • + *
  • Redis mode: establishes connection pool
    • + *
  • JDBC mode: initializes database connection
    • + *
+ * + *

IMPORTANT: This method is called BEFORE the provider is selected by + * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory}. + * The provider must be fully initialized when {@link #isAvailable()} is called during + * provider discovery.

+ * + *

Error Handling: If initialization fails, implementations should throw an + * exception. The plugin will log the error and continue, allowing {@link #isAvailable()} + * to return false so a fallback provider can be selected.

+ * + * @throws Exception if initialization fails + */ + public abstract void initialize() throws Exception; + + /** + * Shuts down this coordination mode provider. + * + *

Called during plugin shutdown (PluginImpl.stop()) to release any resources + * held by this provider. For example:

+ *
    + *
  • Local mode: no-op (no resources to release)
    • + *
  • Hazelcast mode: shuts down Hazelcast instance gracefully
    • + *
  • Redis mode: closes connection pool
    • + *
  • JDBC mode: closes database connections
    • + *
+ * + *

Error Handling: Implementations should handle errors gracefully and + * not throw exceptions, as this is called during shutdown and exceptions cannot + * be meaningfully handled.

+ */ + public abstract void shutdown(); } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/EventClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/EventClaimStrategy.java new file mode 100644 index 000000000..256dcabad --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/EventClaimStrategy.java @@ -0,0 +1,93 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.spi; + +import com.sonymobile.tools.gerrit.gerritevents.dto.events.GerritTriggeredEvent; +import edu.umd.cs.findbugs.annotations.NonNull; + +/** + * Abstract base class for event claiming strategies in different deployment modes. + * Prevents duplicate build processing when multiple Jenkins instances receive + * the same Gerrit event in distributed scenarios. + * + *

This abstract class enables switching between local (standalone) and coordination modes:

+ *
    + *
  • Local mode: Always claims events - no coordination needed
    • + *
  • Hazelcast mode: Uses distributed coordination to ensure only one replica processes each event
    • + *
  • Future modes: Redis, JDBC, etc.
    • + *
+ * + *

Fluent API Pattern:

+ *

Uses fluent API pattern similar to Jenkins Queue and ACL for automatic resource management:

+ * 
+ * claimStrategy.withClaim(event, () -> {
+ *     processEvent(event);
+ * })
+ * .notClaimed(() -> {
+ *     logger.debug("Event already claimed, skipping");
+ * })
+ * .onError((ex) -> {
+ *     logger.error("Error processing event", ex);
+ * });
+ *
+ * + *

Implementations are discovered via the Extension Points pattern using + * {@link CoordinationModeProvider}.

+ * + * @see CoordinationModeProvider + */ +public abstract class EventClaimStrategy { + + /** + * Attempts to claim an event and execute the given action if successful. + * Automatically handles claim lifecycle (acquire + release). + * + *

The claim is automatically released after the action executes (success or failure), + * so implementations do not need manual cleanup code.

+ * + *

Usage Example:

+ * 
+     * claimStrategy.withClaim(event, () -> {
+     *     // This code runs only if claim was acquired
+     *     // Claim is automatically released after this block
+     *     processEvent(event);
+     * })
+     * .notClaimed(() -> {
+     *     // Optional: runs if claim was not acquired
+     *     logger.debug("Another instance is processing this event");
+     * })
+     * .onError((ex) -> {
+     *     // Optional: runs if an exception occurs during processing
+     *     logger.error("Failed to process event", ex);
+     * });
+     *
+ * + * @param event the Gerrit event to claim + * @param claimed action to execute if claim succeeds (runs with claim held, auto-released) + * @return ClaimResult for chaining notClaimed/onError handlers + * @see ClaimResults for shared result implementations + */ + @NonNull + public abstract ClaimResult withClaim(@NonNull GerritTriggeredEvent event, @NonNull Runnable claimed); +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/NotificationClaimStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/NotificationClaimStrategy.java index 0888fc613..88fa3dc55 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/NotificationClaimStrategy.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/NotificationClaimStrategy.java @@ -28,42 +28,105 @@ /** * Abstract base class for notification claiming strategies in different deployment modes. - * Subclasses handle the coordination of which Jenkins instance should send - * notifications to Gerrit in HA/HS deployments. + * Prevents duplicate notification sending when multiple Jenkins instances need to send + * feedback to Gerrit in distributed scenarios. * - *

This abstract class enables switching between local (standalone) and cluster modes:

+ *

This abstract class enables switching between local (standalone) and coordination modes:

*
    *
  • Local mode: Always claims notification rights - no coordination needed
    • - *
  • Cluster mode: Uses distributed coordination (e.g., Hazelcast) to ensure - * only one replica sends feedback to Gerrit per event
    • + *
  • Hazelcast mode: Uses distributed coordination to ensure only one replica sends feedback
    • + *
  • Future modes: Redis, JDBC, etc.
    • *
* + *

Fluent API Pattern:

+ *

Uses fluent API pattern similar to Jenkins Queue and ACL for automatic resource management:

+ * 
+ * claimStrategy.withClaim(event, "build-completed", () -> {
+ *     sendNotificationToGerrit(event, buildResult);
+ * })
+ * .notClaimed(() -> {
+ *     logger.debug("Another replica sent notification, skipping");
+ * })
+ * .onError((ex) -> {
+ *     logger.error("Failed to send notification", ex);
+ * });
+ *
+ * *

Implementations are discovered via the Extension Points pattern using - * {@link com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider}.

- *

- * Design Note: This is an abstract class (not an interface) to allow - * adding concrete helper methods in the future without breaking existing implementations. - * This follows Jenkins plugin development best practices. + * {@link CoordinationModeProvider}.

* - * @see com.sonyericsson.hudson.plugins.gerrit.trigger.spi.CoordinationModeProvider + * @see CoordinationModeProvider */ public abstract class NotificationClaimStrategy { /** - * Attempts to claim the right to send notification for an event. - * In local mode, always returns true. In cluster mode, uses distributed - * coordination to ensure only one replica sends the notification. + * Attempts to claim the right to send notification and execute the given action if successful. + * Automatically handles claim lifecycle (acquire + release). + * + *

The claim is automatically released after the action executes (success or failure), + * so implementations do not need manual cleanup code.

+ * + *

Notification Claim Scoping:

+ *
    + *
  • Per-job claims (jobIdentifier != null): Each job sends its own notification + * (e.g., build-started notifications where each job notifies independently)
    • + *
  • Per-event claims (jobIdentifier == null): One notification per event + * (e.g., build-completed where feedback is aggregated across all builds)
    • + *
+ * + *

Usage Example:

+ * 
+     * // Per-job claim (build-started)
+     * claimStrategy.withClaim(event, "build-started", jobName, () -> {
+     *     sendBuildStartedNotification(event, build);
+     * });
+     *
+     * // Per-event claim (build-completed)
+     * claimStrategy.withClaim(event, "build-completed", null, () -> {
+     *     sendAggregatedBuildCompletedNotification(event, allBuilds);
+     * });
+     *
+ * + * @param event the Gerrit event to claim notification rights for + * @param notificationType the type of notification (e.g., "build-started", "build-completed") + * @param jobIdentifier optional job identifier for per-job claims, null for per-event claims + * @param claimed action to execute if claim succeeds (runs with claim held, auto-released) + * @return ClaimResult for chaining notClaimed/onError handlers + * @see ClaimResults for shared result implementations + */ + @NonNull + public abstract ClaimResult withClaim(@NonNull GerritTriggeredEvent event, + @NonNull String notificationType, + String jobIdentifier, + @NonNull Runnable claimed); + + /** + * Convenience method without job identifier - creates per-event claim. * - * @param event the Gerrit event - * @return true if this instance should send the notification, false otherwise + * @param event the Gerrit event to claim notification rights for + * @param notificationType the type of notification (e.g., "build-started", "build-completed") + * @param claimed action to execute if claim succeeds + * @return ClaimResult for chaining notClaimed/onError handlers */ - public abstract boolean tryClaimNotificationRight(@NonNull GerritTriggeredEvent event); + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, + @NonNull String notificationType, + @NonNull Runnable claimed) { + return withClaim(event, notificationType, null, claimed); + } /** - * Releases the notification claim for an event. - * Called after notification is sent or on error to clean up resources. + * Legacy convenience method that uses a default notification type. + * + *

Deprecated: Use {@link #withClaim(GerritTriggeredEvent, String, String, Runnable)} instead + * to properly differentiate between notification types and scopes.

* - * @param event the Gerrit event + * @param event the Gerrit event to claim notification rights for + * @param claimed action to execute if claim succeeds + * @return ClaimResult for chaining notClaimed/onError handlers */ - public abstract void releaseNotificationRight(@NonNull GerritTriggeredEvent event); + @NonNull + public ClaimResult withClaim(@NonNull GerritTriggeredEvent event, @NonNull Runnable claimed) { + return withClaim(event, "default", null, claimed); + } } diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/QueueCancellationStrategy.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/QueueCancellationStrategy.java new file mode 100644 index 000000000..3b8a763fc --- /dev/null +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/QueueCancellationStrategy.java @@ -0,0 +1,54 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.spi; + +import hudson.model.Queue.LeftItem; +import edu.umd.cs.findbugs.annotations.NonNull; + +/** + * Abstract base class for queue cancellation strategies in different deployment modes. + * + *

Determines whether a cancelled Jenkins queue item should be ignored because it was + * cancelled by the distributed load balancer (i.e. migrated to another replica), not by a user + * or by a new patchset event.

+ * + *
    + *
  • Local mode: Always returns false — no load balancer present in standalone mode.
    • + *
  • Hazelcast mode: Inspects the item's actions and cause-of-blockage for + * load-balancer markers to avoid sending premature Gerrit feedback.
    • + *
+ * + * @see CoordinationModeProvider + */ +public abstract class QueueCancellationStrategy { + + /** + * Determines whether a cancelled queue item should be ignored because it was + * cancelled by the distributed load balancer moving it to another replica. + * + * @param item the queue item that left the queue as cancelled + * @return true if the cancellation is an distributed load-balancing operation and should be skipped + */ + public abstract boolean isLoadBalancedCancellation(@NonNull LeftItem item); +} diff --git a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/storage/LocalBuildMemoryStorage.java b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/storage/LocalBuildMemoryStorage.java index 0ea8e1a8f..cc6840209 100644 --- a/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/storage/LocalBuildMemoryStorage.java +++ b/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/storage/LocalBuildMemoryStorage.java @@ -116,11 +116,25 @@ public synchronized void cancelled(@NonNull GerritTriggeredEvent event, @NonNull MemoryImprint pb = getOrCreateMemoryImprint(event); pb.set(project); Entry entry = pb.getEntry(project); + // In local (single-replica) mode there are no load-balanced queue moves, so any + // cancelled() call is a genuine cancellation. Always mark buildCompleted=true so that + // isAllBuildsCompleted() can fire Gerrit feedback when the remaining jobs finish. entry.setCancelled(true); entry.setCancelling(false); entry.setBuildCompleted(true); } + @Override + public synchronized void setCancelling(@NonNull GerritTriggeredEvent event, @NonNull Job project) { + MemoryImprint pb = getMemoryImprint(event); + if (pb != null) { + Entry entry = pb.getEntry(project); + if (entry != null) { + entry.setCancelling(true); + } + } + } + /** * Gets or creates a MemoryImprint for the given event. * @@ -318,4 +332,13 @@ public synchronized Map getAllEvents() { // Return a copy to avoid concurrent modification issues return new TreeMap<>(memory); } + + @Override + public boolean eventsMatch(@NonNull GerritTriggeredEvent event1, @NonNull GerritTriggeredEvent event2) { + // Use logical equality so that deserialized event instances (e.g. GerritCause.tEvent + // loaded from disk) are correctly matched against in-memory events. + // Most trigger events extend ChangeBasedEvent, whose equals() compares eventType, + // change, and patchSet fields — all stable across serialization boundaries. + return event1.equals(event2); + } } diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EmbeddedHazelcastTestServer.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EmbeddedHazelcastTestServer.java new file mode 100644 index 000000000..f20f8b8c3 --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EmbeddedHazelcastTestServer.java @@ -0,0 +1,159 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.config.Config; +import com.hazelcast.config.JoinConfig; +import com.hazelcast.config.NetworkConfig; +import com.hazelcast.core.Hazelcast; +import com.hazelcast.core.HazelcastInstance; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.net.ServerSocket; + +/** + * Starts an embedded Hazelcast member (server) in the test JVM so that + * {@link HazelcastManager#initialize()} (which creates a client) has a server to connect to. + *

+ * The server binds to a free port on {@code localhost} chosen at {@link #start()} time, rather + * than a fixed port. Maven Surefire runs multiple forked JVMs concurrently ({@code forkCount} + * in the pom), and a fixed port would let two forks' embedded servers collide on the same + * address, corrupting each other's cluster state mid-test. Callers must read {@link #getPort()} + * after {@link #start()} and point the Hazelcast client at it (see + * {@link HazelcastServerTestListener}). + *

+ * This is required because since member mode was removed the plugin only creates a + * Hazelcast client, so tests must supply the server themselves. + */ +public final class EmbeddedHazelcastTestServer { + + private static final Logger logger = LoggerFactory.getLogger(EmbeddedHazelcastTestServer.class); + + private static volatile HazelcastInstance serverInstance = null; + private static volatile int port = -1; + private static final Object LOCK = new Object(); + + private EmbeddedHazelcastTestServer() { + // utility class + } + + /** + * Starts the embedded Hazelcast server if not already running, binding it to a free + * port on localhost. + * Idempotent — safe to call multiple times. + */ + public static void start() { + synchronized (LOCK) { + if (serverInstance != null && serverInstance.getLifecycleService().isRunning()) { + logger.debug("Embedded Hazelcast test server already running on port {}", port); + return; + } + + int chosenPort = findFreePort(); + logger.info("Starting embedded Hazelcast test server on localhost:{}", chosenPort); + try { + Config config = buildServerConfig(chosenPort); + serverInstance = Hazelcast.newHazelcastInstance(config); + port = chosenPort; + logger.info("Embedded Hazelcast test server started: {}", serverInstance.getName()); + } catch (Exception e) { + logger.error("Failed to start embedded Hazelcast test server", e); + throw new RuntimeException("Failed to start embedded Hazelcast test server", e); + } + } + } + + /** + * Returns the port the embedded server is bound to. + * + * @return the port, or -1 if the server has not been started yet + */ + public static int getPort() { + return port; + } + + private static int findFreePort() { + try (ServerSocket socket = new ServerSocket(0)) { + return socket.getLocalPort(); + } catch (IOException e) { + throw new RuntimeException("Failed to find a free port for the embedded Hazelcast test server", e); + } + } + + /** + * Shuts down the embedded Hazelcast server if running. + * Idempotent — safe to call multiple times. + */ + public static void stop() { + synchronized (LOCK) { + if (serverInstance == null) { + return; + } + try { + logger.info("Stopping embedded Hazelcast test server"); + serverInstance.shutdown(); + logger.info("Embedded Hazelcast test server stopped"); + } catch (Exception e) { + logger.warn("Error stopping embedded Hazelcast test server", e); + } finally { + serverInstance = null; + port = -1; + } + } + } + + /** + * Returns true if the server is running. + * + * @return true if running + */ + public static boolean isRunning() { + HazelcastInstance current = serverInstance; + return current != null && current.getLifecycleService().isRunning(); + } + + private static Config buildServerConfig(int testPort) { + Config config = new Config(); + + config.setClusterName(HazelcastConfig.DEFAULT_CLUSTER_NAME); + config.setInstanceName("gerrit-trigger-test-server"); + config.setProperty("hazelcast.logging.type", "slf4j"); + // Suppress startup banner in test output + config.setProperty("hazelcast.shutdownhook.enabled", "false"); + + NetworkConfig network = config.getNetworkConfig(); + network.setPort(testPort); + network.setPortAutoIncrement(false); + network.getInterfaces().setEnabled(true).addInterface("127.0.0.1"); + + // TCP-IP with only localhost — no multicast, no Kubernetes discovery + JoinConfig join = network.getJoin(); + join.getMulticastConfig().setEnabled(false); + join.getTcpIpConfig().setEnabled(true).addMember("127.0.0.1:" + testPort); + + return config; + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest.java new file mode 100644 index 000000000..1ddafdaba --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest.java @@ -0,0 +1,222 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; +import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.CompareType; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.GerritProject; +import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.Setup; +import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.TestUtils; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated; +import com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock; + +import java.util.Collections; +import hudson.model.FreeStyleBuild; +import hudson.model.FreeStyleProject; +import org.apache.sshd.server.SshServer; +import org.junit.After; +import org.junit.Before; +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExternalResource; +import org.jvnet.hudson.test.BuildWatcher; +import org.jvnet.hudson.test.JenkinsRule; +import org.jvnet.hudson.test.recipes.LocalData; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import static com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock.GERRIT_STREAM_EVENTS; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; + +/** + * Smoke test that exercises Hazelcast coordination mode during the default + * {@code mvn test} run, without requiring the {@code -Ptest-hazelcast} profile. + *

+ * {@link BuildCancellationHazelcastIntegrationTest} and its {@link HazelcastTestRule} only run + * under that profile, because the coordination mode system property has to be set before the JVM + * starts (see {@link HazelcastTestRule} for why setting it from an instance {@code @Rule} is too + * late). This test avoids that requirement by starting its own embedded Hazelcast server and + * setting the coordination properties from a {@code @ClassRule}, which is guaranteed by JUnit4 to + * run before any instance {@code @Rule} - including {@link JenkinsRule} - regardless of field + * declaration order. + *

+ * The point of this test is narrow: give normal CI a fast, self-contained signal that Hazelcast + * coordination mode still wires up and can trigger and complete a build end-to-end. It is not a + * replacement for the broader cancellation-race coverage in + * {@link BuildCancellationHazelcastIntegrationTest}, which remains opt-in due to its cost and + * flakiness surface. + */ +public class HazelcastCoordinationSmokeTest { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastCoordinationSmokeTest.class); + + private static final String COORDINATION_MODE_PROPERTY = "gerrit.trigger.coordination.mode"; + private static final String HAZELCAST_MODE = "hazelcast"; + // Generous: the in-JVM Hazelcast client's CPU contention slows the SSH handshake on busy agents. + private static final int SERVER_WAIT = 20000; + private static final int BUILD_TIMEOUT = 30000; + + /** + * Starts an embedded Hazelcast server and points the coordination system properties at it + * before {@link JenkinsRule} boots Jenkins - see the class Javadoc for why a {@code @ClassRule} + * is required here rather than {@code @Before}/{@code @Rule}. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JUnit ClassRule. + @ClassRule + public static final ExternalResource HAZELCAST_SERVER = new ExternalResource() { + + private String originalMode; + private String originalAddresses; + + @Override + protected void before() { + originalMode = System.getProperty(COORDINATION_MODE_PROPERTY); + originalAddresses = System.getProperty(HazelcastConfig.CLIENT_ADDRESSES_PROPERTY); + + EmbeddedHazelcastTestServer.start(); + System.setProperty(COORDINATION_MODE_PROPERTY, HAZELCAST_MODE); + System.setProperty(HazelcastConfig.CLIENT_ADDRESSES_PROPERTY, + "localhost:" + EmbeddedHazelcastTestServer.getPort()); + logger.info("Smoke test: embedded Hazelcast server ready on port {}", + EmbeddedHazelcastTestServer.getPort()); + } + + @Override + protected void after() { + // Jenkins (via JenkinsRule teardown, which nests inside this ClassRule and has + // already completed by the time this runs) shuts down every CoordinationModeProvider + // - including HazelcastCoordinationProvider, which calls HazelcastManager.shutdown() + // itself. Calling it again here would race the plugin's own shutdown and intermittently + // throw HazelcastClientNotActiveException, so only the server this class started is + // stopped here. + EmbeddedHazelcastTestServer.stop(); + restoreProperty(COORDINATION_MODE_PROPERTY, originalMode); + restoreProperty(HazelcastConfig.CLIENT_ADDRESSES_PROPERTY, originalAddresses); + } + + private void restoreProperty(String key, String value) { + if (value == null) { + System.clearProperty(key); + } else { + System.setProperty(key, value); + } + } + }; + + /** + * An instance of Jenkins Rule. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JenkinsRule. + @Rule + public final JenkinsRule jenkins = new JenkinsRule(); + + /** + * Outputs build logs to std out. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JenkinsRule. + @Rule + public final BuildWatcher watcher = new BuildWatcher(); + + private SshServer sshd; + private SshdServerMock serverMock; + private GerritServer gerritServer; + + /** + * Sets up the SSH server mock before each test. + * + * @throws Exception if setup fails + */ + @Before + public void setUp() throws Exception { + SshdServerMock.generateKeyPair(); + serverMock = new SshdServerMock(); + sshd = SshdServerMock.startServer(serverMock); + serverMock.returnCommandFor("gerrit ls-projects", SshdServerMock.EofCommandMock.class); + serverMock.returnCommandFor(GERRIT_STREAM_EVENTS, SshdServerMock.CommandMock.class); + serverMock.returnCommandFor("gerrit review.*", SshdServerMock.EofCommandMock.class); + serverMock.returnCommandFor("gerrit version", SshdServerMock.SendVersionCommand.class); + gerritServer = PluginImpl.getFirstServer_(); + if (gerritServer != null) { + SshdServerMock.configureFor(sshd, gerritServer, true); + } + } + + /** + * Tears down the SSH server and clears Hazelcast state. + * + * @throws Exception if teardown fails + */ + @After + public void tearDown() throws Exception { + HazelcastTestHelper.clearAllMaps(); + if (sshd != null) { + sshd.stop(true); + sshd = null; + } + } + + /** + * Verifies that Hazelcast coordination mode is actually active, and that a Gerrit event + * triggers and completes a build end-to-end through Hazelcast-backed build memory, event + * claiming, and notification claiming - not just the mode selection. + * + * @throws Exception if unexpected errors appear. + */ + @Test + @LocalData("common") + public void testHazelcastCoordinationModeTriggersAndCompletesBuild() throws Exception { + CoordinationModeFactory factory = CoordinationModeFactory.get(); + String storageClass = factory.getStorage().getClass().getSimpleName(); + assertEquals("Expected Hazelcast storage - falling back to local mode would defeat " + + "the point of this smoke test", "HazelcastBuildMemoryStorage", storageClass); + assertNotNull("Expected a selected coordination mode", factory.getSelectedMode()); + assertEquals("Expected Hazelcast mode", "Hazelcast (Distributed)", + factory.getSelectedMode().getModeName()); + + FreeStyleProject project = jenkins.createFreeStyleProject(); + GerritTrigger trigger = Setup.createDefaultTrigger(project); + trigger.setGerritProjects(Collections.singletonList( + new GerritProject(CompareType.ANT, "**", + Collections.singletonList(new Branch(CompareType.ANT, "**")), + null, null, null, false))); + project.addTrigger(trigger); + trigger.start(project, false); + + serverMock.waitForCommand(GERRIT_STREAM_EVENTS, SERVER_WAIT); + + PatchsetCreated patchset = Setup.createPatchsetCreated(); + gerritServer.triggerEvent(patchset); + + TestUtils.waitForBuilds(project, 1, BUILD_TIMEOUT); + FreeStyleBuild build = project.getLastBuild(); + assertNotNull("Build should have been triggered via Hazelcast-backed coordination", build); + jenkins.assertBuildStatusSuccess(build); + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastServerTestListener.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastServerTestListener.java new file mode 100644 index 000000000..0abbd2bae --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastServerTestListener.java @@ -0,0 +1,81 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import org.junit.platform.launcher.TestExecutionListener; +import org.junit.platform.launcher.TestPlan; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * JUnit Platform {@link TestExecutionListener} that starts an embedded Hazelcast server + * before any test in the forked JVM runs. + *

+ * Activated by Maven Surefire when the {@code test-hazelcast} profile is active. This listener + * is discovered via the {@code META-INF/services/org.junit.platform.launcher.TestExecutionListener} + * service file, which is the correct discovery mechanism for the JUnit Platform provider (surefire + * 3.x) — unlike the JUnit 4 {@code } property, which is not invoked for JUnit 5 tests. + *

+ * Starting the server here ensures that when Jenkins initialises and + * {@code PluginImpl.gerritStart()} calls {@link HazelcastManager#initialize()} (which creates a + * Hazelcast client), the server is already listening. Without this, the client hangs for + * several minutes trying to reach a non-existent server. + *

+ * The server binds to a free port chosen per forked JVM (see {@link EmbeddedHazelcastTestServer}), + * so this listener also points the client at that port via + * {@link HazelcastConfig#CLIENT_ADDRESSES_PROPERTY}. Using a fixed, shared port here would let + * concurrent Surefire forks collide on the same address and corrupt each other's cluster state. + * + * @see EmbeddedHazelcastTestServer + */ +public class HazelcastServerTestListener implements TestExecutionListener { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastServerTestListener.class); + + private static final String COORDINATION_MODE_PROPERTY = "gerrit.trigger.coordination.mode"; + private static final String HAZELCAST_MODE = "hazelcast"; + + @Override + public void testPlanExecutionStarted(TestPlan testPlan) { + String mode = System.getProperty(COORDINATION_MODE_PROPERTY); + if (!HAZELCAST_MODE.equalsIgnoreCase(mode)) { + logger.debug("Coordination mode is '{}', skipping embedded Hazelcast server start", mode); + return; + } + logger.info("=== Starting embedded Hazelcast test server (coordination mode: {}) ===", mode); + EmbeddedHazelcastTestServer.start(); + String address = "localhost:" + EmbeddedHazelcastTestServer.getPort(); + System.setProperty(HazelcastConfig.CLIENT_ADDRESSES_PROPERTY, address); + logger.info("=== Embedded Hazelcast test server ready on {} ===", address); + } + + @Override + public void testPlanExecutionFinished(TestPlan testPlan) { + if (EmbeddedHazelcastTestServer.isRunning()) { + logger.info("=== Stopping embedded Hazelcast test server ==="); + EmbeddedHazelcastTestServer.stop(); + logger.info("=== Embedded Hazelcast test server stopped ==="); + } + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestHelper.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestHelper.java new file mode 100644 index 000000000..dd1dd12fe --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestHelper.java @@ -0,0 +1,125 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import com.hazelcast.core.HazelcastInstance; +import com.hazelcast.map.IMap; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Test helper utilities for Hazelcast tests. + *

+ * Provides methods to clear Hazelcast state between tests, preventing + * state pollution when tests run in sequence with a shared Hazelcast instance. + * + */ +public final class HazelcastTestHelper { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastTestHelper.class); + + /** + * The name of the BuildMemory distributed map in Hazelcast. + */ + private static final String BUILD_MEMORY_MAP_NAME = "gerrit-trigger-build-memory"; + + /** + * Private constructor to prevent instantiation. + */ + private HazelcastTestHelper() { + // Utility class + } + + /** + * Clears all Hazelcast distributed maps used by Gerrit Trigger. + *

+ * This should be called in @After methods to ensure clean state between tests. + * Safe to call even if Hazelcast is not initialized. + */ + public static void clearAllMaps() { + if (!HazelcastInstanceProvider.isInitialized()) { + logger.debug("Hazelcast not initialized, nothing to clear"); + return; + } + + try { + HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + + // Clear BuildMemory map + IMap buildMemoryMap = instance.getMap(BUILD_MEMORY_MAP_NAME); + int entries = buildMemoryMap.size(); + buildMemoryMap.clear(); + logger.debug("Cleared {} entries from BuildMemory map", entries); + + } catch (Exception e) { + logger.warn("Error clearing Hazelcast maps (test cleanup)", e); + // Don't fail tests on cleanup errors + } + } + + /** + * Clears only the BuildMemory distributed map. + *

+ * Useful when you want to clear build state but keep other state intact. + * Safe to call even if Hazelcast is not initialized. + */ + public static void clearBuildMemory() { + if (!HazelcastInstanceProvider.isInitialized()) { + logger.debug("Hazelcast not initialized, nothing to clear"); + return; + } + + try { + HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + IMap buildMemoryMap = instance.getMap(BUILD_MEMORY_MAP_NAME); + int entries = buildMemoryMap.size(); + buildMemoryMap.clear(); + logger.debug("Cleared {} entries from BuildMemory map", entries); + } catch (Exception e) { + logger.warn("Error clearing BuildMemory map (test cleanup)", e); + // Don't fail tests on cleanup errors + } + } + + /** + * Gets the number of entries in the BuildMemory map. + * Useful for debugging test failures. + * + * @return number of entries, or 0 if Hazelcast not initialized + */ + public static int getBuildMemorySize() { + if (!HazelcastInstanceProvider.isInitialized()) { + return 0; + } + + try { + HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + IMap buildMemoryMap = instance.getMap(BUILD_MEMORY_MAP_NAME); + return buildMemoryMap.size(); + } catch (Exception e) { + logger.warn("Error getting BuildMemory size", e); + return 0; + } + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestListener.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestListener.java new file mode 100644 index 000000000..9b2431411 --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestListener.java @@ -0,0 +1,134 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import org.junit.runner.Description; +import org.junit.runner.Result; +import org.junit.runner.notification.RunListener; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * JUnit test listener that initializes Hazelcast once for the entire test suite. + *

+ * This listener is automatically invoked by Maven Surefire when the test-hazelcast + * profile is active. It ensures Hazelcast is initialized before any tests run, + * allowing all tests to run in distributed coordination mode. + *

+ * Configuration: Activated via Maven profile with test-hazelcast profile. + * + */ +public class HazelcastTestListener extends RunListener { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastTestListener.class); + + private static final String COORDINATION_MODE_PROPERTY = "gerrit.trigger.coordination.mode"; + private static final String HAZELCAST_MODE = "hazelcast"; + + private static boolean initialized = false; + private static boolean shouldInitialize = false; + + /** + * Called once before any tests run. + * Initializes Hazelcast if coordination mode is set to hazelcast. + * + * @param description test run description + */ + @Override + public void testRunStarted(Description description) { + String mode = System.getProperty(COORDINATION_MODE_PROPERTY); + + if (HAZELCAST_MODE.equalsIgnoreCase(mode)) { + logger.info("=== Hazelcast Test Suite Initialization ==="); + logger.info("Coordination mode property: {}", mode); + + // Ensure the embedded server is running before the client tries to connect. + // HazelcastServerTestListener (JUnit Platform SPI) starts it for JUnit 5 tests, + // but this guard covers any JUnit 4 fork where the platform listener did not fire. + EmbeddedHazelcastTestServer.start(); + + if (!HazelcastManager.isInitialized()) { + try { + logger.info("Initializing Hazelcast for test suite..."); + com.hazelcast.core.HazelcastInstance instance = HazelcastManager.initialize(); + + if (instance != null) { + initialized = true; + shouldInitialize = true; + logger.info("Hazelcast initialized successfully for test suite"); + logger.info("Cluster: {}", + HazelcastInstanceProvider.getInstance().getConfig().getClusterName()); + logger.info("Instance: {}", + HazelcastInstanceProvider.getInstance().getName()); + logger.info("Members: {}", + HazelcastInstanceProvider.getInstance().getCluster().getMembers().size()); + } else { + logger.error("Failed to initialize Hazelcast for test suite"); + } + } catch (Exception e) { + logger.error("Exception initializing Hazelcast for test suite", e); + } + } else { + logger.info("Hazelcast already initialized"); + initialized = true; + } + + logger.info("=== Hazelcast Test Suite Initialization Complete ==="); + } else { + logger.debug("Coordination mode is '{}', Hazelcast listener will not initialize", mode); + } + } + + /** + * Called once after all tests complete. + * Shuts down Hazelcast if it was initialized by this listener. + * + * @param result test run result + */ + @Override + public void testRunFinished(Result result) { + if (shouldInitialize && initialized) { + logger.info("=== Hazelcast Test Suite Cleanup ==="); + try { + logger.info("Shutting down Hazelcast..."); + HazelcastManager.shutdown(); + initialized = false; + shouldInitialize = false; + logger.info("Hazelcast shutdown complete"); + } catch (Exception e) { + logger.error("Error shutting down Hazelcast", e); + } + logger.info("=== Hazelcast Test Suite Cleanup Complete ==="); + } + } + + /** + * Checks if Hazelcast was initialized by this listener. + * + * @return true if initialized + */ + public static boolean isInitialized() { + return initialized; + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestRule.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestRule.java new file mode 100644 index 000000000..38fae9dba --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestRule.java @@ -0,0 +1,200 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast; + +import org.junit.rules.ExternalResource; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * JUnit rule for initializing and cleaning up Hazelcast for tests. + *

+ * This rule handles the complete lifecycle of Hazelcast for integration tests: + *

    + *
  • Sets coordination mode system property
    • + *
  • Initializes Hazelcast embedded instance
    • + *
  • Verifies Hazelcast is ready
    • + *
  • Cleans up after tests complete
    • + *
+ * + *

Usage:

+ * 
{@code
+ * public class MyHazelcastTest {
+ *     @Rule
+ *     public HazelcastTestRule hazelcast = new HazelcastTestRule();
+ *
+ *     @Rule
+ *     public JenkinsRule jenkins = new JenkinsRule();
+ *
+ *     @Test
+ *     public void testWithHazelcast() {
+ *         // Test will run with Hazelcast coordination mode active
+ *     }
+ * }
+ * }
+ * + *

Important: This rule must be declared BEFORE JenkinsRule + * in the test class to ensure Hazelcast is initialized before Jenkins starts.

+ * + */ +public class HazelcastTestRule extends ExternalResource { + + private static final Logger logger = LoggerFactory.getLogger(HazelcastTestRule.class); + + private static final String COORDINATION_MODE_PROPERTY = "gerrit.trigger.coordination.mode"; + private static final String HAZELCAST_MODE = "hazelcast"; + + private String originalModeValue; + private boolean initializedByThisRule = false; + + /** + * Sets up Hazelcast before the test runs. + *

+ * This method: + *

    + *
  1. Saves the original coordination mode property
    2. + *
  2. Sets coordination mode to "hazelcast"
    3. + *
  3. Initializes Hazelcast embedded instance
    4. + *
  4. Verifies initialization succeeded
    5. + *
+ * + * @throws Exception if Hazelcast initialization fails + */ + @Override + protected void before() throws Exception { + logger.info("=== Hazelcast Test Setup START ==="); + + // Skip if not running with -Ptest-hazelcast. CoordinationModeFactory is initialised + // by JenkinsRule (which runs before this rule despite field order), so Hazelcast mode + // can only be active if the property was set at JVM startup via the Maven profile. + String preconfiguredMode = System.getProperty(COORDINATION_MODE_PROPERTY); + org.junit.Assume.assumeTrue( + "Skipping Hazelcast integration test - run with -Ptest-hazelcast to enable", + HAZELCAST_MODE.equalsIgnoreCase(preconfiguredMode)); + + // Save original property value + originalModeValue = preconfiguredMode; + logger.info("Original coordination mode: {}", originalModeValue); + + // Set coordination mode to hazelcast + System.setProperty(COORDINATION_MODE_PROPERTY, HAZELCAST_MODE); + logger.info("Set coordination mode to: {}", HAZELCAST_MODE); + + // Initialize Hazelcast if not already initialized + if (!HazelcastManager.isInitialized()) { + logger.info("Initializing Hazelcast for test..."); + com.hazelcast.core.HazelcastInstance instance = HazelcastManager.initialize(); + + if (instance == null) { + throw new IllegalStateException("Failed to initialize Hazelcast for test"); + } + + initializedByThisRule = true; + logger.info("Hazelcast initialized successfully"); + } else { + logger.info("Hazelcast already initialized, reusing existing instance"); + initializedByThisRule = false; + } + + // Verify Hazelcast is available + if (!HazelcastInstanceProvider.isInitialized()) { + throw new IllegalStateException("Hazelcast instance not available after initialization"); + } + + // Clear notification flags map to avoid test pollution within this test's scope + // Note: This only helps tests that use HazelcastTestRule explicitly. In the full test suite, + // tests using the same mock events (e.g., Setup.createPatchsetCreated()) may still collide + // if run concurrently, since they generate identical event IDs. Maven's retry mechanism + // handles these transient failures. + try { + com.hazelcast.core.HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + com.hazelcast.map.IMap notificationFlags = + instance.getMap("gerrit-trigger-notification-flags"); + int clearedCount = notificationFlags.size(); + notificationFlags.clear(); + if (clearedCount > 0) { + logger.info("Cleared {} notification claim(s) before test", clearedCount); + } + } catch (Exception e) { + logger.warn("Failed to clear notification flags map", e); + } + + logger.info("Hazelcast instance: {}", HazelcastInstanceProvider.getInstance().getName()); + logger.info("Cluster size: {}", + HazelcastInstanceProvider.getInstance().getCluster().getMembers().size()); + logger.info("=== Hazelcast Test Setup COMPLETE ==="); + } + + /** + * Cleans up Hazelcast after the test completes. + *

+ * This method: + *

    + *
  1. Shuts down Hazelcast (if initialized by this rule)
    2. + *
  2. Restores original coordination mode property
    3. + *
+ *

+ * Cleanup is best-effort - errors are logged but don't fail the test. + */ + @Override + protected void after() { + logger.info("=== Hazelcast Test Cleanup START ==="); + + try { + // Only shutdown if we initialized it + if (initializedByThisRule && HazelcastManager.isInitialized()) { + logger.info("Shutting down Hazelcast..."); + HazelcastManager.shutdown(); + logger.info("Hazelcast shutdown complete"); + } else if (!initializedByThisRule) { + logger.info("Hazelcast was not initialized by this rule, leaving it running"); + } else { + logger.info("Hazelcast already shut down"); + } + } catch (Exception e) { + logger.error("Error shutting down Hazelcast (test cleanup)", e); + // Don't fail the test on cleanup errors + } + + // Restore original property + if (originalModeValue == null) { + System.clearProperty(COORDINATION_MODE_PROPERTY); + logger.info("Cleared coordination mode property"); + } else { + System.setProperty(COORDINATION_MODE_PROPERTY, originalModeValue); + logger.info("Restored coordination mode to: {}", originalModeValue); + } + + logger.info("=== Hazelcast Test Cleanup COMPLETE ==="); + } + + /** + * Checks if Hazelcast was initialized by this rule. + * + * @return true if this rule initialized Hazelcast + */ + public boolean wasInitializedByThisRule() { + return initializedByThisRule; + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/extensions/GerritTriggeredBuildListenerTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/extensions/GerritTriggeredBuildListenerTest.java index 23f62f4cf..dcef58ef3 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/extensions/GerritTriggeredBuildListenerTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/extensions/GerritTriggeredBuildListenerTest.java @@ -27,6 +27,8 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastInstanceProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastManager; import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.DuplicatesUtil; import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.Setup; import com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock; @@ -78,6 +80,17 @@ public class GerritTriggeredBuildListenerTest { * * @throws Exception throw if so. */ + @Before + public void clearHazelcastFlags() { + // Clear Hazelcast notification flags between test methods to prevent claim key collision. + // Test events use a fixed eventCreatedOn timestamp, producing identical claim keys across tests. + if (HazelcastManager.isInitialized()) { + HazelcastInstanceProvider.getInstance() + .getMap("gerrit-trigger-notification-flags") + .clear(); + } + } + @Before public void setUp() throws Exception { SshdServerMock.generateKeyPair(); @@ -123,7 +136,7 @@ public void testListenTriggeredBuild() throws Exception { server.waitForCommand(GERRIT_STREAM_EVENTS, 2000); gerritServer.triggerEvent(Setup.createPatchsetCreated()); - assertTrue("Time out", buildListenerLatch.await(15, TimeUnit.SECONDS)); + assertTrue("Time out", buildListenerLatch.await(30, TimeUnit.SECONDS)); } /** diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/rest/BuildCompletedRestCommandJobHudsonTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/rest/BuildCompletedRestCommandJobHudsonTest.java index a1a5d7ec4..bce10b564 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/rest/BuildCompletedRestCommandJobHudsonTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/rest/BuildCompletedRestCommandJobHudsonTest.java @@ -27,6 +27,8 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastInstanceProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastManager; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.CompareType; @@ -77,6 +79,13 @@ public class BuildCompletedRestCommandJobHudsonTest { @Before public void unlockInstance() throws Exception { Setup.unLock(j); + // Clear Hazelcast notification flags between test methods to prevent claim key collision. + // Test events use a fixed eventCreatedOn timestamp, producing identical claim keys across tests. + if (HazelcastManager.isInitialized()) { + HazelcastInstanceProvider.getInstance() + .getMap("gerrit-trigger-notification-flags") + .clear(); + } } /** @@ -193,8 +202,12 @@ public String getUrlName() { * @throws IOException if so. */ public void doDynamic(StaplerRequest2 request, StaplerResponse2 response) throws IOException { - lastPath = request.getRestOfPath(); - lastContent = IOUtils.toString(request.getReader()); + String path = request.getRestOfPath(); + // Only track review calls, not plugin-availability checks that may race with the assertion + if (!path.startsWith("/plugins/")) { + lastPath = path; + lastContent = IOUtils.toString(request.getReader()); + } response.setContentType("application/json"); PrintWriter writer = response.getWriter(); diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintTest.java index 65cdfeb41..3aa1290d2 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/MemoryImprintTest.java @@ -25,6 +25,7 @@ package com.sonyericsson.hudson.plugins.gerrit.trigger.gerritnotifier.model; import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.Setup; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated; import hudson.model.AbstractBuild; import hudson.model.AbstractProject; import hudson.model.Job; @@ -37,6 +38,8 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.ArgumentMatchers.same; import static org.mockito.Mockito.doReturn; @@ -51,6 +54,10 @@ */ public class MemoryImprintTest { + private static final long TRIGGERED_TS = 1000L; + private static final long STARTED_TS = 2000L; + private static final long COMPLETED_TS = 3000L; + private static int nameCount = 0; private AbstractProject project; private AbstractBuild build; @@ -174,4 +181,64 @@ public void testResetTwoPreviousBuilds() { assertNull(imprint.getEntries()[1].getBuild()); assertFalse(imprint.getEntries()[0].isBuildCompleted()); } + + /** + * Round-trips an {@link EntryData} through {@link BuildMemory.MemoryImprint.Entry#fromEntryData(EntryData)} + * and {@link BuildMemory.MemoryImprint.Entry#toEntryData()} and asserts every field is preserved. + *

+ * Notably the timestamps are carried verbatim: unlike the {@code setBuild}/{@code setBuildCompleted} + * setters, the data round-trip must not re-stamp them with the current time. + */ + @Test + public void testEntryDataRoundTrip() { + EntryData data = new EntryData(); + data.setProjectFullName("some/project"); + data.setBuildId("42"); + data.setBuildCompleted(true); + data.setCancelling(true); + data.setCancelled(true); + data.setQueueLeft(true); + data.setCustomUrl("http://example.test/custom"); + data.setUnsuccessfulMessage("nope"); + data.setTriggeredTimestamp(TRIGGERED_TS); + data.setStartedTimestamp(STARTED_TS); + data.setCompletedTimestamp(COMPLETED_TS); + + EntryData roundTripped = BuildMemory.MemoryImprint.Entry.fromEntryData(data).toEntryData(); + + assertEquals("some/project", roundTripped.getProjectFullName()); + assertEquals("42", roundTripped.getBuildId()); + assertTrue(roundTripped.isBuildCompleted()); + assertTrue(roundTripped.isCancelling()); + assertTrue(roundTripped.isCancelled()); + assertTrue(roundTripped.isQueueLeft()); + assertEquals("http://example.test/custom", roundTripped.getCustomUrl()); + assertEquals("nope", roundTripped.getUnsuccessfulMessage()); + assertEquals(TRIGGERED_TS, roundTripped.getTriggeredTimestamp()); + assertEquals(Long.valueOf(STARTED_TS), roundTripped.getStartedTimestamp()); + assertEquals(Long.valueOf(COMPLETED_TS), roundTripped.getCompletedTimestamp()); + } + + /** + * Round-trips a {@link BuildMemory.MemoryImprint} through {@link BuildMemory.MemoryImprint#toData()} + * and {@link BuildMemory.MemoryImprint#fromData(MemoryImprintData)} and asserts the event and entries + * survive intact, resolving back to the same Jenkins objects. + */ + @Test + public void testMemoryImprintRoundTrip() { + PatchsetCreated event = Setup.createPatchsetCreated(); + BuildMemory.MemoryImprint imprint = new BuildMemory.MemoryImprint(event); + imprint.set(project, build, true); + imprint.getEntry(project).setCustomUrl("http://example.test/x"); + + BuildMemory.MemoryImprint restored = BuildMemory.MemoryImprint.fromData(imprint.toData()); + + assertSame(event, restored.getEvent()); + assertEquals(1, restored.getEntries().length); + BuildMemory.MemoryImprint.Entry entry = restored.getEntries()[0]; + assertEquals(project, entry.getProject()); + assertEquals(build, entry.getBuild()); + assertTrue(entry.isBuildCompleted()); + assertEquals("http://example.test/x", entry.getCustomUrl()); + } } diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest.java new file mode 100644 index 000000000..ad9d5974e --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest.java @@ -0,0 +1,371 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger; + +import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; +import com.sonyericsson.hudson.plugins.gerrit.trigger.Messages; +import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.CoordinationModeFactory; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestRule; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.BuildCancellationPolicy; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.CompareType; +import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.GerritProject; +import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.Setup; +import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.TestUtils; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.ChangeAbandoned; +import com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated; +import com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock; + +import java.util.Collections; +import hudson.model.FreeStyleBuild; +import hudson.model.FreeStyleProject; +import hudson.model.Result; +import org.apache.sshd.server.SshServer; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.jvnet.hudson.test.BuildWatcher; +import org.jvnet.hudson.test.JenkinsRule; +import org.jvnet.hudson.test.SleepBuilder; +import org.jvnet.hudson.test.recipes.LocalData; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import static com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock.GERRIT_STREAM_EVENTS; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +/** + * Integration tests for build cancellation in Hazelcast distributed mode. + *

+ * These tests verify that build cancellation works correctly when using + * Hazelcast-backed BuildMemoryStorage instead of local TreeMap storage. + *

+ * This is critical for distributed scenarios where multiple Jenkins + * instances share state via Hazelcast. + * + */ +public class BuildCancellationHazelcastIntegrationTest { + + private static final Logger logger = LoggerFactory.getLogger( + BuildCancellationHazelcastIntegrationTest.class); + + /** + * Hazelcast test infrastructure - MUST be declared before JenkinsRule. + * This ensures Hazelcast is initialized before Jenkins starts. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JUnit Rule. + @Rule + public final HazelcastTestRule hazelcast = new HazelcastTestRule(); + + /** + * An instance of Jenkins Rule. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JenkinsRule. + @Rule + public final JenkinsRule jenkins = new JenkinsRule(); + + /** + * Outputs build logs to std out. + */ + //CS IGNORE VisibilityModifier FOR NEXT 2 LINES. REASON: JenkinsRule. + @Rule + public final BuildWatcher watcher = new BuildWatcher(); + + private SshServer sshd; + private SshdServerMock serverMock; + private GerritServer gerritServer; + private static final int BUILD_TIMEOUT = 30000; + private static final int SERVER_WAIT = 2000; + private static final int SLEEP_10_SEC = 10000; + private static final int SLEEP_HALF_SEC = 500; + + /** + * Sets up the SSH server mock before each test. + * + * @throws Exception if setup fails + */ + @Before + public void setUp() throws Exception { + SshdServerMock.generateKeyPair(); + serverMock = new SshdServerMock(); + sshd = SshdServerMock.startServer(serverMock); + serverMock.returnCommandFor("gerrit ls-projects", SshdServerMock.EofCommandMock.class); + serverMock.returnCommandFor(GERRIT_STREAM_EVENTS, SshdServerMock.CommandMock.class); + serverMock.returnCommandFor("gerrit review.*", SshdServerMock.EofCommandMock.class); + serverMock.returnCommandFor("gerrit version", SshdServerMock.SendVersionCommand.class); + gerritServer = PluginImpl.getFirstServer_(); + if (gerritServer != null) { + SshdServerMock.configureFor(sshd, gerritServer, true); + } + + // Verify Hazelcast mode is active + verifyHazelcastMode(); + } + + /** + * Tears down the SSH server. + * + * @throws Exception if teardown fails + */ + @After + public void tearDown() throws Exception { + // Clear Hazelcast state to prevent pollution between tests + HazelcastTestHelper.clearAllMaps(); + + if (sshd != null) { + sshd.stop(true); + sshd = null; + } + } + + /** + * Verifies that Hazelcast coordination mode is actually active. + * This prevents false positives from tests running in local mode. + */ + private void verifyHazelcastMode() { + CoordinationModeFactory factory = CoordinationModeFactory.get(); + + // Trigger mode discovery by accessing storage + String storageClass = factory.getStorage().getClass().getSimpleName(); + + // Now get the selected mode (will not be null after storage access) + String modeName; + if (factory.getSelectedMode() != null) { + modeName = factory.getSelectedMode().getModeName(); + } else { + modeName = "UNKNOWN"; + } + + logger.info("=== COORDINATION MODE VERIFICATION ==="); + logger.info("Mode: {}", modeName); + logger.info("Storage: {}", storageClass); + logger.info("======================================"); + + assertEquals("Expected Hazelcast storage", "HazelcastBuildMemoryStorage", storageClass); + assertEquals("Expected Hazelcast mode", "Hazelcast (Distributed)", modeName); + } + + /** + * Waits for a build to start (but not necessarily complete). + * + * @param project the project to check + * @param timeoutMs the timeout in milliseconds + * @throws InterruptedException if interrupted while sleeping + */ + private void waitForBuildToStart(FreeStyleProject project, long timeoutMs) throws InterruptedException { + long startTime = System.currentTimeMillis(); + while (project.getLastBuild() == null || !project.getLastBuild().isBuilding()) { + if (System.currentTimeMillis() - startTime >= timeoutMs) { + throw new RuntimeException("Timeout waiting for build to start!"); + } + Thread.sleep(SLEEP_HALF_SEC); + } + } + + /** + * Verifies that a build was interrupted with the expected cause by checking the build log. + * + * @param build the build to check + * @param expectedMessage the expected interruption message + * @throws Exception if unable to read build log + */ + private void assertInterruptionCause(FreeStyleBuild build, String expectedMessage) throws Exception { + String log = jenkins.getLog(build); + assertNotNull("Build log should not be null", log); + assertTrue("Build log should contain interruption message: " + expectedMessage, + log.contains(expectedMessage)); + } + + /** + * Tests that a new patchset cancels a running build of an old patchset in Hazelcast mode. + * This verifies the atomic TriggeredProcessor fix works correctly in distributed mode. + * + * @throws Exception if unexpected errors appear. + */ + @Test + @LocalData("common") + public void testNewPatchsetCancelsRunningBuildHazelcast() throws Exception { + // Create a job with a long-running build and cancellation policy + FreeStyleProject project = jenkins.createFreeStyleProject(); + project.getBuildersList().add(new SleepBuilder(SLEEP_10_SEC)); // 10 second build + + GerritTrigger trigger = Setup.createDefaultTrigger(project); + trigger.setGerritProjects(Collections.singletonList( + new GerritProject(CompareType.ANT, "**", + Collections.singletonList(new Branch(CompareType.ANT, "**")), + null, null, null, false))); + BuildCancellationPolicy policy = new BuildCancellationPolicy(false, false, false, false); + policy.setEnabled(true); + trigger.setBuildCancellationPolicy(policy); + project.addTrigger(trigger); + trigger.start(project, false); + + serverMock.waitForCommand(GERRIT_STREAM_EVENTS, SERVER_WAIT); + + // Trigger build with patchset 1 + PatchsetCreated patchset1 = Setup.createPatchsetCreated(); + patchset1.getChange().setId("Iabc123"); + patchset1.getChange().setNumber("1000"); + patchset1.getPatchSet().setNumber("1"); + + gerritServer.triggerEvent(patchset1); + + // Wait for build to start (not complete) + waitForBuildToStart(project, BUILD_TIMEOUT); + FreeStyleBuild build1 = project.getLastBuild(); + assertNotNull(build1); + assertTrue(build1.isBuilding()); + + // Trigger new patchset 2 (should cancel build1) + PatchsetCreated patchset2 = Setup.createPatchsetCreated(); + patchset2.getChange().setId("Iabc123"); + patchset2.getChange().setNumber("1000"); + patchset2.getPatchSet().setNumber("2"); + + gerritServer.triggerEvent(patchset2); + + // Wait for build1 to be aborted + jenkins.waitUntilNoActivity(); + + // Verify build1 was aborted + assertEquals("Build should be ABORTED (Hazelcast mode)", Result.ABORTED, build1.getResult()); + + // Verify it was aborted with the correct interruption cause + assertInterruptionCause(build1, Messages.AbortedByNewPatchSet()); + + // Verify build2 completed successfully + TestUtils.waitForBuilds(project, 2, BUILD_TIMEOUT); + FreeStyleBuild build2 = project.getLastBuild(); + assertNotNull(build2); + jenkins.assertBuildStatusSuccess(build2); + } + + /** + * Tests that abandoned patchsets cancel running builds in Hazelcast mode. + * + * @throws Exception if unexpected errors appear. + */ + @Test + @LocalData("common") + public void testAbandonedPatchsetCancelsRunningBuildHazelcast() throws Exception { + FreeStyleProject project = jenkins.createFreeStyleProject(); + project.getBuildersList().add(new SleepBuilder(SLEEP_10_SEC)); + + GerritTrigger trigger = Setup.createDefaultTrigger(project); + trigger.setGerritProjects(Collections.singletonList( + new GerritProject(CompareType.ANT, "**", + Collections.singletonList(new Branch(CompareType.ANT, "**")), + null, null, null, false))); + BuildCancellationPolicy policy = new BuildCancellationPolicy(false, false, false, true); + policy.setEnabled(true); + trigger.setBuildCancellationPolicy(policy); + project.addTrigger(trigger); + trigger.start(project, false); + + serverMock.waitForCommand(GERRIT_STREAM_EVENTS, SERVER_WAIT); + + // Trigger build with patchset 1 + PatchsetCreated patchset1 = Setup.createPatchsetCreated(); + patchset1.getChange().setId("Iabc123"); + patchset1.getChange().setNumber("1000"); + patchset1.getPatchSet().setNumber("1"); + + gerritServer.triggerEvent(patchset1); + + waitForBuildToStart(project, BUILD_TIMEOUT); + FreeStyleBuild build1 = project.getLastBuild(); + assertTrue(build1.isBuilding()); + + // Send abandoned event + ChangeAbandoned abandoned = Setup.createChangeAbandoned(); + abandoned.getChange().setId("Iabc123"); + abandoned.getChange().setNumber("1000"); + abandoned.getPatchSet().setNumber("1"); + + gerritServer.triggerEvent(abandoned); + + jenkins.waitUntilNoActivity(); + + // Verify build was aborted + assertEquals("Build should be ABORTED (Hazelcast mode)", Result.ABORTED, build1.getResult()); + + // Verify it was aborted with the correct interruption cause + assertInterruptionCause(build1, Messages.AbortedByAbandonedPatchset()); + } + + /** + * Tests that abortNewPatchsets policy cancels even newer patchsets in Hazelcast mode. + * + * @throws Exception if unexpected errors appear. + */ + @Test + @LocalData("common") + public void testAbortNewPatchsetsPolicyCancelsAnyPatchsetHazelcast() throws Exception { + FreeStyleProject project = jenkins.createFreeStyleProject(); + project.getBuildersList().add(new SleepBuilder(SLEEP_10_SEC)); + + GerritTrigger trigger = Setup.createDefaultTrigger(project); + trigger.setGerritProjects(Collections.singletonList( + new GerritProject(CompareType.ANT, "**", + Collections.singletonList(new Branch(CompareType.ANT, "**")), + null, null, null, false))); + BuildCancellationPolicy policy = new BuildCancellationPolicy(true, false, false, false); + policy.setEnabled(true); + trigger.setBuildCancellationPolicy(policy); + project.addTrigger(trigger); + trigger.start(project, false); + + serverMock.waitForCommand(GERRIT_STREAM_EVENTS, SERVER_WAIT); + + // Trigger build with patchset 2 + PatchsetCreated patchset2 = Setup.createPatchsetCreated(); + patchset2.getChange().setId("Iabc123"); + patchset2.getChange().setNumber("1000"); + patchset2.getPatchSet().setNumber("2"); + + gerritServer.triggerEvent(patchset2); + + waitForBuildToStart(project, BUILD_TIMEOUT); + FreeStyleBuild build1 = project.getLastBuild(); + assertTrue(build1.isBuilding()); + + // Trigger patchset 1 (older, but should still cancel build1 due to policy) + PatchsetCreated patchset1 = Setup.createPatchsetCreated(); + patchset1.getChange().setId("Iabc123"); + patchset1.getChange().setNumber("1000"); + patchset1.getPatchSet().setNumber("1"); + + gerritServer.triggerEvent(patchset1); + + jenkins.waitUntilNoActivity(); + + // Verify build1 was aborted + assertEquals("Build should be ABORTED (Hazelcast mode)", Result.ABORTED, build1.getResult()); + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationIntegrationTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationIntegrationTest.java index 40710a867..9a14cd383 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationIntegrationTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationIntegrationTest.java @@ -26,6 +26,7 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.Messages; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.BuildCancellationPolicy; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.CompareType; @@ -60,7 +61,6 @@ * Integration tests for build cancellation feature that verify actual job cancellation * in Jenkins queue and running executors. * - * @author Ignacio Roncero <ironcero@cloudbees.com> */ public class BuildCancellationIntegrationTest { @@ -108,12 +108,15 @@ public void setUp() throws Exception { } /** - * Tears down the SSH server. + * Tears down the SSH server and clears Hazelcast state if active. * * @throws Exception if teardown fails */ @After public void tearDown() throws Exception { + // Clear Hazelcast state to prevent pollution between tests + HazelcastTestHelper.clearAllMaps(); + if (sshd != null) { sshd.stop(true); sshd = null; diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelperTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelperTest.java new file mode 100644 index 000000000..ca3a6e320 --- /dev/null +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/PipelineAbortHelperTest.java @@ -0,0 +1,105 @@ +/* + * The MIT License + * + * Copyright 2026 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger; + +import hudson.model.FreeStyleBuild; +import hudson.model.FreeStyleProject; +import hudson.model.Result; +import org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition; +import org.jenkinsci.plugins.workflow.job.WorkflowJob; +import org.jenkinsci.plugins.workflow.job.WorkflowRun; +import org.jenkinsci.plugins.workflow.test.steps.SemaphoreStep; +import org.junit.Rule; +import org.junit.Test; +import org.jvnet.hudson.test.JenkinsRule; + +import static org.junit.Assert.assertFalse; + +/** + * Tests for {@link PipelineAbortHelper}. + */ +public class PipelineAbortHelperTest { + + //CS IGNORE VisibilityModifier FOR NEXT 3 LINES. REASON: JUnit Rule. + //CS IGNORE JavadocVariable FOR NEXT 2 LINES. REASON: JUnit Rule. + @Rule + public final JenkinsRule jenkins = new JenkinsRule(); + + /** + * A FreeStyle build is never a Pipeline — should always return false. + */ + @Test + public void testFreeStyleBuildReturnsFalse() throws Exception { + FreeStyleProject project = jenkins.createFreeStyleProject(); + FreeStyleBuild build = project.scheduleBuild2(0).get(); + assertFalse(PipelineAbortHelper.isPipelineNotYetStarted(build)); + } + + /** + * A Pipeline build blocked inside a running step has its FlowExecution attached, + * so isPipelineNotYetStarted() must return false (safe to interrupt). + */ + @Test + public void testPipelineBlockedAtSemaphoreReturnsFalse() throws Exception { + WorkflowJob job = jenkins.createProject(WorkflowJob.class, "pipeline-semaphore"); + job.setDefinition(new CpsFlowDefinition( + "semaphore 'wait'\n" + + "echo 'done'", true)); + + WorkflowRun run = job.scheduleBuild2(0).waitForStart(); + + // Wait until the semaphore step is reached — at this point CPS is fully started + SemaphoreStep.waitForStart("wait/1", run); + + assertFalse("Pipeline blocked at semaphore should report CPS started", + PipelineAbortHelper.isPipelineNotYetStarted(run)); + + // Unblock and let the build finish cleanly + SemaphoreStep.success("wait/1", null); + jenkins.waitForCompletion(run); + jenkins.assertBuildStatusSuccess(run); + } + + /** + * A Pipeline build that has been interrupted and completed should still report + * false — the FlowExecution remains attached after the run finishes, so it + * remains safe (and correct) to report as started. + */ + @Test + public void testAbortedPipelineReturnsFalse() throws Exception { + WorkflowJob job = jenkins.createProject(WorkflowJob.class, "pipeline-aborted"); + job.setDefinition(new CpsFlowDefinition( + "semaphore 'wait-abort'\n" + + "echo 'done'", true)); + + WorkflowRun run = job.scheduleBuild2(0).waitForStart(); + SemaphoreStep.waitForStart("wait-abort/1", run); + + run.getExecutor().interrupt(Result.ABORTED); + jenkins.waitForCompletion(run); + + assertFalse("Aborted, completed pipeline should still report CPS started", + PipelineAbortHelper.isPipelineNotYetStarted(run)); + } +} diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/WorkflowTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/WorkflowTest.java index 61780dbcb..feb43bd70 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/WorkflowTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/WorkflowTest.java @@ -26,6 +26,7 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastInstanceProvider; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.CompareType; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.GerritProject; @@ -41,12 +42,15 @@ import org.jenkinsci.plugins.workflow.job.WorkflowJob; import org.jenkinsci.plugins.workflow.job.WorkflowRun; import org.junit.Assert; +import org.junit.Before; import org.junit.Rule; import org.junit.Test; import org.jvnet.hudson.test.JenkinsRule; import org.jvnet.hudson.test.TestExtension; import org.kohsuke.stapler.StaplerRequest2; import org.kohsuke.stapler.StaplerResponse2; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.io.PrintWriter; @@ -68,6 +72,8 @@ */ public class WorkflowTest { + private static final Logger logger = LoggerFactory.getLogger(WorkflowTest.class); + /** * Jenkins rule. */ @@ -75,6 +81,30 @@ public class WorkflowTest { @Rule public final JenkinsRule jenkinsRule = new JenkinsRule(); + /** + * Clear Hazelcast notification map before each test. + * This prevents test pollution when running multiple WorkflowTest methods in Hazelcast mode, + * since all tests use the same mock event ID. + */ + @Before + public void clearHazelcastNotificationMap() { + // Only clear if Hazelcast is initialized + if (HazelcastInstanceProvider.isInitialized()) { + try { + com.hazelcast.core.HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + com.hazelcast.map.IMap notificationFlags = + instance.getMap("gerrit-trigger-notification-flags"); + int clearedCount = notificationFlags.size(); + notificationFlags.clear(); + if (clearedCount > 0) { + logger.info("Cleared {} notification claim(s) before WorkflowTest", clearedCount); + } + } catch (Exception e) { + logger.warn("Failed to clear Hazelcast notification flags in WorkflowTest", e); + } + } + } + /** * Trigger test. * @throws Exception if there is one. @@ -346,7 +376,9 @@ private static MockGerritServer get(JenkinsRule jenkinsRule) throws IOException private void configure(JenkinsRule jenkinsRule) throws IOException { PluginImpl.getInstance().addServer(this); Config config = (Config)getConfig(); - config.setGerritFrontEndURL(jenkinsRule.getURL().toString() + getUrlName() + "/"); + String frontEndUrl = jenkinsRule.getURL().toString() + getUrlName() + "/"; + + config.setGerritFrontEndURL(frontEndUrl); config.setUseRestApi(true); config.setGerritHttpUserName("user"); config.setGerritHttpPassword("passwd"); diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/replication/ReplicationQueueTaskDispatcherTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/replication/ReplicationQueueTaskDispatcherTest.java index c45c2f82e..783432af8 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/replication/ReplicationQueueTaskDispatcherTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/replication/ReplicationQueueTaskDispatcherTest.java @@ -57,7 +57,9 @@ import jenkins.model.TransientActionFactory; import org.junit.After; +import org.junit.AfterClass; import org.junit.Before; +import org.junit.BeforeClass; import org.junit.Test; import com.sonymobile.tools.gerrit.gerritevents.GerritHandler; @@ -71,6 +73,7 @@ import com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated; import com.sonymobile.tools.gerrit.gerritevents.dto.events.RefReplicated; import com.sonymobile.tools.gerrit.gerritevents.dto.events.RefUpdated; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; import com.sonyericsson.hudson.plugins.gerrit.trigger.events.ManualPatchsetCreated; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritCause; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritManualCause; @@ -93,8 +96,28 @@ public class ReplicationQueueTaskDispatcherTest { private static final int HOURSBEFORECHANGEMERGEDFORPATCHSET = -8; private static final int HOURBEFOREREPLICATIONCACHECREATED = -1; + private static final long ONE_SECOND_MS = 1000; + private static final long TWO_SECONDS_MS = 2000; private MockedStatic jenkinsMockedStatic; + /** + * Clear Hazelcast state before running this test class. + * Ensures clean state regardless of what tests ran before. + */ + @BeforeClass + public static void setUpClass() { + HazelcastTestHelper.clearAllMaps(); + } + + /** + * Clear Hazelcast state after running this test class. + * Prevents pollution of subsequent tests. + */ + @AfterClass + public static void tearDownClass() { + HazelcastTestHelper.clearAllMaps(); + } + /** * Create ReplicationQueueTaskDispatcher with a mocked GerritHandler. */ @@ -419,19 +442,32 @@ public void shouldBlockItemUntilIfPatchSetIsReplicatedToOneSlaveBeforeChangeMerg //to Gerrit for review). //For change merged event, a check on the event timestamp is done to make sure not to unblock the build for //a replica event that was fired before the current change merged event. - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/branch", "someGerritServer", - "slaveA", RefReplicated.SUCCEEDED_STATUS)); + + // Set explicit timestamps AFTER cache creation to avoid expiry issues. + // Timestamps must be relative to ensure old < changeMerged < new ordering. + long baseTime = System.currentTimeMillis() + ONE_SECOND_MS; // Start 1 second in the future + long oldReplicaTime = baseTime; + long changeMergedTime = baseTime + ONE_SECOND_MS; // 1 second after old replica + long newReplicaTime = baseTime + TWO_SECONDS_MS; // 2 seconds after old replica + + RefReplicated oldRefReplicated = Setup.createRefReplicatedEvent("someProject", "refs/heads/branch", + "someGerritServer", "slaveA", RefReplicated.SUCCEEDED_STATUS); + oldRefReplicated.setReceivedOn(oldReplicaTime); + dispatcher.gerritEvent(oldRefReplicated); ChangeMerged changeMerged = Setup.createChangeMerged("someGerritServer", "someProject", "refs/changes/1/1/1"); + changeMerged.setReceivedOn(changeMergedTime); Item item = createItem(changeMerged, new String[] {"slaveA"}); assertNotNull("Item should be blocked as the replica event happened before the change event", dispatcher.canRun(item)); //fire the replica event that will unblock the item - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/branch", "someGerritServer", - "slaveA", RefReplicated.SUCCEEDED_STATUS)); + RefReplicated newRefReplicated = Setup.createRefReplicatedEvent("someProject", "refs/heads/branch", + "someGerritServer", "slaveA", RefReplicated.SUCCEEDED_STATUS); + newRefReplicated.setReceivedOn(newReplicaTime); + dispatcher.gerritEvent(newRefReplicated); assertNull("Item should not be blocked anymore as a newer replica event was received", dispatcher.canRun(item)); @@ -638,14 +674,27 @@ public void shouldNotBlockItemWhenReplicationIsCompletedBeforeDispatcherIsCalled */ @Test public void shouldBlockItemUntilProperReplicationEventIsReceived() { + // Set explicit timestamps to avoid cache expiry issues. + // Old events should be before RefUpdated, new events after. + long baseTime = System.currentTimeMillis() + ONE_SECOND_MS; + long oldEventTime = baseTime; + long refUpdatedTime = baseTime + ONE_SECOND_MS; + long newEventTime = baseTime + TWO_SECONDS_MS; + //send replication events created before the actual event - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/master", "someGerritServer", - "slaveB", RefReplicated.SUCCEEDED_STATUS)); - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/master", "someGerritServer", - "slaveA", RefReplicated.SUCCEEDED_STATUS)); + RefReplicated oldRefRepB = Setup.createRefReplicatedEvent("someProject", "refs/heads/master", + "someGerritServer", "slaveB", RefReplicated.SUCCEEDED_STATUS); + oldRefRepB.setReceivedOn(oldEventTime); + dispatcher.gerritEvent(oldRefRepB); + + RefReplicated oldRefRepA = Setup.createRefReplicatedEvent("someProject", "refs/heads/master", + "someGerritServer", "slaveA", RefReplicated.SUCCEEDED_STATUS); + oldRefRepA.setReceivedOn(oldEventTime); + dispatcher.gerritEvent(oldRefRepA); RefUpdated refUpdated = Setup.createRefUpdated("someGerritServer", "someProject", "master"); + refUpdated.setReceivedOn(refUpdatedTime); Item item = createItem(refUpdated, new String[] {"slaveA", "slaveB"}); //item is blocked since the cached replication events are time stamped before the actual event @@ -657,10 +706,15 @@ public void shouldBlockItemUntilProperReplicationEventIsReceived() { assertTrue(cause.getShortDescription().contains("slaveB")); //send replication events created after the actual event - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/master", "someGerritServer", - "slaveB", RefReplicated.SUCCEEDED_STATUS)); - dispatcher.gerritEvent(Setup.createRefReplicatedEvent("someProject", "refs/heads/master", "someGerritServer", - "slaveA", RefReplicated.SUCCEEDED_STATUS)); + RefReplicated newRefRepB = Setup.createRefReplicatedEvent("someProject", "refs/heads/master", + "someGerritServer", "slaveB", RefReplicated.SUCCEEDED_STATUS); + newRefRepB.setReceivedOn(newEventTime); + dispatcher.gerritEvent(newRefRepB); + + RefReplicated newRefRepA = Setup.createRefReplicatedEvent("someProject", "refs/heads/master", + "someGerritServer", "slaveA", RefReplicated.SUCCEEDED_STATUS); + newRefRepA.setReceivedOn(newEventTime); + dispatcher.gerritEvent(newRefRepA); assertNull("Item should not be blocked", dispatcher.canRun(item)); verify(queueMock, times(1)).maintain(); diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/ParameterModeJenkinsTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/ParameterModeJenkinsTest.java index 1a7e3f2e6..1f5e86f39 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/ParameterModeJenkinsTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/ParameterModeJenkinsTest.java @@ -25,6 +25,7 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTriggerParameters; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; @@ -57,6 +58,7 @@ import hudson.tasks.Builder; import org.hamcrest.MatcherAssert; import org.hamcrest.Matchers; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -68,6 +70,7 @@ import java.util.Collections; import java.util.List; import java.util.Map; +import java.util.concurrent.TimeUnit; import static org.hamcrest.core.StringContains.containsString; import static org.junit.Assert.assertNotNull; @@ -88,6 +91,16 @@ public class ParameterModeJenkinsTest { private FreeStyleProject job; private GerritTrigger trigger; + /** + * Maximum time to wait for {@link #waitForEventToBeBuilt()} to see the event scheduled. + */ + private static final int SCHEDULE_WAIT_SECONDS = 30; + + /** + * Poll interval for {@link #waitForEventToBeBuilt()} while waiting for scheduling. + */ + private static final int SCHEDULE_POLL_INTERVAL_MILLIS = 50; + /** * Shared setup for all tests. * @@ -113,6 +126,16 @@ public void setup() throws IOException { trigger.setEscapeQuotes(false); } + /** + * Clean up Hazelcast state after each test to prevent state pollution. + * This is critical for CommentAdded tests which use BuildMemory.isBuilding() + * to check for duplicate builds. + */ + @After + public void tearDown() { + HazelcastTestHelper.clearAllMaps(); + } + /** * Mock Gerrit server with a version. */ @@ -153,7 +176,7 @@ public void testNameAndEmailParameterModeDefault() throws Exception { assertSame(GerritTriggerParameters.ParameterMode.PLAIN, trigger.getNameAndEmailParameterMode()); Account ac = new Account("Bobby", "rsandell@cloudbees.com"); PluginImpl.getHandler_().triggerEvent(Setup.createPatchsetCreatedWithAccounts(ac, ac, ac)); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -180,7 +203,7 @@ public void testNameAndEmailParameterModeDefaultChangeAbandoned() throws Excepti changeAbandoned.getChange().setOwner(ac); changeAbandoned.setAbandoner(ac); PluginImpl.getHandler_().triggerEvent(changeAbandoned); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -208,7 +231,7 @@ public void testNameAndEmailParameterModeDefaultTopicChanged() throws Exception topicChanged.getChange().setOwner(ac); topicChanged.setChanger(ac); PluginImpl.getHandler_().triggerEvent(topicChanged); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -237,7 +260,7 @@ public void testNameAndEmailParameterModeDefaultChangeRestored() throws Exceptio change.getChange().setOwner(ac); change.setRestorer(ac); PluginImpl.getHandler_().triggerEvent(change); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -263,7 +286,7 @@ public void testNameAndEmailParameterModeDefaultRefUpdated() throws Exception { RefUpdated change = Setup.createRefUpdated(PluginImpl.DEFAULT_SERVER_NAME, "olle", "abc123"); change.setAccount(ac); PluginImpl.getHandler_().triggerEvent(change); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Collections.singletonList(GerritTriggerParameters.GERRIT_EVENT_ACCOUNT); //TODO According to the doc GerritTriggerParameters.GERRIT_SUBMITTER should be set as well but its not? @@ -285,7 +308,7 @@ public void testNameAndEmailParameterModeBase64() throws Exception { trigger.setNameAndEmailParameterMode(GerritTriggerParameters.ParameterMode.BASE64); Account ac = new Account("Bobby", "rsandell@cloudbees.com"); PluginImpl.getHandler_().triggerEvent(Setup.createPatchsetCreatedWithAccounts(ac, ac, ac)); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -308,7 +331,7 @@ public void testNameAndEmailParameterModeNone() throws Exception { trigger.setNameAndEmailParameterMode(GerritTriggerParameters.ParameterMode.NONE); Account ac = new Account("Bobby", "rsandell@cloudbees.com"); PluginImpl.getHandler_().triggerEvent(Setup.createPatchsetCreatedWithAccounts(ac, ac, ac)); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); List params = Arrays.asList( GerritTriggerParameters.GERRIT_CHANGE_OWNER, @@ -334,7 +357,7 @@ public void testCommitMessageParameterModeDefault() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setCommitMessage(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_CHANGE_COMMIT_MESSAGE.name() + "=" @@ -355,7 +378,7 @@ public void testCommitMessageParameterModePlain() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setCommitMessage(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_CHANGE_COMMIT_MESSAGE.name() + "=" @@ -376,7 +399,7 @@ public void testCommitMessageParameterModeNone() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setCommitMessage(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogNotContains(GerritTriggerParameters.GERRIT_CHANGE_COMMIT_MESSAGE.name(), build); } @@ -396,7 +419,7 @@ public void testCommentTextParameterModeDefault() throws Exception { CommentAdded event = Setup.createCommentAdded(); event.setComment(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_EVENT_COMMENT_TEXT.name() + "=" @@ -418,7 +441,7 @@ public void testCommentTextParameterModePlain() throws Exception { CommentAdded event = Setup.createCommentAdded(); event.setComment(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_EVENT_COMMENT_TEXT.name() + "=" @@ -440,7 +463,7 @@ public void testCommentTextParameterModeNone() throws Exception { CommentAdded event = Setup.createCommentAdded(); event.setComment(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogNotContains(GerritTriggerParameters.GERRIT_EVENT_COMMENT_TEXT.name(), build); } @@ -459,7 +482,7 @@ public void testChangeSubjectParameterModeNone() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setSubject(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogNotContains(GerritTriggerParameters.GERRIT_CHANGE_SUBJECT.name(), build); } @@ -478,7 +501,7 @@ public void testChangeSubjectParameterModeDefault() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setSubject(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_CHANGE_SUBJECT.name() + "=" @@ -499,13 +522,34 @@ public void testChangeSubjectParameterModeBase64() throws Exception { PatchsetCreated event = Setup.createPatchsetCreated(); event.getChange().setSubject(expected); PluginImpl.getHandler_().triggerEvent(event); - j.waitUntilNoActivity(); + waitForEventToBeBuilt(); FreeStyleBuild build = job.getLastBuild(); assertLogContains(GerritTriggerParameters.GERRIT_CHANGE_SUBJECT.name() + "=" + GerritTriggerParameters.ParameterMode.encodeBase64(expected), build); } + /** + * Waits for the event fired via {@link PluginImpl#getHandler_()} to be picked up and + * scheduled as a build, then waits for that build to finish. + *

+ * {@code triggerEvent} hands the event to a background worker thread; with Hazelcast + * coordination the worker's event-claim check does a network round trip, so the item may + * not exist in Jenkins' queue yet at the moment this is called. {@link JenkinsRule + * #waitUntilNoActivity()} alone can't tell "nothing scheduled yet" apart from "already + * finished" — an empty queue looks the same either way — so it can return before the + * build was ever created. + * + * @throws Exception if so + */ + private void waitForEventToBeBuilt() throws Exception { + long deadline = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(SCHEDULE_WAIT_SECONDS); + while (job.getLastBuild() == null && !job.isInQueue() && System.currentTimeMillis() < deadline) { + Thread.sleep(SCHEDULE_POLL_INTERVAL_MILLIS); + } + j.waitUntilNoActivity(); + } + /** * Asserts that the log contains something. * diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/SpecGerritTriggerHudsonTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/SpecGerritTriggerHudsonTest.java index 48c92b948..c97c2fde4 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/SpecGerritTriggerHudsonTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/SpecGerritTriggerHudsonTest.java @@ -31,6 +31,8 @@ import com.sonymobile.tools.gerrit.gerritevents.dto.events.PatchsetCreated; import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastInstanceProvider; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; import com.sonyericsson.hudson.plugins.gerrit.trigger.events.ManualPatchsetCreated; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritCause; @@ -41,6 +43,8 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.mock.TestUtils; import com.sonymobile.tools.gerrit.gerritevents.dto.events.TopicChanged; import com.sonymobile.tools.gerrit.gerritevents.mock.SshdServerMock; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import hudson.model.Cause; import hudson.model.FreeStyleBuild; @@ -74,7 +78,7 @@ import static org.junit.Assert.assertSame; import static org.junit.Assert.fail; -//CS IGNORE MagicNumber FOR NEXT 920 LINES. REASON: Testdata. +//CS IGNORE MagicNumber FOR NEXT 960 LINES. REASON: Testdata. /** * Some full run-through tests from trigger to build finished. @@ -83,6 +87,8 @@ */ public class SpecGerritTriggerHudsonTest { + private static final Logger logger = LoggerFactory.getLogger(SpecGerritTriggerHudsonTest.class); + /** * An instance of Jenkins Rule. */ @@ -111,6 +117,8 @@ public class SpecGerritTriggerHudsonTest { */ @Before public void setUp() throws Exception { + // Clear Hazelcast maps before each test to prevent test pollution + clearHazelcastMaps(); SshdServerMock.generateKeyPair(); serverMock = new SshdServerMock(); @@ -125,6 +133,39 @@ public void setUp() throws Exception { SshdServerMock.configureFor(sshd, gerritServer, true); } + /** + * Clear Hazelcast notification and event claim maps before each test. + * This prevents test pollution when running multiple test methods in Hazelcast mode, + * since many tests use the same mock event IDs. + */ + private void clearHazelcastMaps() { + // Only clear if Hazelcast is initialized + if (HazelcastInstanceProvider.isInitialized()) { + try { + com.hazelcast.core.HazelcastInstance instance = HazelcastInstanceProvider.getInstance(); + + // Clear notification flags + com.hazelcast.map.IMap notificationFlags = + instance.getMap("gerrit-trigger-notification-flags"); + int notificationCount = notificationFlags.size(); + notificationFlags.clear(); + + // Clear event claims + com.hazelcast.map.IMap eventClaims = + instance.getMap("gerrit-trigger-event-claims"); + int eventClaimCount = eventClaims.size(); + eventClaims.clear(); + + if (notificationCount > 0 || eventClaimCount > 0) { + logger.info("Cleared {} notification claim(s) and {} event claim(s) before test", + notificationCount, eventClaimCount); + } + } catch (Exception e) { + logger.warn("Failed to clear Hazelcast maps in SpecGerritTriggerHudsonTest", e); + } + } + } + /** * Runs after test method. * @@ -132,8 +173,10 @@ public void setUp() throws Exception { */ @After public void tearDown() throws Exception { + gerritServer.stopConnection(); serverMock.stopServer(sshd); sshd = null; + HazelcastTestHelper.clearAllMaps(); } /** @@ -374,6 +417,9 @@ public void testDoubleTriggeredBuildsOfDifferentChange() throws Exception { System.out.println("Build Started"); PatchsetCreated patchsetCreated = Setup.createPatchsetCreated(); patchsetCreated.getChange().setNumber("2000"); + // Must also set unique Change-Id for different change number + // (in real Gerrit, each change has its own unique Change-Id) + patchsetCreated.getChange().setId("Ibbddeeff987654321"); gerritServer.triggerEvent(patchsetCreated); System.out.println("PatchSet 2 created"); diff --git a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/VoteSameTopicTest.java b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/VoteSameTopicTest.java index 91e574dd2..c694b0b96 100644 --- a/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/VoteSameTopicTest.java +++ b/src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/VoteSameTopicTest.java @@ -2,6 +2,7 @@ import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; import com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl; +import com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastTestHelper; import com.sonyericsson.hudson.plugins.gerrit.trigger.config.Config; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger; import com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch; @@ -126,6 +127,7 @@ public void setup() throws Exception { public void tearDown() throws Exception { server.stopServer(sshd); sshd = null; + HazelcastTestHelper.clearAllMaps(); } /** diff --git a/src/test/resources/META-INF/services/org.junit.platform.launcher.TestExecutionListener b/src/test/resources/META-INF/services/org.junit.platform.launcher.TestExecutionListener new file mode 100644 index 000000000..626942932 --- /dev/null +++ b/src/test/resources/META-INF/services/org.junit.platform.launcher.TestExecutionListener @@ -0,0 +1 @@ +com.sonyericsson.hudson.plugins.gerrit.trigger.coordination.hazelcast.HazelcastServerTestListener diff --git a/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest/common/gerrit-trigger.xml b/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest/common/gerrit-trigger.xml new file mode 100644 index 000000000..48b3ea51b --- /dev/null +++ b/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationSmokeTest/common/gerrit-trigger.xml @@ -0,0 +1,27 @@ + + + + 127.0.0.1 + 29418 + jenkins + /tmp/jenkins-testkey + + 3 + gerrit review <CHANGE>,<PATCHSET> --message 'Build Successful' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Unstable' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Failed' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Started' --verified <VERIFIED> + http://localhost/ + 0 + 0 + 1 + 0 + -1 + 0 + -1 + 0 + true + 1 + 1 + + diff --git a/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest/common/gerrit-trigger.xml b/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest/common/gerrit-trigger.xml new file mode 100644 index 000000000..48b3ea51b --- /dev/null +++ b/src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest/common/gerrit-trigger.xml @@ -0,0 +1,27 @@ + + + + 127.0.0.1 + 29418 + jenkins + /tmp/jenkins-testkey + + 3 + gerrit review <CHANGE>,<PATCHSET> --message 'Build Successful' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Unstable' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Failed' --verified <VERIFIED> + gerrit review <CHANGE>,<PATCHSET> --message 'Build Started' --verified <VERIFIED> + http://localhost/ + 0 + 0 + 1 + 0 + -1 + 0 + -1 + 0 + true + 1 + 1 + +