Fix relay delivery retries and diagnostics for RC2

ARCHITECTURE.md

@@ -74,10 +74,20 @@ and ActivityPub object resolution also uses the relay actor identity. Authorized
 fetch `GET` requests sign `(request-target)`, `Host`, and `Date`; redirected
 fetches are re-signed for the new request target and authority. Delivery `POST`
 requests additionally sign `Digest` and `Content-Type`. Both paths sign the exact
-`Host` authority transmitted on the wire. Each `relay-v2` fan-out attempt also
-atomically records the receiver domain's last success or failure time, total
-successes and failures, and consecutive failure count. Registration traffic does
-not affect these delivery-health observations.
+`Host` authority transmitted on the wire. Each `relay-v2` target receives one initial attempt and five delayed retries
+using Machinery's Fibonacci schedule. The shared activity body remains in Redis
+for fifteen minutes, which exceeds the complete retry horizon. A target reduces
+the shared `remain_count` only after success or final retry exhaustion, so one
+temporary receiver failure cannot delete the payload needed by its own retry or
+by another receiver. Every attempt atomically records receiver delivery health;
+registration traffic remains outside these observations.
+
+Worker logs contain bounded structured correlation fields: stable task UUID,
+storage ID, public activity and object identifiers when present, origin and
+receiver domains, body digest, attempt counters, elapsed time, HTTP status,
+error class, next retry delay, and bounded non-success response text. Raw
+activity bodies, signatures, private keys, and unbounded remote content are not
+logged or exported as metric labels.
 
 The existing Fediverse `Signature`-header profile remains the compatibility
 baseline. RFC 9421 verification and signing are planned as an additive security
@@ -208,8 +218,9 @@ configuration, inbox URLs, actor IDs, error text, and blocked lists.
 - Queue admission is bounded and atomic.
 - Remote JSON and error bodies are size-limited, including non-success actor
   and object fetch responses.
-- Worker failures include actionable bounded response text and are retried by
-  the task backend.
+- Worker failures include bounded structured correlation fields and response
+  text. Retriable attempts preserve the shared activity body; success and final
+  exhaustion are the only terminal target outcomes that reduce `remain_count`.
 - A receiver-health recording failure is logged but does not change the delivery
   result or retry decision.
 - An observability bind failure prevents the API server from accepting traffic,

CHANGELOG.md

@@ -2,6 +2,25 @@
 
 ## [Unreleased]
 
+## [2.5.0-rc2] - 2026-07-31
+### Added
+- Add bounded structured delivery-attempt diagnostics containing task and
+  activity correlation identifiers, origin and receiver domains, attempt
+  counters, elapsed time, HTTP status, error class, retry schedule, body digest,
+  and bounded non-success response text.
+- Add regression tests proving a retriable receiver failure preserves the shared
+  activity body and a later retry can deliver it successfully.
+### Changed
+- Give each `relay-v2` fan-out target five delayed retries using Machinery's
+  Fibonacci schedule after the initial delivery attempt.
+- Retain shared fan-out activity bodies for fifteen minutes, exceeding the full
+  retry horizon with an operational margin.
+### Fixed
+- Decrement a shared activity's `remain_count` only after successful delivery or
+  final retry exhaustion, rather than after every failed attempt.
+- Prevent transient receiver timeouts from deleting the payload required by
+  their own delayed retries or by other receivers.
+
 ## [2.5.0-rc1] - 2026-07-30
 
 ### Added

api/relay_activity_store_test.go

@@ -4,6 +4,8 @@ import (
 	"context"
 	"testing"
 	"time"
+
+	"github.com/thystra/Activity-Relay/internal/deliverypolicy"
 )
 
 func TestStoreRelayActivityReturnsSuccessAndPersistsPayload(t *testing.T) {
@@ -36,7 +38,8 @@ func TestStoreRelayActivityReturnsSuccessAndPersistsPayload(t *testing.T) {
 	if err != nil {
 		t.Fatalf("read relay activity TTL: %v", err)
 	}
-	if ttl <= 0 || ttl > 2*time.Minute {
-		t.Errorf("TTL = %v; want > 0 and <= 2m", ttl)
+	minimumTTL := deliverypolicy.ActivityRetention - time.Minute
+	if ttl < minimumTTL || ttl > deliverypolicy.ActivityRetention {
+		t.Errorf("TTL = %v; want between %v and %v", ttl, minimumTTL, deliverypolicy.ActivityRetention)
 	}
 }

api/relay_retry_policy_test.go

@@ -0,0 +1,32 @@
+// File: api/relay_retry_policy_test.go
+package api
+
+import (
+	"testing"
+
+	"github.com/thystra/Activity-Relay/internal/deliverypolicy"
+)
+
+func TestRelayTaskUsesBoundedRetryPolicy(t *testing.T) {
+	signature := relayTask(
+		"https://receiver.example/inbox",
+		"activity-storage-id",
+	)
+
+	if signature.RetryCount != deliverypolicy.RetryCount {
+		t.Fatalf(
+			"RetryCount = %d; want %d",
+			signature.RetryCount,
+			deliverypolicy.RetryCount,
+		)
+	}
+	if signature.RetryTimeout != deliverypolicy.InitialRetryTimeoutSeconds {
+		t.Fatalf(
+			"RetryTimeout = %d; want %d",
+			signature.RetryTimeout,
+			deliverypolicy.InitialRetryTimeoutSeconds,
+		)
+	}
+}
+
+// EOF: api/relay_retry_policy_test.go

api/resolver.go

@@ -12,6 +12,7 @@ import (
 	"github.com/RichardKnop/machinery/v2/tasks"
 	"github.com/google/uuid"
 	"github.com/sirupsen/logrus"
+	"github.com/thystra/Activity-Relay/internal/deliverypolicy"
 	"github.com/thystra/Activity-Relay/models"
 )
 
@@ -136,8 +137,9 @@ func enqueueRegisterActivity(inboxURL string, body []byte) {
 }
 func relayTask(inboxURL string, activityID string) *tasks.Signature {
 	return &tasks.Signature{
-		Name:       "relay-v2",
-		RetryCount: 0,
+		Name:         "relay-v2",
+		RetryCount:   deliverypolicy.RetryCount,
+		RetryTimeout: deliverypolicy.InitialRetryTimeoutSeconds,
 		Args: []tasks.Arg{
 			{
 				Name:  "inboxURL",
@@ -160,7 +162,7 @@ func storeRelayActivity(activityID string, body []byte, targetCount int) error {
 		[]string{"relay:activity:" + activityID},
 		body,
 		targetCount,
-		2*60,
+		deliverypolicy.ActivityRetentionSeconds,
 	).Int()
 	if err != nil {
 		recordRedisOperationFailure("api", "activity_store")

debian/changelog

@@ -1,3 +1,15 @@
+activity-relay (2.5.0~rc2-1) noble; urgency=medium
+
+  * Retry each relay fan-out target five times after the initial attempt.
+  * Retain shared activity bodies for fifteen minutes across the retry horizon.
+  * Reduce remain_count only on successful or finally exhausted target delivery.
+  * Add bounded structured task, activity, receiver, timing, status, retry, and
+    remote-response diagnostics without logging activity bodies or key material.
+  * Add regression tests for retriable failure retention, successful retry,
+    terminal cleanup, retry horizon, metadata extraction, and response bounds.
+
+ -- thystra <noreply@github.com>  Fri, 31 Jul 2026 17:23:00 -0400
+
 activity-relay (2.5.0~rc1-1) noble; urgency=medium
 
   * Sign remote actor and canonical-object GET requests with the relay actor