Skip to content

Work-request resource actions can override known CRUD phase and break reconciliation #86

Description

@HaoyL666

Summary

OSOK's shared WorkRequestAsyncAdapter.ResolvePhase treats every non-empty work-request resource actionType as if it must identify the controller's CRUD phase.

Several OCI services legitimately return resource actions such as IN_PROGRESS, RELATED, or FAILED while the actual operation is already known from the top-level operationType or the controller's persisted reconcile phase. The adapter rejects those responses with:

unmodeled async action "IN_PROGRESS"

This can turn a healthy asynchronous create/update/delete into a reconciliation error, or obscure a real OCI failure instead of recording the failed CRUD phase.

Live symptom

Queue create returned a valid work request with:

operationType = CREATE_QUEUE
status        = IN_PROGRESS
actionType    = IN_PROGRESS

The controller had already derived phase Create, but ResolvePhase rejected the resource action.

Affected resources

Repository-wide SDK/resolver tracing and package-level synthetic regression tests produced this matrix. "Rejects" means the package's real resolver passes the SDK-valid action to the shared adapter, which returns unmodeled async action.

Resource IN_PROGRESS RELATED FAILED
Queue Rejects Rejects* Not in SDK
Service Connector Rejects Rejects* Not in SDK
ZPR Configuration Rejects Rejects* Rejects
ZPR Policy Rejects Rejects* Rejects
AI Data Platform Ignored Rejects* Rejects
AI Language Project Ignored Rejects* Not in SDK
DevOps Build Pipeline Ignored Rejects* Rejects
DevOps Project Ignored Rejects* Rejects
Managed Kafka Cluster Ignored Ignored Rejects
OPSI Chargeback Plan Report Ignored Ignored Rejects
OPSI Enterprise Manager Bridge Ignored Ignored Rejects
Recovery Protected Database Ignored Rejects* Rejects

RELATED normally describes a secondary resource, so rows marked * are conditional on OCI returning it for the primary entity selected by that controller.

The normal IN_PROGRESS path is directly affected for:

  • Queue
  • Service Connector
  • ZPR Configuration
  • ZPR Policy

Their vendored SDK WorkRequestResource contracts state that a primary resource being created, updated, or deleted remains IN_PROGRESS until work completes.

Root cause

pkg/servicemanager/async_status.go currently follows this sequence:

  1. Resolve an expected phase from explicit/persisted reconciliation state.
  2. If rawAction is non-empty, require it to map to a configured Create/Update/Delete action.
  3. Reject every other action before the known expected phase can be used.

operationType, work-request status, and resource actionType are separate dimensions:

  • operationType / persisted phase: Create, Update, or Delete
  • status: Pending, Succeeded, Failed, Canceled, or Needs Attention
  • resource actionType: supplemental state or relationship of an affected entity

Proposed behavior

When a reliable expected phase already exists:

  • Continue accepting recognized CRUD actions as consistency evidence.
  • Continue rejecting recognized CRUD actions that contradict the expected phase.
  • Do not let a non-CRUD resource action override or invalidate the known phase.
  • Continue returning an error when neither action nor fallback/persisted state can determine a phase.

Conceptually:

resolvedPhase, ok := adapter.phaseForAction(actionToken)
if !ok {
    if expectedPhase != "" {
        return expectedPhase, nil
    }
    return "", fmt.Errorf("unmodeled async action %q", rawAction)
}

Regression evidence

Temporary package-level tests used each package's real SDK WorkRequest, action resolver, adapter, and known Create fallback:

  • Before the proposed behavior, every matrix entry marked Rejects failed with the expected error.
  • With the central fallback applied experimentally, all 12 candidate package suites passed.
  • go test ./pkg/servicemanager/... passed across the complete service-manager tree.
  • Existing phase-conflict protection continued to pass.

The temporary experiment was reverted after validation.

Acceptance criteria

  • A known Create/Update/Delete phase survives SDK-valid non-CRUD resource actions.
  • Known CRUD action conflicts still fail, for example expected Create with UPDATED.
  • Unknown/non-CRUD action with no fallback or persisted phase still fails.
  • Shared table-driven tests cover IN_PROGRESS, RELATED, FAILED, missing phase, and action/phase conflict.
  • Regression tests cover Queue, Service Connector, ZPR Configuration, and ZPR Policy.
  • Representative failure-action tests cover at least one service whose SDK supports ActionTypeFailed.
  • go test ./pkg/servicemanager/... passes.

OCI reference

OCI documents work-request status separately from the affected resource list and its entityType/actionType fields:

https://docs.oracle.com/en-us/iaas/Content/API/Concepts/workrequests.htm

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions