'Proxy' object has no attribute 'lsvolumesnapshot' on CreateVolume-from-snapshot when SVC firmware lacks the snapshot-2.0 framework

[bluecmd]

'Proxy' object has no attribute 'lsvolumesnapshot' on CreateVolume-from-snapshot when SVC firmware lacks the snapshot-2.0 framework

Disclosure: this report was drafted by an AI coding assistant (Claude) working under my direction. I ran the reproductions on my own cluster, reviewed the analysis, and am posting it from my account; the prose, code citations, and suggested patches were AI-generated and have not been validated against IBM's internal test suite. Treat the patch diffs as suggestions, not as a tested fix.

TL;DR

On a Spectrum Virtualize backend whose CLI schema does not expose addsnapshot / lsvolumesnapshot, the driver behaves inconsistently when a StorageClass sets virt_snap_func: "true":

Path	Behavior	Expected
CreateSnapshot from PVC	Silently falls back to legacy FlashCopy (mkfcmap) ✅	OK
CreateVolume from PVC clone (dataSourceRef: PersistentVolumeClaim)	Fails with misleading error Snapshot function is enabled but not supported with object : csi_pvc-<destination-uid>	Should fall back to legacy FlashCopy clone, OR error must blame the source, not the destination
CreateVolume from VolumeSnapshot (dataSourceRef: VolumeSnapshot)	Uncaught AttributeError: 'Proxy' object has no attribute 'lsvolumesnapshot' — controller returns Internal to the CSI external-provisioner, which retries forever	Should detect missing capability and either fall back to legacy FlashCopy lookup or fail cleanly with VirtSnapshotFunctionNotSupportedMessage

The CreateSnapshot path correctly gates on _is_vdisk_support_addsnapshot() before using the snapshot-2.0 API; the CreateVolume-from-source paths do not perform the equivalent check, so a single virt_snap_func: "true" on the StorageClass produces an environment where snapshots can be created but never consumed.

Closest existing report: #844 ("1.13 create a VM", open, no follow-up) — that one shows the related AttributeError: volume_group_name traceback under the same conditions. Filing this separately because the underlying defect class (no firmware-capability gating on the CreateVolume-from-source code paths) has at least three distinct manifestations.

Environment

• Driver: quay.io/ibmcsiblock/ibm-block-csi-driver-controller:1.13.1
• Sidecars: csi-provisioner v4.0.1, csi-attacher v4.9.0, csi-snapshotter v8.3.0, csi-resizer v1.14.0
• Kubernetes: 1.35.2 (Talos)
• Backend: IBM Spectrum Virtualize (SVC). Symptom-defined: CLI schema does not include addsnapshot / lsvolumesnapshot; legacy FlashCopy commands (mkfcmap, lsvdisk, etc.) are present. _is_addsnapshot_supported() returns False.
• CDI: v1.65.0 (KubeVirt 1.8.2) — but the bug reproduces without CDI; see the bare kubectl apply recipes below.

StorageClass under test

Mirrors the documented Snapshot-2.0 setup:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: tera-compressed
provisioner: block.csi.ibm.com
parameters:
  pool: Pool0_CSI
  SpaceEfficiency: dedup_compressed
  volume_name_prefix: csi
  virt_snap_func: "true"
  csi.storage.k8s.io/fstype: ext4
  csi.storage.k8s.io/secret-name: tera-secret
  csi.storage.k8s.io/secret-namespace: ibm-block-csi
allowVolumeExpansion: true

VolumeSnapshotClass:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata: {name: ibm-block}
driver: block.csi.ibm.com
deletionPolicy: Delete
parameters:
  csi.storage.k8s.io/snapshotter-secret-name: tera-secret
  csi.storage.k8s.io/snapshotter-secret-namespace: ibm-block-csi

Reproduction

Step 1 — source PVC + snapshot (these succeed)

apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: src, namespace: vdi}
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: tera-compressed
  volumeMode: Filesystem
  resources: {requests: {storage: 2Gi}}
---
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata: {name: snap, namespace: vdi}
spec:
  volumeSnapshotClassName: ibm-block
  source: {persistentVolumeClaimName: src}

kubectl get vs -n vdi snap:

NAME   READYTOUSE   ...
snap   true         (created in ~2s)

VolumeSnapshotContent snapshotHandle is SVC:<id>;<wwn> — a regular volume handle, not a snapshot-2.0 handle, confirming the driver took the legacy _create_snapshot path (mkfcmap).

Controller log excerpt:

INFO  array_mediator_svc.py:create_snapshot:1165 - creating snapshot 'CSI_snapshot-...' from volume '<wwn>'
INFO  array_mediator_svc.py:create_snapshot:1189 - finished creating snapshot 'CSI_snapshot-...' from volume '<wwn>'

Step 2 — clone PVC from source (Bug A)

apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: clone, namespace: vdi}
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: tera-compressed
  volumeMode: Filesystem
  resources: {requests: {storage: 2Gi}}   # exact source size, intentionally
  dataSourceRef:
    kind: PersistentVolumeClaim
    name: src

PVC stays Pending, event:

Warning  ProvisioningFailed  ...  failed to provision volume with StorageClass "tera-compressed":
  rpc error: code = InvalidArgument
  desc = Snapshot function is enabled but not supported with object : csi_pvc-<destination-uid>

Controller log:

DEBUG csi_controller_server.py:CreateVolume:107 - volume was not found. creating a new volume with parameters: {... 'virt_snap_func': 'true' ...}
ERROR exception_handler.py:handle_exception:35 - Snapshot function is enabled but not supported with object : csi_pvc-<destination-uid>
controllers.array_action.errors.VirtSnapshotFunctionNotSupportedMessage: Snapshot function is enabled but not supported with object : csi_pvc-<destination-uid>

Source of the check is in create_volume (array_mediator_svc.py ~L934):

if is_virt_snap_func and source_ids:
    if self._is_vdisk_support_addsnapshot(source_ids.uid):
        self._create_cli_volume_from_source(...)
    else:
        raise array_errors.VirtSnapshotFunctionNotSupportedMessage(name)

Two problems:

1. Misleading message. The unsupported object is the source (source_ids.uid), not the destination name being created. VirtSnapshotFunctionNotSupportedMessage(source_ids.uid) would be accurate.
2. No fallback. A legacy FlashCopy clone (the same mkfcmap path used by CreateSnapshot) would satisfy the request. Falling back when _is_addsnapshot_supported() is False would let virt_snap_func: "true" StorageClasses keep working on older firmware instead of silently breaking all clone operations.

Step 3 — restore PVC from snapshot (Bug B — the AttributeError)

apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: restore, namespace: vdi}
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: tera-compressed
  volumeMode: Filesystem
  resources: {requests: {storage: 2Gi}}
  dataSourceRef:
    apiGroup: snapshot.storage.k8s.io
    kind: VolumeSnapshot
    name: snap

PVC stays Pending, event:

Warning  ProvisioningFailed  ...  failed to provision volume with StorageClass "tera-compressed":
  rpc error: code = Internal
  desc = 'Proxy' object has no attribute 'lsvolumesnapshot'

Controller log:

ERROR exception_handler.py:handle_exception:35 - 'Proxy' object has no attribute 'lsvolumesnapshot'
Traceback (most recent call last):
  File "/driver/controllers/array_action/array_mediator_svc.py", line 2039, in _lsvolumesnapshot
    return self.client.svcinfo.lsvolumesnapshot(**kwargs).as_single_element
AttributeError: 'Proxy' object has no attribute 'lsvolumesnapshot'

Call chain (HEAD = 1ebeb4a):

1. csi_controller_server.CreateVolume computes use_snap_object = bool(virt_snap_func and not partition_name).

2. With use_snap_object=True and source_type == "snapshot", calls array_mediator.get_object_by_id(source_id, source_type, use_snap_object).

3. get_object_by_id (array_mediator_svc.py ~L994):

   if is_virt_snap_func and object_type == controller_settings.SNAPSHOT_TYPE_NAME:
       cli_snapshot = self._get_cli_snapshot_by_id(object_id)        # ← no capability gate
       ...

4. _get_cli_snapshot_by_id → _lsvolumesnapshot → self.client.svcinfo.lsvolumesnapshot(...) → AttributeError.

Contrast with get_snapshot (array_mediator_svc.py ~L975) which does gate:

def get_snapshot(self, volume_id, snapshot_name, pool, is_virt_snap_func):
    if is_virt_snap_func:
        if self._is_addsnapshot_supported():
            cli_snapshot = self._get_cli_snapshot_by_name(snapshot_name)
            ...
        raise array_errors.VirtSnapshotFunctionNotSupportedMessage(volume_id)

get_object_by_id and create_snapshot's branch are missing the same guard.

Suggested fix (proposed patch, untested upstream)

--- a/controllers/array_action/array_mediator_svc.py
+++ b/controllers/array_action/array_mediator_svc.py
@@ -994,6 +994,8 @@ class SVCArrayMediator(ArrayMediatorAbstract, VolumeGroupInterface):

     def get_object_by_id(self, object_id, object_type, is_virt_snap_func=False):
         if is_virt_snap_func and object_type == controller_settings.SNAPSHOT_TYPE_NAME:
+            if not self._is_addsnapshot_supported():
+                raise array_errors.VirtSnapshotFunctionNotSupportedMessage(object_id)
             cli_snapshot = self._get_cli_snapshot_by_id(object_id)
             if not cli_snapshot:
                 return None

That converts the uncaught AttributeError (gRPC Internal, retry storm) into a deterministic InvalidArgument, matching how get_snapshot already handles the same case.

A more user-friendly fix is to fall back to the legacy lookup path (_get_cli_volume_by_wwn) when _is_addsnapshot_supported() is False, since the snapshot that exists on the array is a legacy FlashCopy target (created by the same driver, same release, against the same SC). That would let users keep virt_snap_func: "true" on the SC and have clones / snapshot restores actually work via mkfcmap.

A small auxiliary fix for Bug A:

--- a/controllers/array_action/array_mediator_svc.py
+++ b/controllers/array_action/array_mediator_svc.py
@@ -940,7 +940,7 @@ class SVCArrayMediator(ArrayMediatorAbstract, VolumeGroupInterface):
             if self._is_vdisk_support_addsnapshot(source_ids.uid):
                 self._create_cli_volume_from_source(name, pool, io_group, volume_group, source_ids,
                                                     source_type)
             else:
-                raise array_errors.VirtSnapshotFunctionNotSupportedMessage(name)
+                raise array_errors.VirtSnapshotFunctionNotSupportedMessage(source_ids.uid)

(Cosmetic — accurately blames the source.)

Workaround (in case anyone else hits this)

Provision a parallel StorageClass without virt_snap_func: "true". PVC→PVC clones via dataSourceRef: PersistentVolumeClaim then take the legacy FlashCopy path and bind in ~5–7 s. VolumeSnapshot creation still works on the virt_snap_func: "true" SC; only CreateVolume-from-snapshot is broken on this firmware.

Related

• 1.13 create a VM #844 — same firmware class, different traceback (AttributeError: volume_group_name from _get_partition_name_of_cli_volume). Same root cause class: response objects don't carry the fields the driver assumes.
• rpc error: code = Internal desc = an internal exception occurred : 'Proxy' object has no attribute 'mkvolume' #315 / PR CSI-2915 add svc version check #435 — earlier 'Proxy' object has no attribute … (mkvolume) was solved by a firmware-version check at connect time. The same pattern would prevent this issue.