Injection status when container does not exist

[rimbesat]

Hi team!

Just wanted to share an issue we face with how we implement the controller. This particular scenario happens if we input a manifest that specifies a container name that does not exist in that pod.

The experiment is defined as the following:

apiVersion: chaos.datadoghq.com/v1beta1
kind: Disruption
metadata:
  name: cpu-pressure-pod
  namespace: chaos-engineering-framework
  annotations:
    chaos.datadoghq.com/environment: "dev"
spec:
  level: pod
  duration: 2m
  selector:
    app.kubernetes.io/name: "fault-injection-showcase"
    app: "fault-injection-showcase--plugin-master"
  containers: 
    - "fault-injection-showcase-does-not-exist" 
  count: 100%
  cpuPressure: 
    count: 100% 
  staticTargeting: true

When running this we are inspecting the injectionStatus field to determine the status of the experiment and when the experiment ends. This is used in our CI/CD platform and helps us feedback to the user. However, this field does not exist at the start of the experiment and still does not exist when the experiment duration completes, which is leaving us in a waiting state for the injectionStatus to update.

(start)

(end)

For reference, this is the behaviour of 7.13.1, where at the start of the experiment the field does not exist, and at the end of the experiment the field does exist and states PreviouslyInjected which at least allows us to determine the end of the disruption

(start)

(end)

Ideally, we would like this field to be initialised in a NotInjected state, or any other suggestions are welcome.

[ptnapoleon]

I'll take a look today!

[ptnapoleon]

While we can just add a redundant statusUpdate that will do so, I'm looking into why we aren't notifying the user of this error, and running into a more complex situation than expected. Seems our unwrapping of the type DisruptionError doesn't work as expected

[ptnapoleon]

I've resolved the issue we had with notifying the user, but we're still discussing if we'd prefer to solve this with an extra update of the status, or just by defaulting to NotInjected as soon as the disruption is created

[ptnapoleon]

https://github.com/DataDog/chaos-controller/pull/758/files

[rimbesat]

Thank you Philip!!