Introduce: machine_(create|initialize|join|drain|delete)_duration_seconds

[elankath]

How to categorize this issue?

/area control-plane
/area metering
/kind enhancement
/priority 2

What would you like to be added:

We would like to add the following new Prometheus Gauge metrics for the MCM

• machine_create_duration_seconds
• machine_initialize_duration_seconds
• machine_join_duration_seconds
• machine_drain_duration_seconds
• machine_delete_duration_seconds

These will also need to be exposed on gardener/gardener for prometheus server collection.

Why is this needed:

The gardener/dependency-watchdog will leverage these metrics to adjust the machine_creation_time for shoot clusters of the seed.

[elankath]

Following metrics are agreed to be introduced after team discussion in pkg/util/provider/metrics/metrics.go

	// MachineCreateDurationSeconds is the Prometheus gauge metric representing the time duration
	// in seconds to create a Machine of a MachineDeployment.
	MachineCreateDurationSeconds = prometheus.NewGaugeVec(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: machineSubsystem,
		Name:      "machine_create_duration_seconds",
		Help:      "Duration in seconds to create a Machine of a MachineDeployment.",
	}, []string{"name", "namespace", "machine_deployment"})

	// MachineInitializeDurationSeconds is the Prometheus gauge metric representing the time duration
	// in seconds to initialize a Machine of a MachineDeployment.
	MachineInitializeDurationSeconds = prometheus.NewGaugeVec(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: machineSubsystem,
		Name:      "machine_initialize_duration_seconds",
		Help:      "Duration in seconds to initialize a Machine of a MachineDeployment.",
	}, []string{"name", "namespace", "machine_deployment"})

	// MachineJoinDurationSeconds is the Prometheus gauge metric representing the time duration
	// in seconds for a Machine of a MachineDeployment to join the cluster.
	MachineJoinDurationSeconds = prometheus.NewGaugeVec(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: machineSubsystem,
		Name:      "machine_join_duration_seconds",
		Help:      "Duration in seconds for a Machine of a MachineDeployment to join the cluster",
	}, []string{"name", "namespace", "machine_deployment"})

	// MachineDrainDurationSeconds is the Prometheus gauge metric representing the time duration
	// in seconds to drain a Machine of a MachineDeployment.
	MachineDrainDurationSeconds = prometheus.NewGaugeVec(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: machineSubsystem,
		Name:      "machine_drain_duration_seconds",
		Help:      "Duration in seconds to drain a Machine of a MachineDeployment.",
	}, []string{"name", "namespace", "machine_deployment"})

	// MachineDeleteDurationSeconds is the Prometheus gauge metric representing the time duration
	// in seconds to delete a Machine for a MachineDeployment.
	MachineDeleteDurationSeconds = prometheus.NewGaugeVec(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: machineSubsystem,
		Name:      "machine_delete_duration_seconds",
		Help:      "Duration in seconds to delete a Machine of a MachineDeployment.",
	}, []string{"name", "namespace", "machine_deployment"})