Skip to content

Preserve node_type constraints in control-plane scheduling #21

Description

@artemnikitin

Summary

Carry workload node_type requirements into desired state and enforce them against registered node labels during control-plane scheduling.

The enricher currently groups services by node_type, but the events role flattens all enriched node configs into a plain service list. Registered nodes retain labels, yet the scheduler receives only node ID, vCPU, and memory capacity.

Therefore, node_type affects direct per-label config generation but is not a placement constraint in the control-plane flow.

Current behavior

Relevant paths:

  • internal/enricher/enricher.go: groups services by ServiceSpec.NodeType.
  • internal/controlplane/events.go: flattenNodeConfigs discards the group/node type.
  • internal/controlplane/state.go: registry node records contain Labels.
  • internal/controlplane/controller.go: active-node discovery does not pass labels to the scheduler.
  • internal/scheduler/scheduler.go: scheduler nodes contain only ID, vCPU, and memory.

Examples describe node_type as selecting general-purpose, compute, backend, or other node groups, so current control-plane behavior can place workloads on incompatible nodes.

Impact

  • Architecture-specific workloads can be scheduled to the wrong CPU architecture.
  • Compute-, memory-, storage-, or network-specialized placement is not enforced.
  • Deployment examples imply stronger placement guarantees than the runtime provides.
  • Tenant-dedicated or compliance-specific node groups cannot be expressed safely.
  • A future persistent local-volume implementation needs durable hard placement constraints.

Proposed direction

  • Preserve the required node type in the normalized desired service model.
  • Include registered node labels in scheduler.Node.
  • Filter eligible nodes by required labels before capacity bin-packing.
  • Keep hard requirements separate from preferences such as current anti-affinity.
  • Produce a clear unschedulable reason when no matching node has sufficient capacity.
  • Define compatibility for existing desired-state revisions and direct Git mode.

A minimal first version can map the existing single node_type value to a required node label. A later schema can generalize this to selectors and preferred constraints.

Acceptance criteria

  • A service with node_type: compute is scheduled only to a ready node advertising the matching label.
  • Nodes with sufficient capacity but missing the required label are rejected.
  • Existing placement is not preserved if the node no longer satisfies the hard constraint.
  • Scheduling failure identifies the service and missing or incompatible constraint.
  • Services without new generalized selector fields retain documented compatibility.
  • Tests cover matching labels, no matches, label removal, capacity interaction, and deterministic selection.
  • Configuration and architecture documentation explain hard requirements versus preferences.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions