[RFC]: v1beta2 schema - OscCluster

[jfbus]

Problem to Solve

The v1beta1 schema is cluttered with redundant information and contains a lot of deprecated/unused attributes.

Here are some simplifications that could be applied:

• Roles where introduced to manage references between resources. In v1beta1, names are both names (as displayed in Cockpit) and references. Names could be dropped (except in securityGroups, purely for documentation purposes). Name tags would be 100% auto-generated (e.g. <Resource type> for <cluster name>), and references would only be role-based.
• NAT Services could be 100% autogenerated (1 NAT service per subnet having the nat role)
• Disabling NAT Services could be a possibility (e.g. when a firewall/WAF appliance is used)
• The network attribute could be dropped, and all subattributes moved up 1 level,
• Replace disable with an object, to match useExisting,
• To simplify SG rules, a more compact <protocol>[/<fromPort>:<toPort>] syntax could be used (eg tcp/8080:8088 = all tcp ports from 8080 to 8088, tcp = all tcp ports from 1 to 65535)

Some additional features are discussed as comments below...

Note: the v1beta1 schema is still being supported (as long as roles are properly set), and you will still be able to write/read v1beta1 schemas, even if the resources are stored and reconciled as v1beta2.

Proposed Solution

Here is a proposal for a new OscCluster schema:

credentials:
    fromFile: <string>
    fromSecret: <string>
    profile: <string>
allowFromIPRanges:
- <cidr>
allowToIPRanges:
- <cidr>
bastion:
  enable: <bool>
  vm: <see VM definition>
useExisting:
  net: <bool>
  securityGroups: <bool>
disable:
  internet: <bool>
  loadBalancer: <bool>
subregions:
- <string>
net:
  ipRange: <cidr>
  resourceId: <id> # The Id of the Net to reuse (if useExisting.net is set)
subnets:
  roles:
  - <role>
  subregion: <cidr>
  ipRange: <cidr>
  resourceId: <id> # The id of the Subnet to reuse (if useExisting.net is set)
natPublicIpPool: <string>
netAccessPoints:
- <service id>
netPeering:
  enable: <bool>
  managementAccountId: <id>
  managementCredentials:
    fromFile: <string>
    fromSecret: <string>
    profile: <string>
  managementNetId: <id>
  managementSubnetId: <id>
routeTables:
- role: <role>
  routes:
  - destination: <cidr>
    targetType: <gateway|nat-service>
    targetRole: <custom role>
    subregion: <string>
securityGroups:
- name: <name>
  authoritative: <bool>
  resourceId: <id> # When useExisting.securityGroup is set, the id of an existing securityGroup to use.
  roles:
  - <role>
  rules:
  - name: <string>
    flow: <string>
    ports:
    - <string>
    ipRanges:
    - <cidr>
additionalSecurityRules:
- roles:
  - <role>
  rules:
  - name: <string>
    flow: <string>
    ports:
    - <string>
    ipRanges:
    - <cidr>
loadBalancer:
  healthCheck:
    port: <int>
    protocol: <string>
    checkinterval: <int>
    timeout: <int>
    healthythreshold: <int>
    unhealthythreshold: <int>
  listener:
    backendport: <int>
    backendprotocol: <string>
    loadbalancerport: <int>
    loadbalancerprotocol: <string>
    loadbalancername: <string>
    loadbalancertype: <string>
reconciliationRules:
- appliesTo:
  - <string>
  mode: <onChange|always|random>
  reconciliationChance: <int>

Migration

• CAPOSC v1.x had a dual role/name reference management. CAPOSC will require roles, and roles need to be added to existing resources

Environment Details

- Kubernetes version (`kubectl version`):
- OS (`/etc/os-release`):
- Kernel (`uname -a`):
- cluster-api-provider-outscale version:
- cluster-api version:
- Install tools:
- Kubernetes distribution:
- Kubernetes distribution version:

[jfbus]

Drop names from resources & use autogenerated Name tags (e.g. <Resource type> for <cluster name>)

References between resources will purely be based on roles.

A description attribute may replace name to store a description of each resource for documentation purposes. description has no impact on IaaS resources.

👍 to drop, 👎 to keep, 👀 to abstain

[jfbus]

Remove NAT service spec and use autogenerated NAT services in every subnet tagged "nat"
Remove Internet service spec

Rationale: there is nothing to configure except the NAT service subregion, inheritable from the subnet, and names, which can be autogenerated.

👍 to drop, 👎 to keep as is, 👀 to abstain

[jfbus]

Allow disabling of NAT Services

disable:
  nat: true

Rationale: a WAF ou a firewall appliance might handle NAT.

Note: this is useless if we switch to fully automatic NAT services based on roles: if no subnet have the nat role, no NAT services will be created.

👍 to add, 👎 to keep as is, 👀 to abstain

[jfbus]

Replace disable with an object, to match useExisting.

Current:

useExisting:
  net: <bool>
  securityGroups: <bool>
disable:
- internet

Proposal:

useExisting:
  net: <bool>
  securityGroups: <bool>
disable:
  internet: <bool>
  loadBalancer: <bool>

Rationale: coherence

👍 to change, 👎 to keep as is, 👀 to abstain

[jfbus]

Add a simplified protocol/port definition for security group rules and allow a list of ports:
Current:

- name: <string>
  flow: Inbound
  ipProtocol: tcp
  fromPortRange: 8080
  toPortRange: 8088
  ipRanges:
  - 1.2.3.4/32
- name: <string>
  flow: Inbound
  ipProtocol: tcp
  fromPortRange: 22
  toPortRange: 22
  ipRanges:
  - 1.2.3.4/32

Proposal:

- name: <string>
  flow: Inbound
  ports:
  - tcp/8080:8088
  - tcp/22
  ipRanges:
  - 1.2.3.4/32

The syntax for ports is : <protocol>[/<fromPort>:<toPort>] (e.g. tcp/8080:8088 = all tcp ports from 8080 to 8088, tcp = all tcp ports from 1 to 65535)

Rationale: there may be a lot of rules, and the new syntax would significantly reduce the number of lines used (improving readability and maintenability).

👍 to change, 👎 to keep as is, 👀 to abstain

[jfbus]

Allow manual creation of multiple nets, with automatically created net peerings between nets.
Current:

net:
  ipRange: <cidr>
  resourceId: <id> # The Id of the Net to reuse (if useExisting.net is set)
subnets:
  roles:
  - <role>
  subregion: <cidr>
  ipRange: <cidr>
  resourceId: <id> # The id of the Subnet to reuse (if useExisting.net is set)
[ and all other attributes ]

Proposal:

networks:
- net:
    ipRange: <cidr>
    resourceId: <id> # The Id of the Net to reuse (if useExisting.net is set)
  subnets:
    roles:
    - <role>
    subregion: <cidr>
    ipRange: <cidr>
    resourceId: <id> # The id of the Subnet to reuse (if useExisting.net is set)
  [ and all other attributes ]

subnets will need to use custom roles (e.g. worker-a, worker-b), to allow linking resources to the right net.

Rationale: some network intensive applications might suffer from bottlenecks, splitting nodes on multiple VPCs would improve throughput

👍 to change, 👎 to keep as is, 👀 to abstain

[jfbus]

Add tags to resources

Proposal 1: A global definition
The same tags are added to every resources

tags:
  <key>:<value>

👍 to add, 👎 to keep as is, 👀 to abstain

[jfbus]

Add tags to resources

Proposal 2: A different tag for each resource:

net:
  [ ... ]
  tags:
    <key>:<value>
subnets:
- name: ...
    tags:
      <key>:<value>
[ etc. ]

👍 to add, 👎 to keep as is, 👀 to abstain

[jfbus]

Allow creation of multiple support VMs.
Current:

bastion:
  enable: <bool>
  vm: <see VM definition>

Proposal:

vms:
- role: bastion
  [ other vm attributes, see the discussion about the new VM schema>
- role: waf
  [ ... ]

Rationale: this would allow deploying appliances (provided they can be deployed using a standard VM definition aka a VM based on an OMI, configured by cloud-init)

👍 to add, 👎 to keep as is, 👀 to abstain

[jfbus]

Add routes to NICs.

This allows routing trafic though VMs.

We could target a role:

routes:
  - destination: <cidr>
    targetRole: waf
    subregion: <string>

Rationale: If we allow deployment of WAF/firewall VMs using CAPOSC, replacing NAT services, we may need to add routes to those VMs.

👍 to add, 👎 to keep as is, 👀 to abstain