Migration to a new server #6

Activity

1. alessandrocornacchia

   added

   enhancementNew feature or requestNew feature or request

   documentationImprovements or additions to documentationImprovements or additions to documentation

   on Jun 10, 2026
2. alessandrocornacchia commented on Jun 10, 2026

   

   alessandrocornacchia

   on Jun 10, 2026

   Member

   More actions

   Migration Notes

   Current VM inventory

   All VMs and their configurations live in:

   • host_vars/restsrv01.yaml — list of all VMs
   • host_vars/restvm-*.yaml / host_vars/restsrv01-smartdata*.yaml — per-VM config (dataplane IPs, users)

   VM	Users	Dataplane IPs
   restsrv01-smartdata01	zhihaow	10.10.0.11–12 (SR-IOV, ens5f0 VFs 0–7)
   restsrv01-smartdata02	zhihaow	10.10.0.17–18 (SR-IOV, ens5f1 VFs 8–9)
   restvm-dauin-01	asacco, aangi	10.10.0.13–14
   restvm-mamanj-01	mamanj	10.10.0.19–20
   restvm-mspina-01	mspina	10.10.0.21–22
   restvm-mspina-02	mspina	10.10.0.23–24

   P4 SDE configuration (per VM)

   Defined in group_vars/all.yaml and applied by roles/bootstrap/:

   • SDE path: /opt/p4-sde/bf-sde-9.13.2
   • Kernel module: bf_kdrv.ko (loaded via roles/tofino-sde/)
   • Convenience aliases: p4, sde, p4-build, tfm, iftofinoup/down, switch-intel
   • Sudoers for P4 ops: see roles/tofino-sde/tasks/main.yaml

   P4 switch control plane settings

   • host_vars/rest-bfsw01.yaml and host_vars/rest-bfsw02.yaml — control CPU IPs (10.10.0.2–5)
   • Tenant accounts on the switches are managed by playbooks/bootstrap-tofino.yaml
   • Authorized users: group_vars/all.yaml → restart_users

   Migration with CrownLabs VMs

   Since VMs will be created via CrownLabs (not KVM), skip the kvm_provision role and run only:

   # Bootstrap networking + SDE on each new CrownLabs VM
   ansible-playbook playbooks/bootstrap-vms.yaml -i inventory.yaml -K

   Key things to keep in mind:

   1. No ProxyCommand needed if CrownLabs VMs are directly reachable (unlike the current restsrv01 bastion setup in inventory.yaml)
   2. Reuse or reassign dataplane IPs — each VM still needs 2 consecutive IPs from 10.10.0.0/24; see the IP table in playbooks/README.md
   3. P4 switch user accounts already exist — no switch re-bootstrap needed unless switches were reset
   4. SR-IOV is server-specific — restsrv01-smartdata01/02 use SR-IOV PCI passthrough defined in host_vars/restsrv01.yaml; CrownLabs VMs will use standard bridged networking instead
   5. Base VM image must have P4 SDE 9.13.2 pre-installed at /opt/p4-sde/bf-sde-9.13.2 (same as the existing ubuntu20.04-p4-sde image used for KVM VMs)