[IMP] runbot: allow to link a child to multiple builds

[Xavier-Do]

Currently, a build can have only one parent.
This means that if a build generates multiple sub-builds that could actually be shared across multiple branches/batches of the same branch, they will systematically be recomputed.

Most of the time this is not an issue, since — as a simple example — the parallel testing children depend on the parent's installed database, so they are uniquely linked to the parent. The de-duplication is made at the parent level, by the batch slot (one build can be referenced in multiple batches).

For upgrades, this is a little different since the latest refactoring. A parent build will just create children with specific parameters (source database and treehash, target treehash, upgrade treehash).

It is possible in theory to have multiple sub-builds having the same parameters. A simple example is when rewording a commit, or when starting an upgrade stable build on a branch without an upgrade branch, when a staging.master batch reaches master and only a few source databases were updated...

The current hacky mechanism will check, when creating the child, if another build with the same params exists, and instantly mark the child as done with the same result. Unfortunately it can only work if the other build is finished, meaning that force-pushing a branch with a simple reword will always restart the upgrade build (and kill the previous one).

The intuition is to link the builds together, so that the new build can copy the state of the other existing one. This is similar to the duplicate id mechanism that existed years ago, but that relation was confusing and complex since the duplicate was a special case, and it was removed in favour of batch.slot, which is a lot easier to understand and maintain and avoids special cases.

A closer solution to batch slots is to replace the parent_id relationship and have a many2many between children and parents. Since one of these links may need to be orphaned, an intermediate model (build link) replaces the many2many in order to hold the link metadata.

The way to compute the globals (state and result) is simple to adapt, since we "just" need to add the linked builds to the child builds in the compute.

The way to choose a build to link is not too complex, since it is similar to the previous similar_build_result mechanism, but it still needs some thought (should we link an orphaned result, a red result, a killed result, ...?).

The params should 100% define the build to make it work, so the parent_id is now stored as a param in some cases

The more complex questions concern the interface, child rebuild, build gc (autokill), manual kill, ...

Is it allowed to kill a child linked to multiple other builds?

If the build is exclusively contained in the same branch, why not. But the only reason to do that would be to be able to rebuild the whole parent. So having a way to mark a link as killed (unlinked) could be needed. Once all links to a build are in this state, we could kill the build. To avoid this flag, a simpler solution could be to allow the rebuild of the parent in all cases (use the orphan flag), removing the need to kill it manually and leaving this responsibility to the build gc. This is actually something that could be applied to all builds, since waiting for a child's effective kill when we want to rebuild is not needed. The parent should be marked kilable on rebuild, and a child should inherit that kill able flag (only if all linked parent are killable).

So a simple solution could be to forbid killing a build if it is linked to multiple parents, but still allow a rebuild of the parent (or of the current build).
For admin, killing is needed, but a wizard showing all linked parents and asking for confirmation could be a good solution.

What about the rebuild?

If a build is red and linked to many parents, it may make sense for a rebuild to be linked to all parents. If green, I'm not sure, since an invalid user operation could make a master branch and all linked dev branches pending again.

The simplest route would be to always do it on all links, and maybe display a wizard in all cases explaining what would be accomplish and chose a strategy: all links, same branch link (if a single branch on the parent build), only current parent.
Note also that we may not act on a link that is orphaned since it was most likely already rebuild and has no effect on parent. We only want to re-link it to build that will be impacted by the rebuild.

How to define the most relevant parent

This is only possible if we have the batch in the url, without that the breadcrumb cannot work as well as the rebuild/kill logic specific to a parent. Having the parent in the url could be handy too but may complexify the routes while the batch should in theory be sufficient to retrace the correct path. Batch->parent->child. If a parent was rebuild, we just need to take the most recent one.

How to define when to kill a linked child build?

Waiting until all parents are killable looks like a good solution, but we could miss a window where the build is not yet linked to the new upgrade build. So a build has the potential to be linked again if a new batch is preparing, or has a build with the same config as the parent that is not done yet. So keeping them looks like a good idea. On the other hand, keeping it may also keep slots busy, preventing the parent from running (the parent should have priority, to make this issue less visible and allow for faster links).

Rebuild of parent

We may only link existing build if parent is a not rebuild? Or maybe only link green ones?
A rebuild of the parent will be most likely by laziness to rebuild the corresponding children, or to regenerate the correct parameters for a child if the logic changes. So linking still make sense, but it could be confusing. The simplest route here would be to relink in all case and we will see later.

parent_path

All logic around the parent path to find all children will likely be broken , to check:

• top_parent:maybe that having all top parents could be needed here in some cases
• ancestors
• all_children_ids
• _add_child max depth
• _ask_kill (this one is actually to check from the previous points)
• load_info
• build errors management

Concurrency

It is still possible that two parents create params and/or builds at the same in concurrent transaction.
We can ignore this case, saying that it is not a big deal since most of the time a staging or master build was created before any other build, but for future cross version upgrade it could be more frequent (branch in 17 and 18 linked together and testing 17->18 upgrade) So having a mechanism to ensure that params are unique and maybe first build for a params is unique could be great. (or build not in done state for a param could be unique but this complexify the not kill before rebuild)

General checklist

• Make parent priority on runbot, but not the child
• Implement a nice display
• Add test for states computation
• Add test build gc
• Add test for manual kill
• Manage kill case
• Manage rebuild case
• Manage automatic gc case