Roll back

Two mechanisms return a machine to a version that works. If a new version fails its first boot, the machine returns to the previous version without help. You roll the fleet back deliberately when you point the Cluster at an earlier version.

The automatic fallback

Every machine keeps two boot slots, A and B. The machine runs from one slot and writes downloaded releases into the other. An upgrade reboots into the new slot one time only, as a trial:

• If the new kernel panics, the machine resets, and the firmware boots the proven slot. No software is involved.
• If the new version boots but does not rejoin the cluster in ten minutes, a watchdog reboots the machine, and the machine starts again on the proven slot.

In both cases, the machine serves on the version it ran before. Its phase shows Blocked, its conditions show RejectedLastBoot, and status.boot.systemRejection records what happened. The rejection stays until you point spec.version at a different version, so the machine does not try the failed version again.

A bad release is never published again. To correct a bad release, publish a release with the next serial number, add it to the catalog, and point spec.version at it.

Deliberate rollback

To move the fleet back to an earlier release, point spec.version at that release:

kubectl edit cluster

The version must still be in spec.releases.catalog. This is a reason to keep the old entries. The rollout is the same as for an upgrade: each machine downloads the older release into its inactive slot, verifies it, and reboots on its granted turn, one machine at a time. The cluster continues to serve during the rollout.