Skip to content

Hold the API server cutover until the new API server is ready - #5139

Open
caseydavenport wants to merge 2 commits into
tigera:masterfrom
caseydavenport:casey-apiserver-readiness-gate
Open

Hold the API server cutover until the new API server is ready#5139
caseydavenport wants to merge 2 commits into
tigera:masterfrom
caseydavenport:casey-apiserver-readiness-gate

Conversation

@caseydavenport

@caseydavenport caseydavenport commented Aug 4, 2026

Copy link
Copy Markdown
Member

Description

Alternative fix for EV-6821, opened alongside #5103 so the two can be compared.

On a direct upgrade the operator creates the new API server and repoints the projectcalico.org/v3 APIService at it in the same pass, without checking that the new pod can serve. A leftover default-deny policy denies that pod, and that policy can only be deleted through the API server it denies.

This PR holds the repoint - along with the removal of the old API server's namespace and RBAC - until the new API server reports ready. The old one keeps serving until then, so the installation controller's existing delete of the policy lands and the upgrade continues.

Nothing here knows the name of the deprecated policy, so any other reason the new pod fails to come up also leaves a working API server in place. Already-deadlocked clusters aren't repaired. Not validated on a live cluster yet.

Release Note

Fixed a deadlock on upgrade where the Calico API server was moved before a deprecated policy blocking it was removed, leaving the projectcalico.org/v3 API permanently unavailable.

For PR author

  • Tests for change.
  • If changing pkg/apis/, run make gen-files - no API changes.
  • If changing versions, run make gen-versions - no version changes.

The v3.projectcalico.org APIService is only repointed once the API server
it would point at reports ready replicas.
Comment thread pkg/controller/apiserver/cutover.go Outdated
Comment thread pkg/controller/apiserver/cutover.go Outdated
An error reading the APIService or the Deployment is not evidence that the
cutover is safe.
@caseydavenport
caseydavenport marked this pull request as ready for review August 5, 2026 14:23
@caseydavenport
caseydavenport requested a review from a team as a code owner August 5, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants