Note

Argo Rollouts: canary and blue-green with metric-driven auto-rollback

A standard RollingUpdate declares success once pods pass the readiness probe. Argo Rollouts turns the release into a testable hypothesis: stepped canary traffic, blue-green with preview, PromQL-driven AnalysisTemplates and SLO burn rate as an automatic rollback.

kubectl rollout status replied «successfully rolled out» — but what does that prove? Only that the new pods passed their readiness probe. The probe checks «the process is up», not «the service works»: a regression that lifts the error rate from 0.1 % to 2 % is invisible to a standard RollingUpdate — it will finish replacing pods and report success. The incident starts later, when 100 % of the traffic already sits on the new version. Progressive delivery flips the model: a release is a hypothesis — «the new version is no worse than the old one» — traffic arrives in slices, metrics test the hypothesis, and a failure rolls the release back with no human in the loop. In the CNCF stack this layer belongs to Argo Rollouts.

Rollout instead of Deployment

Argo Rollouts is a controller (installed once per cluster) plus the Rollout CRD: a replacement for the Deployment with the same pod template and one extra field, strategyblueGreen or canary. Nothing breaks in the GitOps loop: CI builds the image and bumps the tag in the gitops repository, ArgoCD syncs the manifests, and Rollouts decides how the new version receives traffic. The separation of concerns stays clean: GitOps answers «what are we deploying», progressive delivery answers «how fast and with what safety net».

Blue-green: instant rollback at double the price

The blueGreen strategy keeps two full versions running at once: activeService routes production traffic to the stable pods, previewService to the new ones. With autoPromotionEnabled: false promotion becomes a manual gate: the new version can be smoke-tested through the preview address while production traffic never sees it. Promotion is a Service switch; rollback is the same switch in reverse — seconds either way. The price: double infrastructure for the duration of the release, and extra care with database migrations — both versions look at the same schema.

Canary: traffic in steps, verdict by PromQL

The canary strategy describes the release as a sequence of steps: setWeight: 20 → pause → 5080 → full promotion. The decision maker is not the pause but the analysis: the analysis block attaches an AnalysisTemplate — a set of PromQL queries with thresholds that the controller evaluates at every step from the one you specify. A failed analysis is an automatic abort: the canary weight is removed, all traffic returns to stable, and the object is marked Degraded. Nobody pages the on-call to decide whether to roll back — the rollback has already happened.

The metrics decide the quality of the gate. Success rate counts only 5xx: 4xx are client errors, and counting them means rolling back releases over bugs on the client side. Latency is compared at the P99, not the average: P50 hides the tail, and users complain precisely about the tail. Both queries usually exist in the cluster already — they are the same SLIs that feed the SLO alerts.

Burn rate: the SLO as a deploy fuse

Wiring in the SLO raises the gate to the next level: the analysis watches not the raw error rate but the speed at which the error budget burns. At a 99.9 % SLO the monthly error budget is 43 minutes; a 10× burn rate eats it in three days. Hence the rule «at 10× — halt deployments»: a single PromQL condition api:burn_rate:ratio_rate1h > 10 inside an AnalysisTemplate turns the SLO from a reporting metric into an automatic fuse. A bad release does not drain the budget to zero, and a healthy one is not blocked out of overcaution: while the budget is intact, a minor error bump is a ticket, not a rollback.

Choosing a strategy, and three traps

The choice is pragmatic. Blue-green — when releases are rare, observability is modest, and instant rollback is critical; you pay with double infrastructure. Canary — when releases are frequent and the metrics are mature: cheaper, more gradual, friendlier to database migrations. A proven promotion pattern across environments: dev — blue-green with manual promotion; staging and prod — canary with automatic promotion.

The traps that catch first adoptions. First: the HPA must target the Rollout (scaleTargetRef with kind: Rollout), not the Deployment — otherwise autoscaling and the rollout fight over replicas. Second: installed via Kustomize plus Helm, ArgoCD shows the CRDs as OutOfSync forever — the API server mutates the schema after apply; the cure is ignoreDifferences on .spec, .status and the annotations. Third: during a canary, secrets live in two versions — canary pods see the new one, stable pods the old one, and a rollback brings the old one back; secret rotation has to account for this in the checklist rather than discover it during an incident.

Progressive delivery is not a luxury reserved for Google-scale platforms. One controller, one CRD and one AnalysisTemplate with two PromQL queries — and the release stops being a leap of faith: the metrics you already collect for alerting start protecting every release before it turns into an incident.

© 2026 axyi.ru · CC BY 4.0