Continuous Deployment Strategies

Why this matters

You can now build a traceable image (module 03) and you know how to make a pipeline apply it (track 07 module 08's kubectl set image). But how the new version replaces the old one is a strategy choice with real consequences: done naively, a bad release takes down all users at once; done well, you can expose a new version to 1% of traffic, watch it, and abort before anyone notices. This module covers the three core deployment strategies — rolling update, blue/green, and canary — conceptually and applied to Kubernetes/AKS Deployments. It's the conceptual groundwork for GitOps (module 05) and progressive delivery (module 06): you can't automate a canary until you understand what a canary is.

Concepts

Recap: the rolling update you already have

Track 07 module 08 used kubectl set image, which triggers Kubernetes' default strategy: the rolling update. Kubernetes replaces old pods with new ones gradually, governed by two Deployment fields: maxUnavailable (how many pods can be down during the roll) and maxSurge (how many extra pods can be created above the desired count). It brings up some new pods, waits for them to pass readiness probes, then terminates some old ones, and repeats until all pods run the new version. kubectl rollout status (track 07 module 08) blocks until this finishes or fails. The rolling update is the baseline every other strategy is compared against.

Rolling update: strengths and the blind spot

Rolling updates are cheap (no extra infrastructure — you reuse the same Deployment's replica budget) and zero-downtime if readiness probes are correct. Their limitation: during the roll, both versions serve real traffic simultaneously and you can't control the ratio precisely — the split is a side effect of how many pods have rolled, not something you set. And once the roll completes, 100% of traffic is on the new version — there's no "hold at 10% and watch" step. A bad version that passes its readiness probe but is subtly broken (returns wrong data, is slow) rolls all the way out and hits everyone. Rolling updates change the pods; they don't give you a traffic control knob. That knob is what blue/green and canary add.

Blue/green: two full environments, instant cutover

Blue/green runs two complete versions side by side: "blue" (current, live) and "green" (new, receiving no user traffic yet). You deploy green in full, test it in isolation (smoke tests against it directly), then flip 100% of traffic from blue to green in one atomic switch — on Kubernetes, by repointing a Service's selector from the blue pods to the green pods. Advantages: the cutover is instant and complete (no mixed-version window), and rollback is instant too — flip the selector back to blue, which is still running. Cost: you run double the pods during the transition (both full environments), and stateful concerns (in-flight sessions, database schema) need care. Blue/green trades resources for a clean, instant, reversible cutover.

Canary: expose a small slice, then widen

A canary release sends a small percentage of real traffic (say 5%) to the new version while the rest stays on the old one, watches the new version's health/metrics, and — only if it looks good — progressively widens: 5% → 25% → 50% → 100%. If metrics degrade at any step, you abort and route everything back to old. It's named after the "canary in a coal mine": a small, sacrificial exposure that warns you before the whole population is affected. Canary gives you the precise traffic knob rolling updates lack, and limits blast radius far better than blue/green's all-at-once flip — at the cost of complexity: you need a way to split traffic by percentage and, ideally, automated analysis of the canary's metrics to decide promotion (module 06's Argo Rollouts). On plain Kubernetes you can approximate a canary crudely with replica-count ratios (below); doing it properly by traffic percentage needs a mesh or ingress controller that supports weighting.

Approximating canary on plain Kubernetes with replica counts

Without a service mesh, you can approximate a traffic split using the fact that a Service load-balances evenly across all its endpoints. Run two Deployments (v1 and v2) selected by the same Service via a shared label, and set their replica counts to the ratio you want: 9 replicas of v1 + 1 of v2 ≈ 10% canary. It's coarse (the granularity is limited by replica count, and it conflates "how much traffic" with "how many pods"), it doesn't do session affinity, and scaling changes the ratio — but it's a real, mesh-free way to see the concept work, and it's the "simple manual traffic-split" approach module 06 contrasts with Argo Rollouts. Precise, pod-count-independent weighting needs an ingress controller (nginx canary annotations) or a mesh (track 13).

Choosing a strategy, and where readiness/rollback fit

No strategy is universally right:

• Rolling — the sensible default for most stateless services; simplest, no extra infra.
• Blue/green — when you need an instant, all-or-nothing, instantly reversible cutover and can afford double resources (e.g. a risky release where you want zero mixed-version time).
• Canary — when blast radius matters most and you can measure the new version's health to gate promotion (the highest-confidence path, and the prerequisite for true continuous deployment from module 00).

Two things every strategy depends on: correct readiness probes (track 03 — without them, all strategies route traffic to pods that aren't ready) and a rollback plan (kubectl rollout undo for rolling; selector flip for blue/green; re-weight to 0% for canary). A deployment strategy without a tested rollback is just a slower way to break production.

Command reference

kubectl for rollouts and the objects that make each strategy work.

Hands-on exercises

Use your AKS cluster (track 07) or a local kind/minikube cluster from track 03 — the Kubernetes objects are identical. Have a demo app image with two visibly-different versions (v1 and v2, e.g. different response strings).

1. Observe a rolling update in slow motion. Deploy demo:v1 with 6 replicas. In one terminal run kubectl get pods -l app=demo -w; in another, kubectl set image deployment/demo demo=<acr>/demo:v2. Watch pods terminate and start in waves. Then inspect spec.strategy.rollingUpdate and note the default maxSurge/ maxUnavailable. Set maxUnavailable: 0 and maxSurge: 1, roll to a v3, and observe the roll is slower but never dips below full capacity.

2. See the mixed-version window. During a roll (slow it with a startup delay or many replicas), curl the Service repeatedly in a loop and observe responses flipping between v1 and v2 — proof that a rolling update serves both versions simultaneously and you don't control the ratio. Note you can't hold it at, say, 20%.

3. Roll back a bad rolling update. Roll to a deliberately broken image (crash-loops on startup). Watch kubectl rollout status fail (track 07 module 08). Recover with kubectl rollout undo deployment/demo, confirm with kubectl rollout status that it's healthy again, and use kubectl rollout history to see the revisions.

4. Blue/green: deploy green alongside blue. Have two Deployments, demo-blue (label version: blue, running v1) and demo-green (label version: green, running v2), plus a Service selecting app: demo, version: blue. Confirm 100% of curls return v1. Smoke-test green directly by port-forwarding to a green pod (not through the Service) and confirm it serves v2.

5. Blue/green: the atomic cutover and instant rollback. Flip the Service selector to version: green (kubectl patch service demo -p '{"spec":{"selector":{"app":"demo","version":"green"}}}'). Confirm curl now returns v2 for every request (no mixed window) and kubectl get endpoints demo lists only green pods. Then simulate a bad green: flip the selector back to blue and confirm instant recovery to v1 — the whole point of keeping blue running.

6. Canary by replica ratio. Make blue and green share the same Service selector (app: demo, no version in the selector) so the Service load-balances across both. Set blue (v1) to 9 replicas and green (v2) to

   1. Loop curl ~50 times and tally responses — roughly 10% should be v2. Increase green to 3 and blue to 7 and confirm the ratio shifts toward ~30%. This is the coarse, mesh-free canary approximation.

7. Feel the limits of the replica-ratio canary. From exercise 6, scale the whole app up (both Deployments) and watch the ratio stay roughly the same but the granularity problem appear: you can't get 5% with only a handful of pods, and an HPA (track 07 module 05) scaling one Deployment would silently change your traffic split. Write two sentences on why this conflation of "pod count" and "traffic share" is exactly what a real traffic-splitting tool (module 06) fixes.

8. Diagnose and fix: the cutover that routed nowhere. You'll reproduce the classic blue/green mistake. Patch the Service selector to version: green but with a typo (version: gren) or before green's pods are Ready. Observe curl now returns connection errors / 503 and kubectl get endpoints demo shows <none> — the Service matches no pods. Diagnose by comparing the Service's spec.selector to the green pods' actual labels (kubectl get pods --show-labels) — the exact selector-must-match-labels rule from track 03. Fix the selector (or wait for readiness), confirm endpoints populate and traffic flows to v2, and note why "the deploy succeeded" (the objects all exist) didn't mean "traffic is flowing."

Independent challenge

No commands given — reason it out from track 03's Services/labels/probes, track 07 module 08's rollout mechanics, module 03's image tagging, and this module's strategies. On a real cluster, take one app and demonstrate all three strategies deploying the same v1→v2 change, then write a short comparison of what each cost you and protected you from. For the rolling update, tune maxSurge/maxUnavailable so capacity never drops and capture the mixed-version window with a curl loop. For blue/green, stand up a full second environment, smoke-test it out-of-band, cut over atomically by Service selector, and prove you can roll back instantly by flipping back. For the canary, approximate a 10%→50%→100% progression using replica ratios across two Deployments behind one Service, tallying traffic at each step, and explicitly note where the approximation is lying to you (pod count vs. traffic share, HPA interference, granularity). Every version must be referenced by an immutable SHA tag (module 03), never latest, and every strategy must have a rollback you actually execute, not just describe. Conclude with a one-paragraph recommendation: for this app, which strategy would you run in production and why — and which module-06 tool would make your chosen strategy safe enough for true continuous deployment.

Common mistakes & troubleshooting

• Assuming a rolling update gives you a traffic knob. It doesn't — the old/new ratio during a roll is a side effect of progress, not something you set, and it always ends at 100% new. For a held partial exposure you need canary/blue-green.
• Blue/green cutover before green is ready. Flipping the selector to pods that aren't Ready (or don't exist) routes traffic to nothing — endpoints shows <none>, users get 503s. Wait for readiness; verify endpoints after the flip.
• Selector/label mismatch on cutover. A typo in the Service selector means it matches no pods (track 03's endpoints=<none> symptom). Always reconcile the Service spec.selector against actual pod labels.
• Forgetting blue/green's double cost. Running two full environments doubles resource use during the transition; on a small cluster this can fail to schedule. Ensure capacity (or scale down blue after a confident cutover).
• Treating the replica-ratio canary as precise. It conflates pod count with traffic share, can't do fine percentages, and breaks the moment an HPA (track 07 module 05) rescales either Deployment. It's a teaching approximation; real canaries weight traffic independently of pod count (module 06).
• No rollback rehearsal. Every strategy needs a tested rollback: rollout undo (rolling), selector-flip-back (blue/green), re-weight to 0 (canary). Discovering your rollback doesn't work during an incident is the worst time.
• Broken readiness probes undermining all of it. If readiness probes pass on unhealthy pods, every strategy happily routes traffic to broken pods. Correct probes (track 03) are the foundation under all three.

Checkpoint quiz

Write down your answer to each question before expanding it — checking without attempting first is the single easiest way to fool yourself into thinking you've learned this.

1. What are maxSurge and maxUnavailable, and how would you set them so a rolling update never drops below full capacity?
2. What can a canary or blue/green deployment do that a rolling update fundamentally cannot?
3. Describe the blue/green cutover on Kubernetes mechanically — what object changes, and why is rollback instant?
4. What's the resource cost of blue/green, and when is it worth paying?
5. How do you approximate a 20% canary on plain Kubernetes with no service mesh, and name two ways that approximation is inaccurate.
6. Why would an HPA (track 07 module 05) sabotage a replica-ratio canary?
7. After a blue/green cutover, curl returns 503 and kubectl get endpoints shows <none>. What's the most likely cause and how do you confirm it?
8. Which strategy is the prerequisite for true continuous deployment (module 00) and why?

Next

05-gitops-and-argocd — stop pushing deploys from the pipeline and let an in-cluster controller pull the desired state from Git instead.