Canary releases for AI models: what changes vs traditional software

A hallucinating language model returns HTTP 200.

A model that's drifted off-distribution responds to every request — it just responds worse. If your canary release monitoring stops at error rates and latency, you have no instrumentation for either of those failures. That's the core problem this article addresses.

Traditional canary releases were built to catch crashes, 5xx errors, and latency spikes. Those are binary failures — they either happen or they don't. AI model degradation doesn't work that way.

Output quality can quietly regress while every infrastructure metric stays green. Fixing that requires a different monitoring layer, different traffic routing logic, and different rollback triggers — not just more dashboards.

This article is for engineers, PMs, and data teams who are shipping AI models to production and want their canary process to actually catch the failures that matter. Here's what you'll learn:

• Why standard canary monitoring creates a false sense of safety for AI models — and what the signal gap actually looks like
• Which output quality metrics to track during a canary, and how to make rollback decisions defensible
• How to structure traffic routing and hold periods around statistical requirements, not just baking time
• How to design rollback triggers that fire on quality degradation, not just error thresholds
• How to treat the canary release as a controlled experiment so every rollout generates evidence, not just a pass/fail result

Each section builds on the last, moving from the conceptual gap to the specific instrumentation, routing, and rollback decisions you'll need to make before your next model release goes live.

Why standard canary release logic breaks down for AI models

If you've shipped software with canary deployments before, the pattern feels reliable: send a small slice of traffic to the new version, watch the dashboards, and either promote or roll back based on what you see. For deterministic software, this works well.

For AI models, it creates a dangerous illusion of safety — you can watch every metric stay green while your model silently degrades in ways that matter enormously to users.

How traditional canary releases work — and what they're designed to catch

The mechanics of a traditional canary release are straightforward. Route somewhere between 1% and 5% of production traffic to the new version, monitor infrastructure signals over a validation window, and either expand the rollout or revert.

Google's SRE practice captures the intent precisely: canary servers exist to "detect dangerous effects from the behavior of the new software under real user traffic."

That definition sounds comprehensive. The problem is what it implicitly assumes about what "dangerous effects" look like. Traditional canaries are optimized for binary failure detection — crashes, HTTP 5xx errors, latency spikes, memory exhaustion, throughput collapse.

These are events that either happen or don't. A bug either causes a measurable failure or it doesn't. The signal is immediate and unambiguous.

This works because deterministic software has a predictable failure signature. When something goes wrong, the infrastructure tells you.

The signal layer traditional canaries rely on

The specific metrics that feed traditional canary dashboards reflect this binary assumption: HTTP error rates, p95 and p99 latency, CPU and memory utilization, crash rates, and request throughput. Each of these is a proxy for a single underlying question: is the system still running correctly?

For infrastructure regressions, these signals are well-calibrated. A bad deployment that introduces a memory leak or a broken API endpoint will show up quickly in these numbers. The monitoring layer and the failure mode are matched.

The structural problem is that these signals say nothing about what the system is producing. They measure the health of the pipe, not the quality of what flows through it.

Where those signals fail for AI models — concrete examples

Consider what a degraded AI model actually looks like at the infrastructure layer. A language model that begins hallucinating returns HTTP 200. Systematic bias in completions for certain demographic inputs runs at normal latency with normal throughput — nothing in the infrastructure layer registers it.

A code generation model whose suggestions compile but contain logical errors passes every uptime check. Drift off-distribution produces the same response pattern as a healthy model; the outputs are just worse.

None of these failure modes produce an error event. None of them spike a latency metric. From the perspective of your canary monitoring dashboard, the deployment looks healthy.

NVIDIA has acknowledged this gap directly in the context of AI model monitoring: "Currently, these detection capabilities for AI models are identical to those used for monitoring any other sensitive data — no detection capability focuses on the unique nature of AI/ML." The default tooling treats an AI model like any other software artifact, which means it's instrumented to catch the wrong class of failures entirely.

It's worth noting that even well-engineered traditional canary systems can miss non-binary failures in deterministic software — the Google 2019 outage is a documented case where canaries didn't catch a configuration side effect before it cascaded.

If that's possible with conventional software, the problem is compounded significantly when the failure mode is probabilistic and gradual rather than discrete.

The conceptual shift required — from infrastructure health to output quality

The gap isn't a tooling deficiency you can patch by adding more infrastructure metrics. It's a category error. Traditional canaries answer the question "is the system healthy?" AI canaries have to answer a different question: "are the outputs good?"

These require different instrumentation entirely. Output quality metrics, user engagement signals, task completion rates, drift metrics, and guardrail violation rates are not derivable from CPU utilization and error logs.

They require a second monitoring layer, sitting above infrastructure observability, purpose-built for evaluating what the model is actually producing and how users are responding to it.

Landon Smith, Head of Post-Training at Character.AI, describes this operational reality directly: "We can compare different modeling techniques from the perspective of our users — guiding our research in the direction that best serves our product." That phrase — from the perspective of our users — identifies exactly the signal layer that traditional canary monitoring cannot provide.

Platforms like GrowthBook are built around this gap, connecting model performance directly to user outcomes rather than infrastructure metrics, and enabling rollback decisions based on quality signals rather than error thresholds. The subsequent sections of this article address how to instrument that signal layer, structure the exposure window, and design rollback triggers that can actually detect the failures that matter.

The metrics that actually matter: monitoring AI output quality during a canary release

When Anthropic's Claude Code team identified a quality regression in early 2025, they confirmed it publicly: "We fixed a Claude Code harness issue that was introduced on 1/26. This was rolled back on 1/28 as soon as we found it." Two days between introduction and detection — at a company with substantial ML infrastructure investment.

The issue wasn't caught by error rates or uptime monitors. It was caught by daily benchmarks specifically instrumented to track output quality.

That gap between "infrastructure healthy" and "model actually degrading" is exactly the problem that output quality metrics exist to close. A model can return HTTP 200 on every request while producing responses that are subtly less accurate, more verbose, increasingly off-distribution, or simply less useful to users.

Infrastructure telemetry has no surface area for any of that. If your canary release monitoring stops at error rates and p99 latency, you are operationally blind to the failure modes that matter most for AI systems. What follows is the metric framework that closes that gap.

Inference latency and throughput — the baseline you still need

Latency and throughput remain necessary, but they're the floor of your monitoring stack, not the ceiling. For generative models specifically, median latency is a misleading signal — output length varies dramatically by request, which means tail latency (p95, p99) is far more meaningful than p50.

A model that handles most requests quickly but occasionally hangs on long-form generation will look fine at the median while degrading user experience at the tail. Track both, treat latency as a guardrail metric, and don't mistake it for a quality signal.

Accuracy and drift signals — detecting what error rates miss

This is the core instrumentation gap. Task-specific accuracy scores, output distribution drift, and hallucination rate proxies all require deliberate instrumentation that sits entirely outside standard observability tooling.

Output distribution drift is particularly easy to overlook: are responses shifting in average length, topic coverage, or tone in ways that correlate with the model version change?

A new model checkpoint might produce responses that are technically coherent but systematically shorter, more hedged, or less complete than the previous version — none of which triggers an alert in your infrastructure layer. The Claude Code incident is instructive here precisely because Anthropic is not a resource-constrained team.

Even with significant investment in ML infrastructure, detecting a quality regression required dedicated daily benchmarks. Teams that haven't built equivalent instrumentation are running canary releases for AI models without the sensors needed to detect the most common failure mode.

User-outcome metrics — where model benchmarks and business reality diverge

Model-level metrics tell you how the model is behaving. User-outcome metrics tell you whether that behavior is actually serving users better. Both layers are required, and neither substitutes for the other — a model can score well on internal accuracy benchmarks while producing outputs users find less useful in practice.

The metrics that operationalize this user-outcome perspective include task completion rate, engagement depth (session length, follow-up query rate, retry rate), and downstream conversion or action rates. Warehouse-native experiment platforms address this layer directly — connecting experiment assignments to behavioral data stored in the data warehouse to surface whether a model change is actually moving user outcomes, not just model benchmarks.

Token efficiency and cost signals

Token cost is both a quality proxy and a direct business metric, and it belongs in your canary monitoring stack for both reasons. Verbose or repetitive outputs inflate token usage without improving user outcomes — so a rising cost-per-successful-task-completion signal often indicates output quality degradation before accuracy metrics catch it.

Raw token count is the wrong unit; cost per successful task completion is the right one.

The right framing treats cost efficiency and quality as jointly optimized, not in tension — a rising cost-per-task signal is evidence of quality degradation, not just a budget concern.

Why rollback decisions require reproducible analysis, not black-box dashboards

The instrumentation architecture matters as much as the metric selection. During a canary, rollback decisions need to be defensible — which means the analysis behind them needs to be reproducible and queryable, not locked in a black-box analytics layer.

Warehouse-native architectures address this directly. GrowthBook runs experiment analysis directly in the customer's data warehouse — Snowflake, BigQuery, Redshift, Databricks, and others — with only aggregates leaving the warehouse. No separate event pipeline, no data duplication, and full reproducibility of every calculation.

That last point matters specifically for canary rollback decisions: when you're deciding whether to halt a model rollout based on a quality metric trend, you need to be able to confirm the numbers independently.

Equally important is the ability to add metrics after a canary begins. Teams frequently discover mid-rollout that they need a signal they didn't anticipate. Merritt Aho, Digital Analytics Lead at Breeze Airways, put it plainly: "Being able to spin up new metrics mid-experiment is a game changer. This was simply never possible before."

For AI canaries specifically, where the failure modes are less predictable than traditional software releases, that flexibility isn't a convenience — it's a requirement.

Incremental traffic routing: structuring the canary exposure window for AI models

The mechanical pattern of traffic splitting in an AI canary release looks identical to what you'd do for any other deployment: route a fraction of requests to the new version, observe it, then decide whether to promote or roll back. The infrastructure primitives are the same.

What's categorically different is how long you need to hold each stage, how you determine when it's safe to advance, and why the sample size requirements for generative AI outputs make traditional "baking period" logic dangerously insufficient.

Initial traffic split mechanics

AWS SageMaker's canary traffic shifting documentation offers a reasonable infrastructure baseline: provision a separate green fleet, route up to 25% of traffic to it initially, and monitor CloudWatch alarms during a baking period. For traditional ML inference endpoints serving deterministic outputs, this is a workable starting point.

For generative AI models, the initial split should typically be considerably lower — the 1–5% range is common practitioner convention — and the reasoning is different from what you'd expect. The concern isn't primarily infrastructure risk.

It's that output quality signals need time to accumulate, and you want to limit user exposure to potentially degraded generative outputs before you have enough signal to evaluate whether degradation is actually occurring. Starting at 25% with a generative model means you've already exposed a quarter of your user base before your quality metrics have reached any meaningful statistical resolution.

The other critical mechanic at this stage is consistent user assignment. A user who hits the canary model on their first request needs to hit it on their second and third requests as well — otherwise your output quality measurements reflect a mix of model versions per user, which corrupts your signal.

Deterministic hashing, where each user is assigned a stable value used to route them to a specific serving path, is the correct implementation pattern here. GrowthBook's SDK uses MurmurHash3 for deterministic hashing, assigning each user a value between 0 and 1 and routing based on configurable ranges, which ensures clean, stable assignment throughout the canary window.

Scaling increments and hold periods

Traditional canary logic is primarily time-gated: hold for N minutes, check whether any alarms fired, promote if clean. This works when the failure mode you're detecting is a crash or a 5xx error — those signals are strong, fast, and unambiguous.

Detecting a 3% regression in output quality scores for a generative model is a fundamentally different statistical problem. The signal is weak, noisy, and slow to accumulate. Advancing the canary on a time-based schedule alone — without validating that your quality metrics have reached statistical significance and show no degradation trend — means you're making a promotion decision on insufficient evidence.

The correct pattern is validation-gated scaling: each increment (say, 1% → 5% → 10% → 25% → 50% → 100%) requires the previous stage's quality metrics to clear defined thresholds before the next increment begins. Elapsed time is a floor, not a ceiling. You don't advance because the clock ran out; you advance because the data cleared the bar.

Compute isolation requirements

AI model serving infrastructure introduces cross-contamination risks that don't exist for stateless application code. If your canary model and your baseline model share the same GPU pool, the same memory cache, or the same request batching queue, they can interfere with each other — a heavily loaded canary endpoint can slow down baseline responses, or a shared memory cache can serve outputs from the wrong model version.

Either scenario corrupts your quality measurements in ways that are hard to diagnose after the fact.

The AWS SageMaker blue/green architecture — where canary and baseline are provisioned as entirely separate fleets — provides the right infrastructure template. Both fleets receive traffic simultaneously during the baking period, but they are isolated at the resource level.

The assignment layer needs to match: canary and baseline users should be routed to separate inference endpoints, not handled as a weighted routing rule on a shared endpoint.

Sample size considerations for generative AI outputs

The asymmetry here is stark. Detecting a server error requires a sample size of one. Detecting a meaningful quality regression in generative outputs may require thousands of interactions, depending on the baseline variance of your quality metrics and the effect size you're trying to detect.

The minimum number of interactions you need to observe should be calculated before the canary starts — based on how large a quality difference you're trying to detect and how noisy your quality metrics are — not set as a fixed time window. Running a canary for 48 hours regardless of how much data you've collected is not a statistical method; it's a schedule.

Running the canary for 48 hours at 1% traffic on a low-volume product might yield 200 interactions, which is almost certainly insufficient to distinguish a real quality regression from noise in a generative model's output distribution.

One practical tool for managing this is sequential testing, which allows valid interim looks at accumulating data without inflating false positive rates. GrowthBook's API exposes a sequentialTestingEnabled parameter that supports exactly this pattern — continuous monitoring during the hold period without the statistical penalty of repeated significance testing.

Combined with warehouse-native analysis that can aggregate quality signals directly from Snowflake, BigQuery, Databricks, and others without requiring a separate event pipeline, this architecture makes it operationally feasible to hold canary stages long enough to accumulate the sample sizes generative AI evaluation actually requires.

Character.AI's use of GrowthBook for incremental model exposure — described by Head of Post-Training Landon Smith as enabling comparison of "different modeling techniques from the perspective of our users" — illustrates what this looks like at production scale: not just a deployment safety mechanism, but a structured process for accumulating user-outcome evidence before committing to a new model version.

The through-line across all four of these considerations is the same: the traffic routing mechanics are familiar, but the exposure window must be designed around the statistical requirements of output quality detection, not the infrastructure requirements of crash detection. Those are different problems with different answers.

Designing rollback triggers beyond error thresholds for AI canary deployments

The instinct when inheriting a canary deployment system is to reach for the same rollback logic you'd use for a microservice: if error rate crosses 1%, roll back; if p99 latency exceeds 500ms, roll back. That pattern works because microservice failures tend to be loud — they surface in HTTP status codes, exception logs, and infrastructure metrics.

AI model degradation is structurally different. A model can produce subtly biased completions, quietly regress on accuracy for a specific user segment, or begin hallucinating in ways that never register as a 5xx error. The rollback trigger layer has to be redesigned from the ground up to account for this.

Why binary thresholds fail for AI model rollbacks

The core problem is that AI models don't fail the way software fails. As models encapsulate learned weights, training data lineage, feature preprocessing pipelines, and implicit behavioral assumptions — failure is distributed across all of these dimensions simultaneously, not concentrated in a single error signal.

A model that's "healthy" by every infrastructure metric can still be silently degrading a niche slice of production traffic. Aggregate error rates look fine; a user segment is being harmed. A hard percentage threshold will never catch this, because the signal you need to act on isn't in the error log at all.

Output quality score triggers and guardrail metric design

The practical alternative is to define guardrail metrics that capture output behavior directly — task completion rates, engagement depth, output quality scores — and wire rollback logic to statistical degradation in those metrics rather than to threshold breaches.

The key design insight from GrowthBook's Safe Rollouts with guardrail metrics is that the trigger threshold should be zero: as soon as there's statistical certainty that a metric is being harmed at all, even by very small amounts, the rollout is marked as failing. This is a fundamentally different posture than "roll back if error rate exceeds 5%." It treats any statistically confirmed harm as sufficient cause to stop, regardless of magnitude.

The mechanism that makes this workable without constant false positives is one-sided sequential testing, which allows rollback decisions to be made as soon as statistical significance is reached without the false positive accumulation that comes from repeatedly peeking at results.

This matters because AI output metrics are inherently noisy — response quality scores fluctuate, engagement signals vary by time of day, and a naive threshold approach will generate rollback events that don't reflect real model regression. One practical constraint worth taking seriously: GrowthBook's documentation explicitly warns that choosing too many guardrail metrics increases false positive risk.

A focused set of critical metrics — the ones that most directly reflect model output quality for your specific use case — will produce more reliable rollback signals than an exhaustive monitoring dashboard.

Anomaly detection and model drift as rollback signals

Beyond quality score triggers, anomaly detection deserves treatment as a first-class rollback signal, not a secondary alert. item.com's canary deployment framework explicitly includes anomaly detection signals alongside latency and accuracy in its rollback trigger specification — the same tier, not a supplementary layer.

The reason is that model drift can originate from data distribution shifts rather than model weight changes, meaning a model that performed well at deployment can degrade as the distribution of incoming requests shifts away from the training distribution. This kind of drift doesn't produce errors; it produces outputs that are subtly off-baseline in ways that only anomaly detection will surface.

Automated vs. human-in-the-loop rollback decisions

Not every rollback signal warrants the same response. When guardrail metrics cross statistical significance thresholds — clear, statistically certain regression — automated rollback is appropriate and fast. GrowthBook's Safe Rollouts support an Auto Rollback toggle for exactly this case, and the system exposes webhook events (feature.saferollout.rollback, experiment.decision.rollback) that can wire rollback decisions directly into CI/CD pipelines.

But ambiguous degradation signals — cases where results are inconclusive or where traffic distribution looks unhealthy rather than regressed — warrant human review rather than automated action.

GrowthBook distinguishes between a "Guardrails Failing" state (regression detected, consider reverting) and an "Unhealthy" state (traffic imbalanced, check implementation), treating these as distinct failure modes that require different responses. The default for genuinely inconclusive results is to ship, not block — a deliberate bias toward action over paralysis that teams should adopt as an explicit policy rather than leaving undefined.

The engineering decision of where to draw the automated/manual boundary is not a tooling question — it's an organizational one. Teams that haven't defined it before the canary starts will make it under pressure, which is the worst time to reason clearly about statistical ambiguity.

Treating the AI canary release as a controlled experiment, not just a safe deployment

There's a meaningful difference between asking "did anything break?" and asking "did this version produce better outcomes than the one before it?" Most canary deployments are designed to answer the first question. The most mature AI teams have learned to answer both simultaneously — and that shift in framing has concrete consequences for how the rollout is designed before a single request is routed.

The defensive posture — monitor for errors, watch latency, roll back if something explodes — is necessary but not sufficient for AI. A model can pass every failure threshold while quietly producing responses that are less helpful, less accurate, or less aligned with what users actually need.

If you're not measuring that, you're not learning anything from the canary beyond "it didn't crash." That's a low bar for a production model release.

Defining 'better' before traffic routes, not after results arrive

The practical implication of treating a canary as an experiment is that you must define what "better" means before you route traffic, not after you've seen the results. This sounds obvious, but it's routinely violated in practice.

Teams deploy a new model version, watch the dashboards for a few days, and then decide whether the numbers look good enough to promote. That's post-hoc rationalization, not evidence.

The experimental framing imposes a pre-condition: write the hypothesis first. What specifically do you expect this model version to improve, for which users, and by how much? GrowthBook's experiment data model enforces this structurally — the schema requires a hypothesis field, making the discipline of defining expected outcomes a required step in the deployment workflow rather than an optional one.

That's not just a UX nicety; it's the platform encoding experimental discipline into the deployment workflow.

Character.AI is a concrete example of this approach in production. Landon Smith, Head of Post-Training at Character.AI, describes it directly: "GrowthBook has been an invaluable tool for Character.AI, helping us develop our models into a great consumer experience. We can compare different modeling techniques from the perspective of our users — guiding our research in the direction that best serves our product."

That framing — comparing modeling techniques from the perspective of user outcomes — is precisely the shift this section is arguing for.

Selecting primary vs. guardrail metrics before rollout

Pre-registering metrics before a canary begins is what separates a controlled experiment from a monitoring exercise. GrowthBook's experiment schema separates three distinct tiers: primary goals, secondary metrics, and guardrails — each with configurable risk thresholds.

Primary metrics answer the hypothesis directly: did task completion improve, did engagement depth increase, did user satisfaction scores move? Guardrail metrics define the floor — the conditions under which the experiment stops regardless of how the primary metrics look. Latency SLA violations, safety guardrail breaches, and cost-per-query spikes are all candidates for guardrail status.

This separation matters because it prevents the peeking problem: if you're free to adjust which metric you're optimizing for after seeing early results, your statistical conclusions are invalid. The same discipline that rigorous A/B testing programs and clinical trials require now applies to model deployment.

Feature flags as the traffic control layer

Feature flags are the mechanical layer that makes canary-as-experiment operationally feasible. They control which users see which model version, generate the assignment data needed for statistical analysis, and enable instant rollback without redeployment.

The key distinction from a pure canary deployment is that the flag assignment is logged and tied to outcome metrics — creating the experimental record that drives the promotion decision.

GrowthBook's phases structure allows configurable coverage percentages and traffic split weights per variation, mapping directly to canary-style incremental exposure within an experiment framework. Reducing the friction of instrumenting every model release as a proper experiment — rather than an ad-hoc rollout — is what makes this discipline sustainable at shipping cadence.

Statistical rigor for promotion decisions

Promoting a model to 100% traffic because "it looked fine" is not the same as having statistical evidence it outperforms the previous version. The promotion decision should be driven by primary metric improvement above a pre-defined threshold, guardrail metrics within bounds, and statistical significance — not just the absence of incidents.

GrowthBook supports both Bayesian and frequentist statistical approaches, so teams can use whichever methodology their data science function prefers. Sequential testing allows the canary to be monitored continuously — checking results as data accumulates — without increasing the risk of false alarms that comes from repeatedly peeking at incomplete data.

Regression adjustment reduces the noise in your metrics, which means you need less data to reach a confident conclusion. Sequential testing is particularly relevant for AI canaries: it allows teams to make promotion decisions as soon as statistical confidence is reached rather than waiting for a fixed sample size, which matters when new model versions are shipping frequently.

Each canary that runs this way becomes a data point in an institutional learning library — evidence about what actually moves user outcomes, not just a deployment event that happened without incident.

Closing the gap between canary safety theater and actual AI deployment confidence

As established at the outset: the through-line of this article is a single category error — treating AI model degradation as if it were software failure. Software fails loudly, models fail quietly. A hallucinating model returns HTTP 200. A drifting model serves every request — just worse.

If your canary process is instrumented only to catch the loud failures, you have a false sense of safety, not actual safety.

The single question that reveals whether your canary can detect AI failure

The most useful thing you can do right now is look at your existing canary setup and ask one question: if your new model version started producing subtly worse outputs tomorrow, would any of your current monitors catch it?

If the honest answer is "probably not," that's your gap. You don't need to rebuild everything at once — but you do need to know where the blind spots are before the next release goes out.

One output quality signal beats zero infrastructure metrics for AI rollback

You don't need a complete observability platform before your next canary. You need one output quality signal that's closer to user outcomes than error rates — task completion, engagement depth, cost per successful interaction — and a rollback trigger wired to statistical degradation in that signal rather than a hard threshold.

That's a meaningful improvement over where most teams start, and it's achievable before your next model release.

Promotion decisions that require evidence, not just the absence of incidents

The shift from "did anything break?" to "did this version produce better outcomes?" is not a tooling upgrade — it's a discipline change. It means writing the hypothesis before routing traffic, pre-registering your primary and guardrail metrics, and treating the promotion decision as something that requires evidence, not just the absence of incidents.

Warehouse-native experiment analysis and statistical guardrails are built specifically to make this workflow operationally feasible — connecting model assignment data to user outcome metrics without a separate event pipeline, and surfacing rollback signals the moment statistical certainty is reached.

This article is meant to be genuinely useful to teams who are shipping AI models and want their deployment process to match the actual risk profile of what they're releasing — not a theoretical framework, but a practical reorientation.

What to do next: Pull up your current canary monitoring setup and identify whether you have any metric that reflects what your model is producing — not just whether it's running. If you have zero output quality signals today, start there: instrument one user-outcome metric (task completion rate is a good default) and define what statistical degradation in that metric would cause you to pause a rollout. That single addition closes the most dangerous gap this article describes, and it's the right first step regardless of what tooling you're using.

Sign up for free

Take Growthbook for a spin, no credit card required.

Create my account