How to use GrowthBook's MCP server to read experiment results in plain English

Plain English should make an experiment result easier to evaluate—not smoother over the warnings, uncertainty, and caveats that determine whether it is valid.

GrowthBook's MCP server can fetch an experiment result and return a decision-ready explanation inside a compatible AI client. The current experiment-analyze skill resolves the experiment, checks snapshot freshness, refreshes when needed, evaluates data quality, branches on the configured statistics engine, and then summarizes the primary metric, guardrails, and next action.

That order is the product. A weak prompt asks, “Is the treatment winning?” and invites a headline. A strong prompt asks for the snapshot timestamp, health checks, sample size, practical effect, interval, and precommitted decision rule before any verdict.

This guide follows the latest workflow in the GrowthBook skills repository, used by the version 2 GrowthBook MCP server.

Know what the agent is reading

The Model Context Protocol lets an AI client call a connected server. For GrowthBook analysis, the server reaches the same experiment and result objects that power the application.

The results call returns two related objects:

• Experiment metadata: name, status, type, phases, variations, statistics engine, CUPED, sequential testing, metrics, and other configuration.
• Result snapshot: timestamp, sample sizes, metric outputs, health checks, and any requested phase or dimension cut.

A snapshot is not a live stream. It records the result of a warehouse computation at a point in time. Every plain-English readout should state the snapshot timestamp so the reader knows how fresh the data is.

The current skill reuses a matching snapshot when it is under 24 hours old. GrowthBook's server commonly refreshes snapshots more frequently, but the skill uses a conservative ceiling to avoid triggering unnecessary warehouse queries. A user can still request a fresh snapshot or a different phase or dimension.

Start with a precise experiment identity

If you know the ID, use it. Names can be duplicated or reused. When you only have a phrase, the skill searches name, tracking key, description, and hypothesis, orders candidates by recent activity, and asks you to choose when more than one match is plausible.

Use:

Read the latest experiment-analyze skill and analyze experiment exp_abc123. Do not modify or stop it.

Or:

Find running standard experiments matching “short checkout.” List IDs, names, tracking keys, projects, and update dates. Do not select one for me.

The skill stops on drafts because there are no results to interpret. It also stops on multi-armed bandits: their dynamic allocation and per-arm probabilities require a different result frame than a standard A/B test.

The GrowthBook experiment analysis documentation remains the full visual source of truth. The MCP summary should include a direct experiment link so a reviewer can inspect SQL, dimensions, and history.

Ask for freshness without refreshing everything

The results endpoint can return “no results found” when an experiment has started but no snapshot exists yet. Otherwise, compare dateUpdated with the current time and the requested analysis slice.

Refresh when:

• the snapshot is over 24 hours old
• the user explicitly requests current data
• the requested phase has no matching snapshot
• the requested dimension cut has no matching snapshot

Reuse when the matching snapshot is recent and the user did not ask for recomputation.

A fresh snapshot is asynchronous. The agent starts it, captures the snapshot ID, and polls every few seconds with a five-minute ceiling. It should not spin indefinitely. If the job is still running at the ceiling, report that state and let the user return later.

Ask:

Fetch experiment exp_abc123 results. Reuse the snapshot only if it is under 24 hours old and matches the default phase with no dimension. Otherwise start one fresh snapshot, poll for up to 5 minutes, and report if it remains in progress. Always show the final snapshot timestamp.

This is more responsible than “always refresh,” particularly when many users share a warehouse and GrowthBook API rate limit.

Put six health checks before lift

The current skill checks six categories in order:

Sample ratio mismatch is especially important. A 50/50 test that consistently records an implausible split may have biased assignment or exposure. The SRM diagnosis paper documents common causes and why downstream metric analysis cannot repair the underlying mismatch.

Ask the agent to stop interpretation after a failed SRM:

Report all data-quality checks before the primary metric. If SRM fails, explain the observed and expected split, list plausible causes grounded in this experiment's setup, and stop. Do not label a winner.

Multiple exposure rates need context. A small amount can be benign; a high rate suggests an unstable assignment unit or missing sticky bucketing. Variation ID mismatch is more direct: the application is sending IDs GrowthBook does not recognize.

The GrowthBook false-positive guide places these controls before result interpretation for the same reason: a clear statistical signal is not useful when the comparison is invalid.

Catch false signals early

Use data-quality, stopping, and multiple-testing controls before a clean summary turns an invalid comparison into a product decision.

Review the False-Positive Guide

Translate the primary metric without changing the method

The experiment's statistics engine determines the vocabulary.

Bayesian result

Report:

• control and treatment values
• relative lift point estimate
• credible interval
• Chance to Win
• practical decision threshold

A plain-English example:

The treatment's estimated conversion lift is 2.4%. The 95% credible interval runs from -0.8% to 5.9%, so the data still supports both a small loss and a useful gain. Chance to Win is 89%, below the team's 95% positive threshold. The result is inconclusive.

Do not call a Bayesian interval a confidence interval. Do not invent a p-value. GrowthBook's Bayesian statistics guide explains the engine and data thresholds.

Frequentist result

Report:

• control and treatment values
• relative lift point estimate
• confidence interval
• whether the interval crosses zero
• decision threshold and planned stopping rule

For example:

The treatment's estimated conversion lift is 2.4%, with a 95% confidence interval from 0.6% to 4.2%. The interval excludes zero under the experiment's planned analysis. Guardrails are stable, but the result should still be compared with the minimum practical effect before shipping.

A p-value is not the probability that the null hypothesis is true. NIST's statistical testing handbook describes hypothesis-test decisions in terms of a predefined rule, not a posterior probability about the hypothesis.

CUPED and sequential testing

When CUPED is enabled, adjusted estimates can differ from raw means and intervals are often narrower. When sequential testing is enabled, confidence sequences are intentionally wider so repeated looks do not silently inflate the false-positive rate.

Say which settings are on. Do not compare a sequential interval directly with an older fixed-horizon result as though only sample size changed. The GrowthBook methodology comparison explains the tradeoffs among Bayesian, fixed-horizon frequentist, and sequential approaches.

Separate statistical, practical, and product conclusions

A strong explanation has three layers:

1. Statistical: What range of effects is compatible with the data under the configured method?
2. Practical: Does that range clear the minimum effect worth implementing?
3. Product: Given guardrails, opportunity cost, reversibility, and strategy, what should the team do?

An interval entirely above zero can still describe an effect too small to justify maintenance cost. An inconclusive result can still rule out the large lift the team hoped for. A guardrail regression can block a treatment with a strong primary gain.

Ask:

Translate the primary result into statistical, practical, and product conclusions. Use our pre-launch MDE as the practical threshold. Distinguish evidence of no useful effect from insufficient evidence. Do not promote a secondary metric if the primary is inconclusive.

That last boundary matters. Secondary metrics are exploratory context. Selecting whichever one moved after the result is visible creates a multiple-comparisons problem. On the frequentist engine, report the configured correction. Guardrails are intentionally handled differently because they are designed to block harm.

Read dimensions without fishing

GrowthBook can analyze a configured unit or experiment dimension, such as country or device. A dimension snapshot multiplies comparisons, so use it to investigate a pre-specified question or diagnose heterogeneity—not to hunt for a winning segment.

Prompt:

Analyze phase 1 of exp_abc123 by experiment dimension country. Run a matching snapshot if needed. Report the global primary result first, then notable country differences with sample sizes and uncertainty. Treat dimensional findings as exploratory.

Average lift is the right first estimate for the randomized population. GrowthBook's heterogeneous treatment effects article explains why segment differences can be useful while still requiring caution and follow-up validation.

Use a fixed plain-English readout

Ask for this structure:

Experiment: <name> (<id>)
Status/type: running, standard
Method: Bayesian or frequentist
Adjustments: CUPED and sequential testing on/off
Phase/dimension: <scope>
Snapshot: <timestamp>
Sample: <total and per variation>

Data quality
- SRM
- multiple exposures
- minimum thresholds
- variation ID mismatch
- suspicious uplift
- guardrails

Primary metric
- control and treatment values
- lift and interval
- Chance to Win OR confidence-interval conclusion
- practical threshold
- verdict

Secondary metrics
- exploratory findings and correction caveat

Recommendation
- wait, investigate, stop as won/lost/inconclusive, or run a follow-up
- direct GrowthBook link

This format keeps the evidence above the recommendation. It also makes two analyses comparable over time.

Copy-ready prompts for common questions

“How is the test doing?”

Analyze <id> with the latest skill. Use a recent matching snapshot, list all health checks first, then explain the primary metric and guardrails in plain English. Include uncertainty and snapshot time. Read only.

“Is the mobile segment different?”

Show the global result first, then run the configured device dimension. Include sample sizes and intervals for each segment. Treat differences as exploratory and do not recommend retargeting from one cut.

“Can we ship?”

Compare the result with the precommitted primary metric, MDE, sample size, stopping rule, and guardrails. Separate statistical and practical conclusions. If the data is valid and conclusive, propose a stop action but do not execute it.

“Why does the dashboard look worse today?”

Compare snapshot timestamps, phase, dimension, sample sizes, statistics settings, and metric definitions before comparing lift. Identify which inputs changed. Do not attribute the change to product behavior without evidence.

Keep the final decision human and recorded

experiment-analyze is read-only. If a running experiment is conclusive, the next workflow is experiment-stop. That skill requires the user to choose an actual variation ID and confirm whether to enable a temporary rollout.

Do not ask the analysis agent to “ship whichever variation is winning.” Interim results can move, health checks can fail, and the operational cost of the treatment is not in the result payload. Instead, preserve the readout, record the product decision, and execute the separate stop action through review.

Community discussion around the first GrowthBook MCP result tools focused on getting experiment data into the client. The current skill adds the more valuable layer: freshness, health checks, method-aware interpretation, and a strict read-only boundary.

The GrowthBook experimentation platform gives you transparent warehouse queries and configurable statistics. MCP makes those outputs accessible in ordinary team language. The best explanation remains faithful to the method, explicit about uncertainty, and clear about what the data cannot decide.

Make the result decision-ready

Use a practitioner checklist for design, data quality, uncertainty, and guardrails before turning an experiment readout into action.

Read the Trustworthy Checklist