How to use GrowthBook MCP server to automate your experiment lifecycle

Experiment automation should remove coordination work without automating away the decisions that make a test trustworthy.

A product experiment is not one API call. It is a chain of decisions: choose a useful question, define a falsifiable hypothesis, select metrics and guardrails, estimate a viable runtime, configure assignment, launch safely, interpret fresh results, record a decision, and retire the temporary flag logic.

GrowthBook's current MCP server can coordinate that chain from a compatible AI client. Version 2 exposes a thin authenticated API bridge plus official, versioned agent skills. The skills describe how to use GrowthBook's REST API in a safe sequence, including what to inspect, when to stop for confirmation, and what failure states to surface. The current GrowthBook skills catalog defines a connected experiment workflow: experiment-brainstorm → experiment-design → experiment-launch → experiment-analyze → experiment-stop → flag cleanup.

This is orchestration, not autonomous product judgment. The agent can gather context and carry out approved steps. Your team still chooses the hypothesis, primary outcome, acceptable downside, launch moment, and final ship-or-kill decision.

Map the lifecycle before you automate it

The Model Context Protocol gives an AI application a standard way to discover and invoke server capabilities. The GrowthBook server adds authenticated access to the same backend used by the app, while procedural skills encode the order of operations.

The distinction between a protocol and a skill matters. MCP makes calls possible; the skill makes a repeatable workflow more likely. The open Agent Skills specification describes this portable packaging model, while the GrowthBook MCP repository provides the live server and generic API call tool.

Start with least privilege. Use a token limited to the projects and actions the workflow needs. Ask for a read-only lifecycle dry run before allowing writes. The GrowthBook AI-native development page describes how agent actions inherit GrowthBook permissions, approval flows, and auditability rather than bypassing them.

Stage 1: Ground ideas in prior experiments

Automation is most useful when it starts with evidence instead of generating a generic list of growth ideas. The experiment-brainstorm skill reads recent stopped experiments, fetches their results, and looks for three patterns: what worked, what stalled, and what remains underexplored.

It intentionally filters to stopped experiments. A running result can change, and a draft has no outcome. The workflow also limits result fetches to respect the API rate limit and calculates a simple settled-test win rate as wins divided by wins, losses, and inconclusive results.

Try this prompt:

Read the latest experiment-brainstorm skill. For project checkout, review up to 20 stopped standard experiments from the last year. Exclude bandits. Summarize winning, losing, and underexplored patterns, then propose 5 experiments grounded in named prior tests. Do not create anything.

A useful proposal should include one falsifiable hypothesis, one primary metric, an expected effect size expressed as an order of magnitude, and a risk metric. It should say when it is adapting a previous loser and what is different this time.

This approach counters outcome-free ideation. As the Center for Open Science explains in its work on preregistration, recording a plan before seeing results separates prediction from post-hoc explanation. Product experiments are not academic studies, but the same discipline helps teams distinguish a real hypothesis from metric fishing.

Stage 2: Convert one idea into a launchable design

experiment-design is a read-only planning step. It converts a chosen idea into an if/then/because hypothesis, control and treatment definitions, goal and guardrail metrics, an assignment unit, and a duration sanity check.

The skill recommends one goal metric, or two at most, because every extra goal makes the final decision less clear. It also asks for 1–3 guardrails. These measure outcomes that should not regress even if the primary metric improves.

Sample-size inputs should be explicit:

• baseline rate or mean for the primary metric
• minimum detectable effect worth acting on
• daily eligible traffic
• planned allocation and number of variations

The agent can produce a rough estimate, but the final calculation belongs in GrowthBook's power calculator. Runs shorter than a full weekly cycle deserve skepticism because weekday and weekend behavior may differ. Runs that require more than four weeks may need a more sensitive metric, a larger practical effect threshold, or a higher-traffic surface.

Use a prompt such as:

Design the selected checkout test using existing GrowthBook metrics. Use one primary metric and up to 2 guardrails. Show baseline, MDE, eligible traffic, estimated sample per variation, expected runtime, project, datasource, assignment query, and hash attribute. Produce a spec only.

The skill does not create missing metrics or datasources. That is a healthy boundary: metric definitions are shared semantic infrastructure, not incidental fields an agent should invent during launch.

Choose metrics that decide

Build a focused primary-metric and guardrail set before agent speed turns an unclear question into a fast but inconclusive test.

Read the KPI Playbook

Stage 3: Launch through a reversible draft

The launch workflow can take either an experiment template or an explicit set of existing GrowthBook objects. It resolves the hash attribute before the datasource and assignment query because the assignment query's identifier type must match the randomization unit. It then filters candidate metrics to the chosen datasource.

The sequence is deliberate:

1. Select a template, or resolve the assignment configuration and metrics.
2. Create the experiment in draft with stable variation IDs.
3. Create a disabled feature flag or verify that an existing flag is compatible.
4. Add an experiment-ref rule to a fresh draft flag revision.
5. Surface direct links for human QA.
6. Start the experiment only after explicit approval.

An existing flag is not automatically safe to reuse. The agent checks whether it is archived, whether its value type and project match, whether its default equals the control value, and whether another running experiment already uses it. A warning is not consent; the user decides whether to proceed.

The review prompt should name the final state:

Launch the approved shorter-checkout-form spec. Reuse flag short-checkout only if its type, project, default, and active rules are compatible. Create the experiment and a draft flag revision, then stop and give me the experiment and feature links. Do not call the start endpoint until I approve the QA receipt.

Starting does more than flip an experiment status. GrowthBook publishes the pending feature revision and enforces the organization's pre-launch checklist. If approval is required, the skill can request review but does not self-approve. If checklist items are incomplete, it returns them and stops. Bypass is never the default.

This is where automation improves reliability: the agent remembers the dependency order and known error branches. It does not eliminate the production gate. The GrowthBook feature management documentation explains the underlying flag and revision model the launch workflow uses.

Stage 4: Monitor health before reading lift

Once traffic accumulates, experiment-analyze fetches experiment metadata and results together. It reuses a snapshot that is less than 24 hours old unless the user asks for a new phase or dimension cut. When refresh is needed, it starts an asynchronous snapshot, polls with a five-minute ceiling, and then refetches the result.

The analysis order is more important than the prose summary. Data-quality checks come before lift:

• sample ratio mismatch (SRM)
• users exposed to multiple variations
• minimum data thresholds
• variation ID mismatch
• suspicious uplift
• guardrail regressions

If SRM fails, the workflow should not continue to a confident ship recommendation. The observed allocation is inconsistent with the planned split, which can indicate a tracking or assignment problem. Microsoft's ExP experimentation guidance and the widely used SRM diagnostic paper both emphasize that trustworthy decisions depend on validating experiment health, not only reading a headline uplift.

Interpretation also branches on the statistics engine. Bayesian results use Chance to Win and credible intervals. Frequentist results use confidence intervals and whether they cross zero. If CUPED or sequential testing is enabled, the summary should explain how those settings affect estimates or intervals rather than comparing unlike outputs.

Ask for a decision-ready readout:

Analyze experiment <id>. Use the cached snapshot only if it is under 24 hours old. Report the snapshot timestamp and all data-quality checks before the primary metric. Then show sample size, practical lift, interval, guardrails, and an explicit won/lost/inconclusive recommendation. Do not stop the experiment.

The result should preserve uncertainty. Underpowered interim data is not transformed into confidence by clearer prose. The GrowthBook experiment analysis docs remain the place to inspect the full result and configured decision framework.

Stage 5: Record the decision and manage the winner

experiment-stop begins by fetching current state and showing the actual variation ID strings. It never infers a winner from an array index or a display name. The user must choose a variation ID, or choose no winner.

The workflow supports two different outcomes:

• stop and record won, lost, inconclusive, or dnf
• stop and enable a temporary rollout that routes eligible traffic to a selected variation

Temporary rollout is useful when the team wants to ship a winner now but delay permanent flag cleanup. The experiment remains in the SDK payload and the existing experiment rule forces the released variation. Without a temporary rollout, stopping does not remove the linked experiment rule; the flag still needs attention.

Use explicit language:

Stop experiment <id> as won, record variation var_treatment as the winner, add this decision summary, and enable a temporary rollout to that same variation. Show the exact payload and ask me to confirm before posting.

The analysis note matters. A future teammate should be able to understand why the test ended, which result drove the call, what guardrails were checked, and whether the winner was rolled out.

Stage 6: Close the loop with flag cleanup

Experiment status and feature flag state are related but separate. After stopping, choose one disposition:

• Ship permanently: replace the experiment rule with a force rule or update the default value to the winner.
• Roll back: remove the experiment rule and return eligible users to control.
• Keep a temporary rollout: monitor the winner while planning permanent code changes.
• Remove the flag: inline the chosen behavior, verify code and deployment, archive the flag, and delete only after a second review.

The feature flag cleanup guide treats ownership and cleanup as part of operating at scale. The agent can search code references and propose replacements, but it should not claim that code deletion is automatically safe. Feature checks may be inverted, reused across services, or coupled to fallback behavior.

Ask the cleanup workflow for a receipt that covers code and GrowthBook state:

Audit flag short-checkout after experiment <id>. Confirm the recorded winner, temporary rollout, live rules, code references, and project ownership. Propose the permanent state. Do not edit code, archive, or delete until I approve each phase.

Build an automation policy, not one giant prompt

A single prompt that says “run the whole experiment” hides too many decisions. A safer automation uses explicit checkpoints and durable outputs.

Keep these checkpoints in your ticket or pull request. They let a product manager review intent, an engineer review implementation, and an experimentation owner review validity without reconstructing the agent session.

Also distinguish MCP from unattended background automation. MCP is well suited to interactive, context-rich work inside an agent. Scheduled refreshes, policy enforcement, and CI checks may be better implemented directly against the REST API. GrowthBook exposes both surfaces, so teams can use the agent where judgment is needed and deterministic jobs where repeatability is the goal.

The experiment lifecycle becomes faster when repeated lookup and coordination work moves to the agent. It becomes better only when hypotheses, metrics, health checks, approvals, and cleanup remain visible. Automate the path between decisions; do not erase the decisions themselves.

Make every result trustworthy

Review the design and analysis checks experienced experimentation teams use before they turn a statistical signal into a product decision.

Use the Experiment Checklist