How to use GrowthBook's MCP server to launch an A/B test in minutes from your IDE

“In minutes” is realistic when the experimentation foundation already exists. It is reckless when a prompt is expected to invent the foundation during launch.

GrowthBook's MCP server can bring experiment configuration into an AI-enabled IDE such as Codex, VS Code, Cursor, or Claude Code. You can describe a test, resolve the matching GrowthBook objects, create a draft experiment, prepare the feature flag rule, review the result, and start the test without manually copying IDs between browser tabs.

The speed comes from reusing approved infrastructure: metrics, datasources, assignment queries, hash attributes, experiment templates, SDK instrumentation, and permissions. If those pieces are missing, the agent should stop and tell you what to configure. It should not guess its way to a running test.

This guide follows the current experiment-design and experiment-launch skills from the GrowthBook skills repository. GrowthBook MCP version 2 uses these workflow skills with a thin API bridge, so older walkthroughs that depend on a fixed create_experiment tool do not describe the current architecture.

Know what the MCP server is launching

The GrowthBook MCP server connects your AI client to GrowthBook's API. The Model Context Protocol standardizes how the client discovers and invokes tools; GrowthBook's skills provide the product-specific procedure.

An IDE launch crosses three systems:

The MCP call does not deploy code. Creating a flag rule does not prove that the application evaluates the flag. Starting an experiment does not prove exposure events are arriving. Treat these as connected but independently verifiable states.

Complete a one-time readiness check

Before trying the fast path, confirm the following:

• GrowthBook MCP is connected and can make a read-only API call.
• The API key can read the project, experiments, flags, environments, attributes, datasources, and metrics in scope.
• The application has a supported GrowthBook SDK initialized once in the correct runtime.
• The SDK sends a stable assignment attribute such as user ID, account ID, or device ID.
• An assignment query in GrowthBook uses that same identifier type.
• Exposure tracking is configured and tested.
• The primary and guardrail metrics exist on the same datasource as the experiment.
• The treatment code is deployed behind a safe flag default.
• The team knows who can approve and publish a production feature revision.

For a team's first test, run an A/A test before a product hypothesis. That validates assignment and measurement while both variations intentionally behave the same. GrowthBook's A/A testing guide explains what to check.

Ask the agent for a readiness report before writes:

Read the latest GrowthBook setup, experiment-design, and experiment-launch skills. For project checkout, verify available templates, hash attributes, datasources, compatible assignment queries, and candidate metrics. Return IDs and missing prerequisites. Do not create or update anything.

This preflight is also a security check. The open GrowthBook MCP implementation forwards authenticated REST calls and gives the agent soft guidance to confirm mutations. Your client approvals and GrowthBook permissions remain part of the control boundary.

Step 1: Write a decision-ready test brief

An A/B test needs more than “try a shorter checkout.” A compact launch brief should contain:

• a falsifiable if/then/because hypothesis
• control and treatment behavior
• one primary metric
• 1–3 guardrail metrics
• baseline value and minimum detectable effect
• eligible daily traffic and a duration estimate
• target project and population
• intended flag key and value type

For example:

If we remove the optional company field from checkout, then completed purchase rate will increase, because fewer required decisions reduce form abandonment. Control is the current form. Treatment hides the company field. Primary metric: completed purchase. Guardrails: payment errors and seven-day refund rate. Boolean flag: short-checkout-form. Target: signed-in US web users.

Put the same brief into a structured handoff so names and values do not drift between design, code, and configuration:

{
  "project": "checkout",
  "featureFlag": {
    "id": "short-checkout-form",
    "valueType": "boolean",
    "controlValue": "false",
    "treatmentValue": "true"
  },
  "hypothesis": "If we remove the optional company field, completed purchase rate will increase because fewer required decisions reduce abandonment.",
  "population": "Signed-in US web users",
  "primaryMetric": "Completed purchase",
  "guardrailMetrics": ["Payment errors", "Seven-day refund rate"]
}

Names are acceptable at this stage. The read-only design pass should resolve them to IDs, report ambiguities, and stop if the requested objects do not exist.

Precommitting to the primary decision limits post-hoc story selection. The Center for Open Science preregistration resources cover the general principle: distinguish what you predicted from what you discovered after seeing data.

Ask experiment-design to resolve, not invent:

Turn this brief into a GrowthBook experiment spec. Use existing templates and metrics only. Prefer one goal metric and 2 guardrails. Match the hash attribute to a compatible assignment query. Estimate sample size from the stated baseline, MDE, and traffic. Return the spec and stop.

The output should name metric IDs, datasource, assignment query, randomization unit, variations, sample per variation, estimated runtime, project, and permanent tracking-key suggestion. If the metric does not exist, create it in GrowthBook before continuing.

Pick a metric that can decide

Use a focused KPI and guardrail framework before a fast launch turns a vague outcome into an expensive inconclusive test.

Read the KPI Playbook

Step 2: Implement both variations behind the flag

Open the code path that controls the experience. Ask the coding agent to follow existing repository conventions and the correct SDK API for your language. The control value should preserve current production behavior.

A Boolean example has four tests worth writing:

1. missing or uninitialized flag returns the safe control
2. explicit false renders the current checkout
3. explicit true renders the short checkout
4. exposure registration happens at the point the user is actually included

Do not place evaluation so early that ineligible users count as exposed. Do not use an outcome influenced by treatment as the assignment identifier. Keep the assignment stable across requests and devices to the degree your experiment design requires.

The GrowthBook feature flag experiment docs explain the relationship between flag values and experiment variations. Your pull request should show the control and treatment code independently of the GrowthBook configuration receipt.

Run tests and deploy the gated code before starting the experiment. Because the new flag begins disabled and defaults to control, deployment and release can remain separate.

Step 3: Create the experiment and linked flag draft

With the spec approved, experiment-launch selects one of two paths:

• Template path: use an existing experiment template for datasource, exposure query, hash attribute, metrics, statistics engine, and targeting.
• Explicit path: resolve each of those objects and send them in the create payload.

Do not mix templateId with explicit datasource and assignment fields. GrowthBook rejects that combination because the template already supplies them.

Then the workflow:

1. creates the experiment in draft
2. records stable variation IDs returned by GrowthBook
3. creates a disabled flag or verifies an existing flag
4. adds an experiment-ref rule to a fresh draft revision
5. inserts both the serialized flag value and returned variation ID for every variation

On the explicit path, the experiment-create request has this shape. The metric, datasource, assignment-query, and project placeholders must come from the preflight rather than from a guess:

{
  "datasourceId": "<DATASOURCE_ID>",
  "assignmentQueryId": "<ASSIGNMENT_QUERY_ID>",
  "hashAttribute": "user_id",
  "trackingKey": "short-checkout-form",
  "name": "Short checkout form",
  "hypothesis": "Removing the optional company field will increase completed purchase rate.",
  "variations": [
    { "key": "0", "name": "Control" },
    { "key": "1", "name": "Treatment" }
  ],
  "metrics": ["<PRIMARY_METRIC_ID>"],
  "guardrailMetrics": ["<PAYMENT_ERROR_METRIC_ID>", "<REFUND_METRIC_ID>"],
  "project": "<PROJECT_ID>"
}

The response supplies experiment.id and a stable variationId for each variation. Carry those exact IDs into the flag rule. A representative draft rule looks like this:

{
  "rule": {
    "type": "experiment-ref",
    "experimentId": "<EXPERIMENT_ID>",
    "enabled": true,
    "allEnvironments": true,
    "variations": [
      { "value": "false", "variationId": "<CONTROL_VARIATION_ID>" },
      { "value": "true", "variationId": "<TREATMENT_VARIATION_ID>" }
    ],
    "description": "Experiment: Short checkout form"
  }
}

The skill posts that body to /api/v2/features/short-checkout-form/revisions/new/rules. Confirm the returned draft version; do not publish it separately, because the experiment start step publishes the pending revision after policy checks.

Reusing a flag needs careful checks. Its value type and project must match. Its default should equal control. An archived flag blocks the flow. An existing rule for a different running experiment requires an explicit decision.

Use this prompt:

Launch the approved short-checkout-form spec through the latest experiment-launch skill. Reuse the flag only if its type, project, default, and active experiment rules are compatible. Create the experiment and draft feature revision, but stop before /start. Return IDs, values, revision, rule scope, and direct GrowthBook links.

At this point the configuration is reversible. Review the experiment setup and feature pages. Confirm that control is variation 0, the treatment value matches the deployed code, the environment and audience are correct, and the primary and guardrail metrics are the intended definitions.

Step 4: Start only after the QA receipt passes

The final start call publishes the pending feature revision and changes the experiment to running. It also enforces the organization's pre-launch checklist.

Approve with exact scope:

I reviewed experiment <exp-id> and flag revision <version>. Start this standard experiment in production. Do not skip the checklist. After the call, read back experiment status, published flag revision, variations, hash attribute, and direct links.

Inside the current experiment-launch skill, the final mutation is equivalent to this bundled-helper call:

gb-call POST /api/v1/experiments/<EXPERIMENT_ID>/start '{}'

Do not add skipChecklist unless the user explicitly chooses to bypass the returned checklist. The success receipt should show status: running, the published feature revision, and direct links to both objects.

Three outcomes matter:

• Success: the rule is published and experiment status is running.
• Approval required: the revision remains in draft; the agent can request review but cannot self-approve.
• Checklist incomplete: the agent surfaces the returned items and stops. Bypass only after an explicit user instruction.

The workflow should never recreate the experiment to escape an approval error. That produces duplicate objects and leaves the policy problem unresolved.

Step 5: Validate the first exposure before declaring success

“Running” proves configuration state, not data quality. After traffic reaches the variation code, validate:

• both variation IDs appear in exposure data
• the observed split is plausible for the configured allocation
• the assignment identifier is stable
• the primary and guardrail queries return data
• there are no obvious multiple-exposure or variation-ID mismatch warnings
• the actual product experience matches the assigned value

Use a read-only prompt:

Check experiment <exp-id> after initial traffic. Report status, snapshot timestamp, per-variation sample counts, SRM, multiple exposures, variation ID mismatch, and metric thresholds. Do not interpret lift or recommend a winner yet.

Sample ratio mismatch should halt result interpretation. The SRM diagnostic paper explains why an unexpected assignment split is evidence of a potentially invalid experiment rather than a metric to wave away.

Community discussions about code-based GrowthBook experiments often emphasize the advantage of shipping variations with application code. That still requires a disciplined handshake between code, flag configuration, and warehouse measurement. MCP accelerates the handshake; it cannot make missing exposure data appear.

A copy-ready IDE sequence

Use separate prompts so each stage has an observable output.

1. Preflight

Verify GrowthBook MCP access and list the current experiment skills. Check project checkout for templates, eligible hash attributes, datasources, assignment queries, and the proposed metrics. Read only.

2. Design

Produce an experiment spec for the shorter checkout hypothesis. One primary metric, up to 2 guardrails, 2 variations, sample-size inputs shown. Do not write.

3. Configure

Create the draft experiment and compatible feature flag revision from the approved spec. Stop before starting. Return a QA receipt.

4. Implement

In this repository, add the short-checkout-form evaluation following the existing GrowthBook SDK pattern. Keep false as current behavior. Add tests. Do not change GrowthBook.

5. Start

The code is deployed and I reviewed the GrowthBook draft. Start experiment <id> without skipping approvals or checklist items. Read back final state.

This sequence may take minutes in a prepared project. If the agent finds missing metrics, an identifier mismatch, a permission gap, or undeployed treatment code, stopping is the correct outcome.

Common launch failures and the real fix

The fastest repeatable setup uses experiment templates for organizational defaults, one stable assignment unit per surface, well-governed metrics, and a standard QA receipt. GrowthBook's experiment templates make those defaults reusable.

Launching from the IDE is valuable because the experiment configuration stays near the code and hypothesis. Keep the browser-quality checks—metric definitions, visual state, checklist, and result health—even when you no longer need the browser for data entry.

Launch with stronger checks

Use a practical experiment checklist to protect the fast path from assignment bugs, weak metrics, and premature decisions.

Review the Launch Checklist