A/B Testing & Experiments

Experiments (A/B tests) let you test design changes with specific player cohorts before rolling out globally. Quest Data uses deterministic hashing so players see consistent variants.

Creating an Experiment

1. Go to Live Ops > Experiments
2. Click New Experiment and fill:
   • Name: e.g., “Tutorial Difficulty Test”
   • Description: What you’re testing and why
   • Traffic %: 50% (test 50% of new players)
   • Variants: Add Control (A) and Treatment (B)
3. Click Start

Traffic Allocation

Allocation	Use Case
10%	High-risk changes, measure impact first
50%	Balanced testing, fast results
90%	Near-rollout, verify before 100%

Players are assigned consistently via hashing on player_id, so they always see the same variant.

Tracking Variants

Check which variant a player is assigned to:

var variant = await QuestData.get_experiment_variant("Tutorial Difficulty Test")

match variant:
  "control":
    difficulty = 1.0
  "treatment":
    difficulty = 0.75  # Easier tutorial

Then track results:

QuestData.track("tutorial_complete", {
  "experiment": "Tutorial Difficulty Test",
  "variant": variant,
  "time_spent_seconds": 300
})

Results Dashboard

Metric	Control	Treatment	Winner
Completion Rate	85%	92%	Treatment
Avg Duration	420s	380s	Treatment
Retention Day 1	42%	48%	Treatment

Statistical Significance

Results are marked:

• Gray: Too small sample size
• Blue: Statistically meaningful
• Green: Winner with >95% confidence

Rolling Out Winners

1. Results show Treatment won over Control
2. Click Rollout to All Players
3. Select: Gradual (10%/day) or Immediate (100%)
4. The variant becomes the new default

API Reference

# Get experiment status
curl "https://api.questdata.io/v1/experiments/tutorial-test" \
  -H "x-game-api-key: YOUR_API_KEY"

# Get player assignment
curl "https://api.questdata.io/v1/experiments/tutorial-test/variant?player_id=player123" \
  -H "x-game-api-key: YOUR_API_KEY"

Response:

{
  "experiment": "tutorial-test",
  "player_id": "player123",
  "variant": "treatment",
  "allocated_at": "2026-04-01T10:00:00Z"
}