A/B experiments are proposed by the learning engine and staged for human approval. When approved, the platform splits incoming conversations across two campaign versions and measures outcomes — booking rate, drop-off, conversation length — until the sample is sufficient.
When the pattern detector identifies a change worth testing against the current configuration — rather than just applying — it creates an EXPERIMENT_PROPOSAL. The proposal shows what is being tested, what split is proposed, and what outcome metrics will be measured. Human approval is required before any traffic is split.
Pattern detection identifies correlations. A/B experiments confirm causation — or disprove it. Testing before adopting means your campaign changes are evidence-based, not assumption-based.
An experiment tests one specific change against the current configuration — a rephrased question, a different branch instruction, a tone adjustment. Isolating one variable means the result is attributable.
How it works: EXPERIMENT_PROPOSAL includes the proposed_changes diff — exactly what differs between control and variant is visible before you approve the experiment.
During an experiment, both versions are active. The learning engine continues to analyse conversations from both. If the variant performs significantly worse, you can stop the experiment and revert to the control at any time.
How it works: Every version is revertable — including experiment versions. Revert creates a new version record so the action itself is auditable.
Completed experiment results are fed back into the learning engine as evidence. If one version significantly outperforms the other, the pattern detector can generate a STATISTICALLY_SUPPORTED proposal to adopt the winning configuration.
How it works: Experiment results become the highest-confidence evidence class — statistical confidence from a controlled split is better than observational correlation.
Experiment results aren't surfaced until the sample size is sufficient for statistical confidence. Running an experiment to 5 conversations and adopting the winner is a common mistake — the platform prevents it.
How it works: Minimum sample threshold applies to experiment results as well as pattern detection — no premature conclusions.
We'll propose a test based on your campaign data and walk through the setup together.