CBSplit CMS Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Why CBSplit was built for scale, not experiments ======

===== The experimentation mindset =====

Most optimization tools are built around experimentation.

Run a test.
Change a variable.
Wait for statistical significance.
Declare a winner.

This mindset works in controlled environments.

Revenue systems are not controlled environments.

CBSplit was built with this reality in mind.

===== Experiments assume stability =====

Classic experiments assume:  
  * Stable traffic sources
  * Consistent user behavior
  * Predictable systems
  * Clean attribution

At scale, these assumptions break.

Payments fluctuate.
Traffic quality shifts.
External systems fail.

CBSplit does not depend on stability to function.

===== Scale exposes system behavior, not variants =====

At low volume, small changes appear meaningful.

At scale, what matters is:  
  * How systems behave under load
  * How retries perform
  * How refunds accumulate
  * How subscriptions survive

These are not experimental questions.

They are operational ones.

CBSplit focuses on operational truth.

===== Experiments optimize parts. Scale tests the whole =====

Experiments isolate components.

Scale tests interactions.

At scale:  
  * Retry logic interacts with traffic quality
  * Upsells interact with refunds
  * Subscription churn interacts with acquisition promises

CBSplit observes these interactions continuously.

===== Experiment tools stop when conditions change =====

Traditional experiment tools struggle when:  
  * Traffic shifts rapidly
  * Funnels are modified frequently
  * Outcomes are delayed
  * External dependencies fail

They require resets, re-baselining, and clean test windows.

CBSplit is designed to operate without resets.

===== Scale demands outcome awareness =====

At scale, optimizing events is dangerous.

What matters is:  
  * Net revenue
  * Payment stability
  * Refund containment
  * Processor health

CBSplit optimizes for these outcomes continuously, not in test windows.

===== CBSplit enables learning under real conditions =====

Instead of isolated experiments, CBSplit enables:  
  * Continuous learning
  * Outcome-based routing
  * Real-time risk awareness
  * Long-term revenue protection

This is how systems survive at scale.

===== Why CBSplit resists the experiment-first narrative =====

CBSplit does not reject experimentation.

It rejects the idea that experiments are the foundation of revenue systems.

At scale, systems must:  
  * Adapt constantly
  * Absorb failure
  * Handle uncertainty
  * Protect revenue

CBSplit was built for this environment.