CBSplit CMS Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Why backend revenue invalidates many tests ======

===== The incomplete evaluation =====

Most tests are concluded at the front end.

Variant A converts better.
Variant B loses.
Traffic is shifted.

The decision feels rational.

Then backend revenue begins to unfold.

CBSplit was built to evaluate performance after the entire revenue lifecycle is visible.

===== Tests usually measure the first transaction =====

Classic tests optimize:

  
  * Conversion rate
  * Front-end revenue
  * Cost per acquisition
  * Early ROAS

They rarely include:

  
  * Refund-adjusted revenue
  * Subscription rebills
  * Churn timing
  * Chargeback exposure
  * Net lifetime value

Without backend data, conclusions are partial.

===== Backend revenue changes the economic picture =====

Backend revenue includes:

  
  * Upsells
  * Downsell recovery
  * Subscription rebills
  * Repeat purchases

Some variants:

  
  * Convert modestly
  * Produce average front-end metrics
  * Generate stronger backend retention

Other variants:

  
  * Convert aggressively
  * Inflate early revenue
  * Collapse during rebills

Front-end winners can become backend losers.

===== Refund timing invalidates early conclusions =====

Refunds often appear:

  
  * Days after purchase
  * After customer interaction
  * After expectation mismatch

A variant that increases impulse purchases may also increase refunds.

When refund-adjusted revenue is applied, test results can reverse.

CBSplit waits for refund reconciliation.

===== Subscription churn reshapes test outcomes =====

Subscription funnels generate revenue over cycles.

Differences in:

  
  * Messaging tone
  * Billing transparency
  * Expectation alignment

Can alter:

  
  * Rebill survival
  * Churn velocity
  * Cohort-based LTV

These effects compound over time.

Short tests cannot capture compounding lifecycle differences.

===== Backend fragility scales into losses =====

When a front-end winner is scaled:

  
  * Refund clusters grow
  * Rebill failures accumulate
  * Processor scrutiny increases
  * Net revenue declines

Backend fragility often remains invisible during early testing.

CBSplit incorporates backend stability before declaring a winner.

===== Event-based metrics ignore revenue durability =====

Tests often optimize events.

Revenue durability depends on outcomes.

Events measure:

  
  * Immediate behavior
  * Short-term engagement

Backend outcomes measure:

  
  * Revenue persistence
  * Customer alignment
  * Risk exposure
  * Net profitability

CBSplit prioritizes outcomes over event frequency.