How *A/B tests* earn trustworthy reads

What to test and what to leave alone#

Test one variable at a time: concept against concept, hook against hook, offer against offer, landing path against landing path. Never test creative and audience together, because the winner teaches nothing when two things change. Leave bidding, budgets and placements matched across cells so behaviour differences trace to the variable. Start at the Meta Ads hub for the family view, and read Meta own Experiments guide for tool mechanics.

Sequence tests by commercial weight: offers before hooks, hooks before layouts, layouts before music. Hook tests run cheapest, and the 3-2-2 method stages them without chaos.

Setting up the Experiments tool#

Build the test in Experiments, not by duplicating ad sets, so audiences split randomly without overlap and budgets stay fixed per cell. Define the success metric before launch, usually cost per new customer acquisition, with thumb stop and CTR as diagnostic seconds. Set runtime to cover at least one full buying cycle, typically seven to fourteen days, and freeze edits mid test.

Size cells from target CPA so each cell can clear meaningful conversions: thin cells produce theatre, not evidence. Keep signal clean with Pixel plus CAPI, and respect edit reset rules by batching any fix into a relaunch.

Reading results without fooling yourself#

Judge on cost per new customer after the full window, split new versus existing so retargeting blends never flatter prospecting. Ignore day two leads; attribution lag means early CPA rarely matches day ten CPA per attribution windows. Demand a margin that survives fees, returns and fulfilment, not platform ROAS alone. Meta explains test reads in its brand lift and split guidance.

Log hypothesis, cells, spend, result and decision for every test; the log teaches more than any single winner. Promote winners by scaling inside consolidated structure from consolidation.

Holdouts that prove incrementality#

Run quarterly holdouts: pause retargeting or a channel briefly and watch total revenue, not platform reported conversions. If total revenue barely moves, the paused spend was harvesting demand other touches created. Calendar holdouts with finance so the business expects the wobble, and pair them with lift studies where budgets allow.

The method in conversion lift settles incrementality arguments, and modeled conversions explain reporting gaps. Holdouts humble every dashboard, which is their value.

Mistakes that corrupt reads#

Five corruptions recur: mid test edits that restart learning, uneven budgets that starve one cell, overlapping audiences that leak exposure, judging on clicks instead of customers, and stopping early on noise. Each produces a confident wrong answer that wastes the next quarter. Freeze, match, separate, judge commercially, and finish the window.

When teams disagree, rerun the test rather than relitigating the spreadsheet. Ask our team for a test design review before spending, which costs less than a corrupted quarter.

Testing calendar that compounds#

Run a rolling monthly test slot: one live test, one test in design, one result being rolled out. Prospecting concepts in month one, hooks and offers in month two, landing paths in month three, then repeat with the library richer each cycle. Never run two tests on overlapping audiences simultaneously, because exposure leaks corrupt both reads.

Share results beyond media: feed winning promises into email, site headlines and sales scripts so the whole business learns from auction evidence. The cadence in fatigue rotation supplies the concepts, and diversification banks each winner into lasting range.

Frequently asked questions#