How incrementality gets proven honestly

What incrementality means here#

Incrementality measures installs and payers that would not have happened without Apple Ads, stripping out organic demand and credit claimed from other journeys. Platform reporting counts touches, while incrementality counts added outcomes, and the two rarely agree. Start from the Apple Ads hub and hold this distinction through every test.

Apple own campaign materials in the success stories hub describe directional patterns, never guaranteed lifts, so treat every vendor lift figure as estimated data. Our reconciliation guide sets the honest baseline before any test spends.

Scope the question tightly before testing: which budget, which markets and which outcome counts as added. Vague incrementality questions produce vague answers whatever the method.

Holdout design that survives contact#

Run clean holdouts: pause Apple Ads in matched geos or audience slices while holding spend elsewhere, then compare payer outcomes across the split. Match geos on history rather than size, keep the holdout period at four weeks minimum and freeze other changes mid test. Our markets guide helps pick matched pairs.

Size the test so the expected lift clears noise: thin markets need longer windows, not bigger claims. Document the design before launch, including the outcome metric and the decision rule, so results cannot be renegotiated afterwards.

Resist peeking daily and calling the test on the first green week. Holdout reads settle late as postbacks land and billing cycles close, so judge at the planned date only.

Geo splits without self sabotage#

Split geos rather than keywords where possible, because keyword splits leak across match types while geo splits hold cleaner boundaries. Pair similar markets, for illustration the UK against Australia or Germany against France, holding budgets proportional to history. Scope splits with location targeting.

Keep creative, pages and bids matched across test and control geos so the only deliberate difference is Apple Ads spend. Log every accidental change, from price tests to featuring wins, because each one clouds the read.

Run one change at a time through the split: overlapping brand relaunches or cross network app campaign bursts invalidate the comparison, so calendar the test in a quiet window wherever possible.

Read windows and reconciled rows#

Read results on reconciled rows over patient windows: store trials and payer mix against MMP postbacks, allowing six weeks for subscription outcomes to mature. Short windows punish campaigns that retain well and flatter campaigns that harvest accidentals. Benchmark medians stay calibration only, per estimated vendor data in the SplitMetrics benchmarks.

Report lift with uncertainty stated plainly: point estimate, range and the assumptions behind both. A narrow honest range beats a bold single number that collapses under questions.

Cross check against social discovery networks generically for context, never as named comparisons, keeping Apple Ads judged on its own added outcomes first.

Turn reads into budget decisions#

Convert reads into budget rules written down: scale where holdouts prove added payers, hold where lift runs marginal and cut where organic covers the ground. Revisit the proof yearly or after major product changes, because incrementality decays as brands grow. Our budget maths sizes the scale behind proven lift.

For illustration, an app proving modest but reliable added trials might widen budgets ten per cent per cycle while rerunning the holdout each half. Treat that framing as illustrative method, and ask our team for a holdout design review.

File every test with its design, rows and decision so next year starts from evidence rather than memory. Compounded test files are the real growth asset.

Frequently asked questions#