Incrementality Testing

Your retargeting campaign is the best line in the account. Some of those buyers were already going to purchase. They had your product in the cart and were getting paid Friday regardless of the ad.

Incrementality testing is the measurement that separates customers you converted from customers who were already converting. The mechanic is a holdout: one group sees your ads, another doesn’t. The difference in conversions between them is the lift your advertising actually caused.

How it shows up in the wild

Haus — 640 Meta experiments across 18 months: The measurement platform published The Meta Report drawing on 640 controlled incrementality experiments. The average lift to primary KPIs across the pool was ~19%.

For DTC brands using 7-day click attribution, Meta under-reported incrementality by 15% on average. Every $100 attributed at the 7DC window corresponded to $115 in actual incremental revenue — click attribution is conservative on prospecting, not generous.

Retargeting ran the other direction. Platform-reported ROAS on retargeting campaigns ran 40–70% above true incremental ROAS, because retargeting captures buyers already in motion. Prospecting and retargeting can show opposite attribution errors inside the same account.

BrandAlley — 4-week Meta Conversion Lift Study: BrandAlley is a UK fashion ecommerce brand running more than 1,000 digital campaigns a year. Their Marketing Mix Model had estimated Meta ROI at 3.91. They ran a 4-week Conversion Lift Study across all Meta campaigns to check whether the model was right.

The study returned a Meta ROI of 4.00 (90% confidence interval: 2.91 to 5.09). The test wasn’t run to improve the Meta number. It was run to decide whether the MMM could be trusted for future budget decisions.

Google Ads — minimum test budget lowered to $5,000: Google reduced the minimum spend required to run a native Conversion Lift experiment from approximately $100,000 to $5,000, using Bayesian statistical methods that prioritize probability over certainty. Tests now reach statistical significance up to 50% more frequently. For D2C brands previously priced out of Google’s native testing, the change makes geo holdout and Conversion Lift experiments accessible at mid-market spend levels.

Why it matters

Platform ROAS describes credit allocation, not causation. It answers “which ad was nearby when the conversion happened?” — not “did this ad change whether the conversion happened?” Incrementality testing is the distinction between those two questions.

My hunch is that most brands running heavy retargeting would find their true incremental ROAS meaningfully below what Ads Manager reports. The gap tends to be largest where buyer intent was already high before the ad appeared.

Related terms

• Attribution Window — the time period in which a conversion is credited to an ad; window choice drives the gap between attributed ROAS and incremental ROAS
• Cross-Channel Attribution — assigns credit across platforms; incrementality testing is the causal check on whether any attribution model’s numbers reflect reality
• Brand Lift Study — measures awareness and consideration rather than conversion lift; a different question from incrementality, but the same holdout mechanic
• Marketing Efficiency Ratio — total revenue divided by total spend; a blended signal that incorporates incrementality indirectly but doesn’t isolate it by channel
• ROAS (Return on Ad Spend) — the platform figure incrementality testing is most commonly used to stress-test

Frequently asked questions

How is incrementality testing different from A/B testing? A/B tests compare two versions of something — creative, landing page, offer — against each other. Incrementality tests compare ad exposure versus no exposure at all. A/B testing tells you which version of an ad performs better. Incrementality testing tells you whether running ads is moving the outcome.

What is a geo holdout? A geo holdout divides your market by geography — states, cities, or postal codes. Ads run normally in treatment regions and are paused in control regions. After the test window, you compare sales across regions. Because geography is the variable rather than individual users, geo holdouts work without cookie or pixel data, making them the most privacy-robust incrementality method available to D2C brands post-iOS 14.

How much budget do I need? Google’s native Conversion Lift tool now requires a $5,000 minimum. Meta’s Conversion Lift Studies need sufficient conversion volume — roughly 100+ conversions per week per ad set for reliable results. Geo holdouts via Triple Whale’s GeoLift module or Meta’s open-source GeoLift package can run at lower spend levels, though tests take longer to reach statistical significance.