Every ad platform will happily take credit for a sale. The question none of them can actually answer on their own is whether that sale would have happened anyway, without the ad ever running. That single question, the counterfactual, is what incrementality testing is specifically built to answer, and it's the piece of the measurement puzzle that platform-reported attribution simply cannot provide, no matter how sophisticated the attribution model gets.
Why platform metrics can't answer this on their own
Attribution models measure correlation: someone saw or clicked an ad, that same person later made a purchase, the platform links the two events together and reports it as an attributed conversion. What attribution models fundamentally cannot measure is the counterfactual, what would have happened to that specific customer if the ad had never been shown to them at all.
A customer who was already going to buy from your brand, whether driven by existing brand loyalty, word of mouth from a friend, or their own organic search for your product, can still get credited to a retargeting ad they happened to see somewhere along their path to purchase. That credited revenue is not incremental, it's revenue that was going to materialize regardless, simply wearing an ad platform's attribution tag because the timing happened to overlap.
What incrementality testing actually does
The standard, widely used approach is a geo-holdout or audience-holdout test. You deliberately withhold ads from a defined portion of your market, either a set of geographic regions or a randomly selected, held-out audience segment, while running campaigns normally everywhere else, for a defined test period. You then compare actual revenue generated between the exposed group and the holdout group over that same window.
The difference in revenue between the two groups is your real, measured incremental lift, the actual additional revenue the ad campaign caused, as opposed to revenue it merely touched or was present alongside. This is a genuinely different number from platform-reported attributed revenue, and it is frequently smaller, sometimes substantially smaller, than what the platform's own dashboard suggests the campaign generated.
Why this matters most for retargeting and branded campaigns
Retargeting and branded search campaigns tend to show the largest gap between platform-reported ROAS and true incremental lift, and there's a clear structural reason for this. These campaign types disproportionately reach people who are already deep in the decision process, people who have already visited your site, added items to a cart, or are actively searching for your brand by name. These are exactly the customers most likely to convert with or without the ad ever being shown to them.
A holdout test run specifically on a retargeting campaign frequently reveals that a meaningful share of its "attributed" revenue would have converted anyway, through organic return visits, direct navigation to the site, or branded search that the customer would have performed regardless of whether a retargeting ad reached them first.
A worked scenario
Consider a brand running an always-on retargeting campaign that shows a strong, consistent 8x ROAS in the platform dashboard, the kind of number that would normally justify aggressively increasing its budget. Running a 2-4 week geo-holdout test on this specific campaign, pausing retargeting ads in a set of comparable regions while continuing normally elsewhere, might reveal that the incremental lift from that spend is meaningfully smaller than the platform ROAS number implies, because a real portion of those "recovered" cart abandoners or site visitors were converting anyway through other paths.
This doesn't mean the retargeting campaign is worthless, defensive retargeting genuinely does have value, capturing customers who might otherwise get distracted by a competitor's ad in the meantime, or simply reinforcing brand recall at a low cost. But it does mean the platform-reported ROAS overstates the case for aggressively scaling that specific line item, and budget decisions made purely on the dashboard number would likely over-invest in it relative to its true incremental contribution.
How to run a basic incrementality test without enterprise infrastructure
You don't need a dedicated data science team or expensive third-party measurement platform to get a directional read. A simple geo-holdout, pausing ads in a handful of regions that are reasonably comparable to your active regions in terms of size, demographics, and typical sales volume, for a defined period of a few weeks, while running normally everywhere else, provides a workable starting point.
Compare total revenue in the held-out regions versus a similarly-sized set of active regions over the same period, ideally using regions with a track record of similar baseline performance before the test began, so the comparison isn't distorted by pre-existing differences between the areas chosen.
Run this periodically, not as a one-time exercise, on your largest budget lines specifically, retargeting and branded search being the highest priority, since that's consistently where the gap between platform-reported performance and true incremental lift tends to be largest and most financially significant.
What this looks like for a cold-audience prospecting campaign versus retargeting
Cold-audience prospecting campaigns, targeting people who have no prior relationship with your brand and haven't visited your site before, typically show a much smaller gap between platform-reported attribution and true incrementality. These customers genuinely wouldn't have converted without some form of marketing exposure, since they had no existing awareness or intent. This is an important asymmetry: incrementality testing tends to validate cold prospecting spend more readily than it validates retargeting and branded spend, which is often the opposite of what teams assume when they see prospecting campaigns reporting lower ROAS than retargeting.
Common mistakes brands make with incrementality testing
Never testing at all, and treating platform ROAS as automatically equivalent to incremental value. This is the default state for most accounts, and it's precisely the gap this testing framework exists to close.
Testing once and treating the result as permanent. Incrementality can shift over time as your brand awareness changes, as competitors' activity changes, and as your customer base matures. A test run a year ago may no longer reflect current reality.
Choosing holdout regions that aren't actually comparable to the active regions, introducing bias into the comparison before the test even begins. Matching regions on prior baseline sales volume and demographics as closely as possible is essential for a clean read.
Testing only cold prospecting campaigns and skipping retargeting and branded search, when those latter two campaign types are precisely where the biggest, most financially consequential gaps tend to exist.
FAQ
Do I need a specialized platform to run incrementality testing? No, a basic geo-holdout can be set up manually using your existing ad platform's geographic targeting controls and your own sales data for comparison. Specialized measurement platforms exist and can add sophistication, but they're not a prerequisite to getting started.
How long should a geo-holdout test run? Typically 2-4 weeks at minimum, long enough to smooth out day-to-day noise and capture a representative sales cycle for your category, though longer-consideration categories may need a longer test window.
Does a lower incremental lift mean I should cut the campaign entirely? Not necessarily. Retargeting with lower measured incrementality can still have genuine defensive value. The point of the test is to right-size budget allocation based on real incremental contribution, not to eliminate every campaign that shows some non-incremental revenue.
How often should incrementality tests be repeated? Periodically, ideally at least annually for your largest budget lines, and sooner if you notice a significant shift in brand awareness, competitive activity, or overall business growth that might change underlying customer behavior.
The takeaway
Platform metrics can't tell you what would have happened without the ad, they can only tell you what happened alongside it. A simple geo-holdout or audience-holdout test, run periodically on your largest budget lines, especially retargeting and branded search, gives you a real, directional read on incremental lift, and that number should carry more weight in budget allocation decisions than platform-reported ROAS alone.