Incrementality testing for Google Shopping, step by step

Your ad account says a product group returns a ROAS of 4.0. It cannot say how many of those orders the ads caused and how many would have arrived anyway. An incrementality test answers that by switching the ads off for a comparable group and counting what happens.

The definition on its own is in the incrementality glossary entry. This page is about running the test on Google Shopping and reading what comes out.

Two groups, one difference

ROAS credits every order that follows an ad. An incrementality test counts only the orders your ads added, by comparing a group that saw them with a comparable group that did not. A test asks the question attribution cannot: what would have happened without the ad?

Picture two groups of 10,000 people. The group that saw your ads placed 200 orders. The group that did not see them placed 150. The ads added 50 orders, and the other 150 would have come anyway.

The test does not say which click earned which order. It says how many orders the ads added in total, and that total is what a budget decision runs on.

How do you calculate incremental ROAS?

Incremental ROAS is incremental revenue divided by ad spend. Incremental revenue is the revenue from the orders the ads added, so it can never be larger than the revenue credited to the ads.

Illustrative data.

The ad spend is €4,000.00 in both ratios. Only the revenue on top changes, so the same campaign is a 4.0 or a 1.0 depending on the question you ask.

Google Ads Help gives the same formula for its Conversion Lift tool: incremental return on ad spend is incremental conversion value divided by total ad spend. The groups need to be the same size. If they are not, scale the control group’s orders to the size of the test group before you subtract.

Holdout, geo split or time-based

A holdout or a geo split gives you a comparison group in the same period, and a time-based test does not. That difference matters more than anything else in the design.

A holdout is the cleanest: you choose, at random, a share of people who will not see the ads. A geo split suits cases where individuals cannot be split. Google Ads Help describes Conversion Lift in both forms, based on users and based on geography, with treatment regions that see your ads and control regions where they are withheld. It also says Conversion Lift is not available for all accounts.

A time-based test is the weakest, because you compare two different weeks. If orders fall when you switch the ads off, you cannot tell the ads from the weather, a competitor’s sale or your own price change. Use it only when nothing else is possible, and treat the result as a hint.

Four things to have before you start

Missing any one of these makes the result impossible to act on.

The control group sets the answer. Keep the 200 orders from the group that saw ads, change only the control, and incremental ROAS swings from 2.0 to 0.2.

Decide with a rule written in advance

Compare incremental ROAS with the product’s break-even ROAS, which is 1 ÷ contribution margin. A result above it means the added orders paid for the ads. A result below it means they did not, whatever the attributed ROAS says.

At a 40% contribution margin, break-even ROAS is 1 ÷ 0.40 = 2.5. The attributed ROAS of 4.0 clears it comfortably. The incremental ROAS of 1.0 does not: the 50 added orders earn €1,600.00 of profit before ad spend, against €4,000.00 spent.

In POAS terms, which is ROAS × contribution margin and has a break-even of 1.0, the attributed return is 1.6 and the incremental return is 0.4. POAS vs ROAS explains the link, and the break-even ROAS calculator gives the floor for your own margin.

Put the other way round, the same €4,000.00 of spend needs a certain number of added orders to pay for itself, and the margin sets that number.

A usable rule has three parts: the number you compare with, what you do above it (increase priority), and what you do below it (lower priority, or fix price and costs first). Add a fourth for when the result is too close to call: run the test again, longer or on a larger group.

Read the result per product group

Two groups with the same ROAS can differ a lot in what the ads added. Group products by something that matches how you run campaigns, such as brand, category or margin band.

One hypothesis to test: products that people already search for by name, and products you retarget, can show a strong ROAS and weak increments. Those shoppers may have been about to buy anyway, so the ads get credit for orders they did not cause. It is a hypothesis, not a finding, and a test is how you check it for your own products.

Each group needs enough orders on its own, so split a test only as far as your volume allows. Product Segmentation segments products by advertising clicks and attributed return, and a test shows how much of that return the ads added.

A simple test plan

Start with one product group and one question, and follow the steps in order. The mistake to avoid is skipping the second step and choosing the rule after the result is in, which turns a test into a story.

When the test ends, calculate incremental orders and incremental ROAS as in the worked example above, and apply your rule. Write down the dates, the split and the result, so the next test can start from them.

What ROAS and MER can’t tell you

ROAS counts the orders the ad platform credits to your ads, and MER counts all shop revenue against all marketing spend. Neither compares what happened with the ads to what would have happened without them.

ROAS can be high because the ads reached people who were going to buy. ROAS says how much revenue each euro of ads is credited with, and what is a good ROAS shows how to read it against break-even.

MER does not depend on one platform’s attribution, but it counts all revenue, including orders that would have come anyway, so it cannot show what the ads added or be read per product. ROAS, MER, POAS and nCAC compares the metrics, and break-even ROAS per product shows how to set the floor for each one.

An incrementality test checks both. It is slower and takes planning, so save it for the decisions that move the most spend.

Test your biggest spender first

The break-even ROAS calculator gives you the floor for the rule.

Pick the product group that takes the most spend and write its decision rule today: its break-even ROAS, and what you will do above and below it. Then plan one holdout or geo split for that group, with a start date and a length you will not change.