Incrementality: how to tell new sales from moved ones
A practical way to check whether a new channel adds sales or just reshuffles the ones you already had.
- A channel can report many sales without adding any.
- Incremental sales are the ones that would not have happened without the channel.
- Simple tests by market or period are enough to get a reliable answer.
The problem with attributed sales
Every channel reports the sales it touched. Add them up and the total is often higher than your actual revenue. That is because the same shopper may click a Shopping ad, a newsletter and a retargeting banner before buying. Each channel takes credit, but only one sale happened.
- DEFINITION Incremental sale
- A sale that would not have happened without a given channel. It is the true value that channel adds.
Attribution tells you which channels were present before a sale. Incrementality tells you which ones caused it. The two answers can be very different, especially for channels that reach shoppers who were already going to buy, such as brand search or retargeting.
| Attribution | Incrementality | |
|---|---|---|
| Question it answers | Which channels touched the sale? | Would the sale have happened anyway? |
| Method | Tracking and attribution rules | Controlled comparison |
| Typical bias | Over-credits channels close to purchase | Needs time and a fair control group |
| Best for | Day-to-day optimisation | Budget decisions between channels |
How to test it
You don’t need a data science team. You need a fair comparison between a group that saw the channel and one that didn’t.
- Pick a split. Compare similar markets, regions or periods, with the channel switched on in one and off in the other.
- Fix the window. Run the test long enough to cover normal weekly swings, usually four weeks or more.
- Compare total sales. Look at all sales in each group, not just the ones the channel reports.
- Calculate the lift. The difference between the two groups is your incremental effect.
Three test designs
| Design | How it works | Watch out for |
|---|---|---|
| Market holdout | Run the channel in some markets, not in similar ones | Markets must be comparable in size and season |
| Regional holdout | Switch the channel off in some regions of one country | Delivery and shopping habits can differ by region |
| On/off periods | Alternate weeks with and without the channel | Seasonality and promotions can distort weeks |
Market and regional holdouts are usually the most reliable, because both groups run at the same time and face the same season, prices and promotions.
Reading the results
If total sales barely move, the channel is mostly taking credit for sales you already had. If they rise, the channel is adding demand. Compare the incremental sales with what the channel cost to see its real cost per new sale.
| Illustrative result | Value |
|---|---|
| Incremental orders | 150 |
| Channel cost | 3.000€ |
| Cost per reported sale (400 reported) | 7,50€ |
| Cost per incremental sale | 20€ |
In this example the channel reports 400 sales, but only 150 are new. Its real cost per new sale is 20€, not 7,50€. That is the number to compare with your margin.
Common mistakes
- Stopping the test after a week because early results look good or bad.
- Comparing markets of very different sizes or seasons.
- Running a big promotion in only one of the groups.
- Looking only at the channel’s own reported sales instead of total sales.
Why comparison traffic tends to be incremental
Shoppers on comparison sites are often still choosing where to buy. Reaching them there can win orders that would otherwise go to a competitor. Testing confirms it for your own catalogue.
How big a test needs to be
Over four weeks, a test comparing two groups can only reliably detect an effect of about twice your normal week-to-week variation or more. Before you start, look at the last few months and see how much weekly sales move on their own.
| Illustrative shop | Weekly orders | Normal weekly swing | Lift you can detect in 4 weeks |
|---|---|---|---|
| Large | 2.000 | ±5% | Around 10% or more |
| Medium | 500 | ±10% | Around 20% or more |
| Small | 100 | ±20% | Around 40% or more, or a longer test |
Smaller shops can still test. They just need longer tests, bigger groups or a bigger expected effect.
Other ways to estimate incrementality
| Method | Good for | Limits |
|---|---|---|
| Holdout tests | A clear answer for one channel | Needs time and comparable groups |
| Platform lift studies | Ads platforms with built-in test tools | Measured by the platform itself |
| Marketing mix modelling | Long-term view across all channels | Needs years of data and expertise |
Turning results into budget decisions
- Rank channels by cost per incremental sale. Not by cost per reported sale.
- Move budget gradually. Shift 10% to 20% at a time and check the effect.
- Retest once a year. Channels change as competitors, prices and platforms change.
Before you start
- Agree what you measure: orders, revenue or margin.
- Freeze big changes such as price campaigns during the test.
- Write down the expected result, so the test can confirm or reject it.
- Decide in advance what result would make you scale, adjust or stop.
An example six-week plan
| Week | What happens |
|---|---|
| 0 | Choose test and control markets, check they performed similarly in recent months |
| 1–4 | Channel on in test markets, off in control markets |
| 5 | Collect total orders and revenue for both groups |
| 6 | Calculate the lift and the cost per incremental sale, then decide |
Quick answers
How long should a test run?
At least four weeks for most catalogues, longer if your sales are seasonal or low in volume.
Can I test without switching a channel off?
Yes. Holding out a market or region while the channel runs elsewhere works well.
What lift is good?
It depends on your costs. Compare the cost per incremental sale with your margin.
How many orders do I need?
Enough that normal week-to-week swings are small compared with the effect you expect. Low-volume shops need longer tests.
Should I test every channel?
Start with the largest budgets and the channels closest to purchase, where over-crediting is most likely.
Get the next shift in European e-commerce before it hits your numbers.
One email a month on Google Shopping, comparison sites and AI search. No spam.
Controller: Komparoo. Purpose: sending our newsletter. You can unsubscribe or exercise your rights at any time. More in our privacy policy.
Sources: Komparoo. Figures in the examples are illustrative.
