AnalysisAdvertising

Your ads might not be tired. Your offer might be.

Separate a weak creative execution from a weak commercial proposition before commissioning another twenty hooks.

AI-generated monochrome still life of four ceramic product vessels arranged as a two-by-two experiment with photographic contact sheets.
CREATIVE × OFFER / FOUR DIFFERENT QUESTIONSAI-GENERATED EDITORIAL ARTWORK
THE USEFUL PART

A better-looking ad and a better offer solve different problems. Test them separately, then judge the result in contribution rather than orders alone.

A falling result is not a diagnosis

The cost per purchase rises. Someone says the creative is tired. Six new hooks arrive, the account gets rearranged, and a few days later the team is looking at another graph with the same problem.

Creative fatigue is one possible explanation. It is not a complete description of everything that can make advertising perform worse. The people receiving the ad may have changed. The auction may cost more. A popular size may be unavailable. The landing page may be slower. Or the offer may have stopped being attractive at its current price.

Those explanations call for different interventions. A new opening shot will not restock the product. A discount can improve response while destroying the contribution that made the campaign worth running. A rising frequency figure can help frame a question about repeated exposure, but it does not, by itself, prove that repetition caused the decline.

The useful research question is narrower: what changed, for whom, and which part of the offer-to-purchase path moved with it? Before commissioning another production sprint, make the competing explanations explicit. That turns a creative refresh from a reflex into an experiment with a chance of teaching you something.

Separate delivery, attention, intention, and economics

Use four layers to organize the evidence. Delivery describes the opportunity to see the ad. Attention describes what happened immediately afterward. Intention describes progress toward the purchase. Economics describes whether the resulting order was worthwhile.

In the delivery layer, inspect spend, impressions, reach, placements, and the audience mix your reporting actually supports. In attention, examine comparable click or viewing measures with their definitions intact. In intention, examine product-page arrivals, checkout starts, purchases, and the elapsed time in which purchases are counted. In economics, examine contribution after the costs that move with those orders.

Do not multiply these observations into a false causal story. Lower click-through and higher frequency appearing together are a reason to investigate, not an identification strategy. The platform may have changed which people it selected, and the remaining audience may be different from the earlier audience.

Keep a dated change log beside the chart: price changes, stock issues, delivery promises, site releases, promotions, tracking changes, and creative launches. If the decline begins when free delivery ends, “the color palette is stale” needs to compete with a much more direct explanation.

This audit can also reveal that the apparent decline is incomplete data. Compare cohorts with equal observation windows. Yesterday's clicks have had less time to produce delayed purchases than last week's clicks. A creative team should not be asked to repair a reporting window.

Test the creative and the offer separately

If the question is whether the execution or the offer needs work, design a comparison that can distinguish them. One proposed setup crosses two creative treatments with two offer treatments. The four combinations make both changes visible instead of bundling them into a single new ad.

This is the logic of a two-factor, two-level full factorial design: each setting of one factor is paired with each setting of the other. NIST's experimental-design guidance describes the general structure. Applying it to advertising still requires suitable assignment, measurement, and delivery controls; four ordinary ads in an optimized campaign are not automatically a randomized experiment. [1]

Keep the changes interpretable. “New creative” could mean a revised product demonstration with the same claim. “New offer” could mean a bundle with explicitly different economics. If the new creative also introduces a different product, price, audience promise, and landing page, you no longer have two clean factors.

Use the experiment mechanism available to the account, with a consistent assignment unit and enough separation to limit people seeing conflicting conditions. Specify the primary outcome and observation window before launch. If the account cannot support the design or the traffic is too sparse, reduce the question. A well-run two-condition test is more useful than four tiny groups dressed up as a research program.

More orders can still be a losing offer

Here is an invented example with 1,000 eligible visits in each condition. Assume the allocation is randomized and each visit represents an independent eligible person for this illustration. Real accounts may need a different unit and analysis.

TreatmentPurchasesPurchase rateContribution per order before adsTotal contribution before ads
Existing creative, existing offer202.0%$35$700
New creative, existing offer222.2%$35$770
Existing creative, new offer313.1%$25$775
New creative, new offer343.4%$25$850

The new offer looks much stronger if the only outcome is purchases. Yet its lower contribution per order makes 31 orders worth only $75 more than the 20-order baseline, before advertising and production costs. The combined treatment produces the largest contribution in this invented sample, but that is an observed result, not a proven winner.

A simple descriptive average gives the creative change a 0.25-percentage-point difference in purchase rate across the two offers. The corresponding offer difference is 1.15 percentage points. Those summaries are useful for reading the table. They do not establish statistical significance, persistence, or what would happen at a different spend level.

If every condition required $900 of acquisition spending, all four would have negative contribution after that spending. A winning conversion-rate chart can therefore sit inside a losing commercial decision. Include variable product, fulfillment, payment, return, and offer costs consistently. Our break-even ROAS tool can help make those assumptions explicit.

FIELD NOTES / A FOUR-CELL EXPERIMENT
EXISTING CREATIVE / EXISTING OFFER20 orders / 2.0%

20 × $35 contribution

$700before advertising
NEW CREATIVE / EXISTING OFFER22 orders / 2.2%

22 × $35 contribution

$770before advertising
EXISTING CREATIVE / NEW OFFER31 orders / 3.1%

31 × $25 contribution

$775before advertising
NEW CREATIVE / NEW OFFER34 orders / 3.4%

34 × $25 contribution

$850before advertising

$900 in advertising per cell puts all four below break-even. More purchases do not settle the economic question.

Invented results from 1,000 visitors per cell. Contribution per order falls from $35 to $25 with the new offer. This illustrates the calculation; it does not establish statistical significance or campaign performance.

Look for interactions, then resist the story

Sometimes the new execution works only with the new offer. That is an interaction: the effect of one change depends on the other. A product demonstration may make a bundle understandable in a way that a lifestyle photograph does not. The combination has a reason to work that neither ingredient has alone.

But a small difference between four noisy estimates is not a license to invent a consumer psychology paper. Write down the plausible mechanism, the competing explanation, and the next test that would separate them. “The new demonstration makes the bundle's contents easier to understand” is testable. “Our audience craves authenticity” is mostly a way to end the meeting.

Check whether the treatment changed the people who arrived. If one ad attracts bargain hunters and another attracts repeat buyers, a post-click comparison may reflect different selection rather than a page treatment's effect. Randomization and assignment need to match the claim you want to make.

Also distinguish a temporary novelty effect from a durable improvement. A fresh execution might help initially and settle later. Report the time pattern without repeatedly moving the evaluation window until the result looks persuasive. If the test needs sequential monitoring, use a method designed for that purpose rather than treating every daily peek as a new fixed experiment. [2]

Give the next creative brief a specific job

The investigation should end with a useful production decision. If attention deteriorated while the offer and purchase path held steady, brief an execution that addresses the opening or demonstration. If the offer created confusion, show the contents, price, eligibility, or tradeoff more clearly. If delivery or tracking broke, fix that before calling the creative team back in.

Keep one surviving control where the design allows it. Retiring every old execution at once makes it harder to tell whether the market changed or your replacements helped. Keep failed concepts too, with a short explanation of what the test could and could not establish.

There is no permanent winner waiting inside the account. There are offers, executions, people, costs, and time. A world-class creative operation learns which of those needs attention before it spends another week polishing the wrong thing.

PUT IT TO WORK / INTERACTIVE TOOLWork through your ad economicsEditable assumptions. Clear calculations. Save your scenario.

Sources & notes

Primary references for the product details and research discussed above. The examples and practical recommendations are our editorial interpretation.

  1. NIST / SEMATECHFull factorial designs ↗

    The e-Handbook describes designs combining every setting of each factor. The advertising example is an original illustration; platform delivery and assignment still need experimental controls.

  2. Johari, Koomen, Pekelis & Walsh / Operations ResearchAlways Valid Inference: Continuous Monitoring of A/B Tests ↗

    Published online August 2021. Explains why ordinary fixed-sample inference can fail under outcome-dependent monitoring and stopping, and develops methods for continuous monitoring.

CHECKED / 11 NOVEMBER 2026

About this article

Prepared by JDGMNT Editorial on 11 November 2026. All visitor counts, order counts, costs, and margins in the four-cell example are invented. Descriptive differences are not significance tests or forecasts of campaign performance. References are dated to the source check above. The artwork was generated for this collection.

← BACK TO RESEARCH

JDGMNT / INDEX

Find a useful idea.

JDGMNT / CONTACT

What could be more accessible?

Tell us about a question, an access barrier, or a possible connection to our work.

Your message goes directly to the JDGMNT team. Please don’t include sensitive personal information.

EXPLORE JDGMNT

MORE KNOWLEDGE. MORE POSSIBILITY.
MAKING ACCESS POSSIBLE.