Nobody publishes the accounts that went sideways, so agency case studies are all survivors. That is fine. The question is not whether the deck is selective — of course it is — but whether the selected example contains enough information to be evidence at all.
Four questions do almost all of the work.
1. What was the baseline?
"Increased ROAS by 220%" is not a result. It is half of one. A return that moved from very poor to merely acceptable is a different achievement from one that improved on an already strong account, and the first is far easier. Ask for the before number, not just the delta.
The same applies to percentage improvements on small bases. Tripling leads from four a month to twelve is real, but it is not the same claim as tripling from four hundred, and the percentage hides which one happened.
2. Over what period, and against what?
Three comparisons are commonly used and only one of them is honest in most cases:
- Against the previous period. Vulnerable to seasonality. An ecommerce account compared October against July will look brilliant for reasons that have nothing to do with the agency. Festive demand and Q4 in Western markets both do this.
- Against the same period last year. Better, because it holds seasonality roughly constant.
- Against a holdout or a geo test. Rare, and the only one that isolates the work from everything else happening in the business.
Also check how long the window is. A one-month win can be a good week and a lucky creative. Six months is harder to fake.
3. What else changed at the same time?
This is the question agencies least enjoy. If the client also launched a new product, cut prices, got press, hired a sales team or fixed a checkout flow during the same quarter, the media work is one of several causes and probably not the largest. A case study that names those other changes is more trustworthy than one that implies the ads did everything.
Attribution inside an agency case study is the same problem as attribution inside your account, with the added difficulty that the person doing the attributing is the one being paid.
If you want to understand why this is genuinely hard rather than merely convenient, attribution models and the gap between platform ROAS and blended MER cover the mechanics.
4. Which number is being reported?
Platform-reported ROAS, blended return, contribution margin and profit are four different things, and they get further apart the more you spend. A case study quoting platform ROAS from inside Meta is quoting the number the platform most wants to claim credit with. Ask what the business's actual revenue did, and whether the finance team would recognise the figure. The blended ROAS calculator shows how large that gap gets.
The questions behind the questions
Two more are worth asking even though they are not about the case study itself. First: is this client still with you? A spectacular result from a client who left eight months ago raises an obvious follow-up. Second: who worked on it, and do they still work here? Agency results travel with people, and the case study on the website may describe a team that has since dispersed.
What good agency case studies have in common
It names the starting position, the time window, the comparison basis, what was changed, what else was going on, and ideally something that did not work along the way. It reports a number the client's finance team would recognise. It is boring in places. Nothing in that list is difficult to produce — which is why its absence is informative.
The most reliable substitute for a case study is work on your own account. An audit of your actual setup tells you more about how an agency thinks than any slide about somebody else's, which is why we offer a free audit before any commitment and publish how we work rather than only what we achieved. The rest of the due diligence is in the questions to ask on the first call.
Questions people ask before hiring
What is the biggest red flag in a marketing case study?
A percentage improvement with no starting number. Without the baseline you cannot tell whether the account went from broken to adequate or from good to excellent, and those are very different pieces of evidence.
Should I trust ROAS figures in a case study?
Treat them as platform-reported unless stated otherwise. Platform ROAS counts conversions the platform claims credit for, which is consistently more generous than what the business actually banked.
Is it reasonable to ask to speak to the client in a case study?
Yes. Many agencies will arrange it for at least one reference. A refusal is not proof of anything, but a willingness usually means the relationship ended well.
Why do agencies compare against the previous period rather than last year?
Because it is easier to produce and often flatters the result. Year-on-year comparison holds seasonality roughly constant, which is why it is the fairer basis and the less common one.
What if an agency has no case studies in my industry?
Not disqualifying on its own. Ask instead how they would approach your category and what they think the constraint is. Category knowledge can be learned in weeks; diagnostic ability cannot.