In my last piece I made the case that the bottleneck in experimentation has moved: Generation got cheap. Selection got expensive. The most costly mistake a testing programme can make isn't a failed hypothesis — it's a failed design that never gave the hypothesis a fair trial.
Several people came back with the same question: okay, but what does pre-launch testing actually change? Not in theory but in practice. Which decisions move? Who owns them? What does Monday morning look like differently?
Most teams treat experimentation as the whole discipline — the process that tells you what works. It is. But "what works" has always depended on a quieter, earlier question: was the thing you tested actually ready to be judged?
First, what it isn't
Pre-launch testing is not a replacement for live experimentation. This is the most common misconception and it's worth clearing up before anything else.
Live experimentation answers a strategy question: does this idea resonate with this audience? Pre-launch testing answers a design question: is the execution sound enough to give that idea a fair hearing? They are different questions, producing different evidence, owned by different people, at different points in the process.
Think of it this way. A live experiment is a question you put to your audience. Pre-launch testing is how you make sure the question is legible before you ask it. If users can't find the CTA, the experiment isn't measuring whether your offer resonates. It's measuring whether your design was clear enough. Those are not the same question, and getting them confused is expensive.
What actually changes
Three things shift when teams adopt pre-launch testing seriously.
- Decisions move earlier. Design sign-off happens before launch. The conversation moves from "let's see what the audience says" to "here's what the attention model predicts, and here's whether that matches our hypothesis."
- Different people own different questions. The people making the design and the people evaluating it are not the same. This isn't about distrust - it's about getting a second, independent set of eyes on the work.
- The experiment gets more accurate. When a variant enters a test having already passed a pre-launch evaluation, the test isolates one variable: does the strategy work? It isn't also silently measuring whether users could find the button, read the headline, or notice the offer. Fewer confounds. A more trustworthy result.
What this looks like in practice
Tinuiti, one of the largest independent digital marketing agencies in the US, runs EyeQuant at three stages — not as a final check, but woven through their workflow from the start. During pitches, audits, and design sprints, it moves the conversation from opinion to evidence.
"EyeQuant is usually my first stop before I look at analytics or behavioural tools," says Spencer Gray, who led this work as CRO Program Manager at Tinuiti and is now CRO Manager at WillScot. "It sets the baseline for understanding how a page's visual hierarchy is working."
"EyeQuant transforms 'I think' into 'the data shows' — allowing us to settle debates quickly and secure client buy-in with confidence."
Theresa Farr, another member of the team, describes the organisational shift it produces: "EyeQuant transforms 'I think' into 'the data shows' — allowing us to settle debates and secure client buy-in quickly and with confidence."
Most design debates are not won by the person with the best argument. They're won by the person with the most authority in the room. Pre-launch testing changes that dynamic. The evidence is independent of anyone's seniority.
Jellyfish, a global digital marketing agency running over 300 EyeQuant analyses a year, put this to work during a competitive pitch — without platform access, live data, or tracking tools.
Using EyeQuant, they redesigned a client's page in Figma and validated the changes before presenting. UX score: 56 to 72. Clarity: 24 to 62. Excitingness: 11 to 53. All before a single real user saw the page.
"We were able to validate those changes and show the value of our services before we even had access to the data," says Fabien Caublot, Customer Experience Director.
The part nobody talks about
Adding this evaluative layer into how a team works requires adjustment. And I want to say that plainly, because it's rarely acknowledged in discussions about workflow change.
Pre-launch testing means changing who decides what, when sign-off happens, and what counts as sufficient evidence to proceed. That's a real shift — and it runs into the same friction teams face during any workflow change — but with big pay-offs if completed.
The teams that have made this transition most successfully started small. One team. One decision type. One stage in the process. From there, adoption tends to spread on its own — not because of a mandate, but because the people who saw what the evidence changed carry it forward.
Pre-launch testing doesn't ask your organisation to trust a new tool. It asks it to trust evidence. That distinction is usually what gets the conversation unstuck.
What it doesn't replace
Pre-launch testing validates the design. It helps answer questions like:
- Does attention land where the hypothesis needs it to?
- Does the message register?
- Is the CTA easy to find?
- Does the visual hierarchy support the intended journey?
It does not validate the strategy. Whether the offer is right, the audience is right, whether the behavioural hypothesis holds — that's what live experimentation is for. You need both.
The goal is not to replace the experimentation test. It's to make sure the test is measuring the idea, not the execution.
Next: why generative AI needs an evaluative counterpart — and why the same system should not both produce and judge.