Conversion rate optimisation checklist

Most conversion advice assumes you can split test. Below roughly a hundred conversions a month you cannot — a test will not reach significance in a useful timeframe, so you end up acting on noise.

So the obvious fixes come first here. For most sites they are the whole programme, and they are where almost all the gain is.

The checklist

4 groups · 23 checks

Before testing anything 5

• Conversion tracking fires once and matches your records
• You know your current conversion rate, by device, from your own data
• You have watched at least ten session recordings of real visitors
• You have read what people actually write in the enquiry form or on the phone
• You have submitted your own form on a phone, this month

The obvious things, before any test 8

• The page states what you do and who for, above the fold
• One primary action per page, not three of equal weight
• The form asks only for what you need to respond — every extra field costs completions
• Errors are shown inline, in words, next to the field that caused them
• Prices, or a "from" price, are visible somewhere
• Proof appears near the action, not only on a separate page
• Page loads under 2.5s on mobile, and nothing moves after it loads
• The thank-you page confirms what happens next and when

Running a test properly 6

• You have a written hypothesis: because we observed X, changing Y will do Z
• Sample size calculated in advance, and you know how long it will take
• One change per variant, so a result tells you something
• The test runs for whole weeks, so weekday and weekend behaviour are both included
• You agreed the stopping rule before starting, and you do not peek and stop early
• The metric is the real conversion, not a click on a button

After the test 4

• The result is recorded whether it won, lost or was inconclusive
• Inconclusive is written down as inconclusive rather than reported as a small win
• The winning variant is actually deployed, and you checked it a week later
• What you learned is written in a sentence someone else could use

Do these before you test anything

Group two of the checklist is the part worth reading twice, because none of it needs a tool, a developer or a sample size.

The two that recover the most: the form asks only for what you need to respond to someone — every extra field costs completions, and “how did you hear about us?” costs more than it teaches — and errors are shown inline, in words, next to the field that caused them. A form that rejects a submission without saying why is indistinguishable from a broken one.

Prioritising with ICE

When you do have a list of ideas, score each one and work down. The workbook does the arithmetic and ranks them.

Axis	The question it asks	The scale
Impact	If this works, how much would it move the primary metric?	1 = barely measurable · 5 = would change the quarter
Confidence	What evidence do you have that this is a real problem?	1 = someone’s opinion · 5 = seen in recordings, data and enquiries
Ease	How much work is it to build and ship?	1 = needs a rebuild · 5 = a copy change this afternoon

Confidence means evidence, not enthusiasm. A 5 requires having seen the problem in session recordings, in the data, and in what people actually say. An idea scoring high on impact and low on confidence is not a test — it is a research task, and treating the two as the same is how testing programmes fill up with inconclusive results.

The test log, and the number that stops you fooling yourself

The log calculates each variant’s rate and the uplift, then returns a verdict. It is deliberately conservative:

• Fewer than 100 conversions per arm returns “Not enough data” rather than a result.
• A difference within 5% returns “Inconclusive” rather than a small win.

That is a sanity check, not a significance calculator — use a proper significance test before acting on a close result. But it exists because the most common CRO failure is stopping a test early, reporting noise as a win, and shipping a change that does nothing.

“The calculator says 95% significant” — the trap

This is the most expensive misunderstanding in conversion testing, and it catches careful people.

A significance calculator will happily return 95% on a few hundred visits and a handful of conversions. The number is arithmetically correct and practically meaningless, for two reasons:

Small samples produce large apparent differences by chance. With 12 conversions against 18, the observed uplift is 50% and the true difference may be nothing at all. The calculator is answering “how surprising is this gap, given these totals?” — it is not answering “will this hold”.

Stopping when it turns green guarantees a false positive. Significance fluctuates day to day as data arrives. If you check daily and stop the first time it crosses 95%, you have selected for a lucky moment rather than measured an effect. This is why the stopping rule has to be agreed in advance, and why the checklist asks for it before the test starts.

That is why the test log ignores the calculator and applies a blunter rule: under 100 conversions per arm it says “Not enough data”, and within 5% it says “Inconclusive”. Blunt beats precise-and-wrong here.

Common mistakes