Two-proportion z-test · not a gut call

You Killed A Winning Ad Because Variant B Was 18% Ahead. It Was Noise.

Four days in, one creative is clearly out front in the spreadsheet, so you shift the budget. A week later the gap has vanished and you cannot say whether you backed the right one or paid to learn nothing. Paste your numbers in here and you get a straight answer: the lead is real at 96% confidence, or it needs roughly 4,000 more impressions per variant before you call it.

Buy Now →

One-time $19 · works offline · nothing leaves your browser

6 metrics
CTR · CPC · CPM · conv. rate
CPA · ROAS, computed per variant
0 API calls
pure local maths in your browser
no accounts, no uploads, no keys
$19
once, for the tool itself
no seats, no subscription, no credits

The spreadsheet tells you who is ahead. It never tells you whether that means anything.

Every creative test ends the same way. Four variants, a column of percentages, and a decision that has to be made on Thursday. So you pick the top row. The trouble is that a difference between two rates is only meaningful once you have enough of them, and nothing in a spreadsheet will ever tell you when that moment arrives:

  • A 20% lift on 300 impressions is nothing. The same lift on 40,000 impressions is a decision. Eyeballing the percentage cannot tell those two situations apart, and both look equally convincing in a cell.
  • Small conversion counts lie constantly. Nine conversions against six feels like a 50% win. It is well inside the range you would expect from two identical ads.
  • CTR and conversion rate disagree. The hook that wins the click often loses the sale, and one of those numbers is the one that actually pays.
  • Nobody catches fatigue in time. Your best performer quietly slides off its own peak over ten days, and because it is still top of the table it keeps the budget.
  • Rebuilding the formulas every test. CPA, CPM and ROAS get retyped into a fresh tab each time, and one wrong cell reference sends the wrong ad to the client.
You do not need a data scientist. You need the one number that says “call it” or “keep running”.

It does the statistics you would do if you had the time

Enter what your ad platform already gives you: spend, impressions, clicks and conversions per variant. Everything after that is computed for you.

A verdict, in plain English

A two-proportion z-test runs between the current leader and every other variant on the metric you pick. You get “Winner: Variant B, 96% confidence, call it” or “No clear winner yet” — with the z-score and p-value underneath if you want to show your working to a client.

How much longer to run it

When the lead is not yet real, it tells you roughly how many more impressions or conversions each variant needs at current rates. That turns “let’s give it a few more days” into a number you can put in a status update.

Every rate, side by side

CTR, CPC, CPM, conversion rate and CPA for each variant, plus ROAS as soon as you add a revenue-per-conversion figure. Sortable by any column, with the best value in each one highlighted so the trade-offs are obvious.

Creative fatigue, caught early

Log the same variant again in a few days and it tracks that variant’s CTR across every reading. When it drops more than 20% off its own peak you get a warning and a trend line, before the ad quietly stops working.

Every test, kept

Tests are saved in your browser and listed on a dashboard with their platform, status and winner. Reopen a test from six weeks ago, add another reading, or clean up the ones that are done.

Client-ready in one click

“Copy as Markdown” gives you a clean summary table and the verdict, ready to paste into a Slack update or an email. There is a full CSV of every variant across every test, and a JSON backup you can restore on another machine.

What it looks like

The same three variants, judged on CTR and then on conversion rate. One of those answers is a decision; the other is a warning.

Q3 retargeting hooks
Meta · RT-EU-Q3
VariantSpendImpr.CTR CPCConv.CPAROAS
A — Founder to camera $520.40119,9001.09% $0.4047$11.0710.84×
B — Customer testimonial $524.10120,3001.60% $0.2782$6.3918.78×
C — Numbers on screen $358.4082,5001.32% $0.3341$8.7413.73×
Winner: B — Customer testimonial, 100% confidence, call it.
Primary metric: CTR · leader 1.60% on 120,300 impressions · beats A at z = 10.82, p < 0.001 and C at z = 5.04, p < 0.001.
Switch the primary metric to conversion rate: no clear winner yet.
B leads at 4.27% against C’s 3.76%, but only 50% confidence (z = 0.95, p = 0.499). Needs roughly 22,400 more clicks per variant before that gap means anything. Same three ads, opposite decision.

Illustrative figures from the built-in example test, not a screenshot of your data.

What you get

  • Unlimited tests, each with unlimited creative variants
  • CTR, CPC, CPM, conversion rate, CPA and ROAS per variant
  • Sortable comparison table, best value in each column highlighted
  • Two-proportion z-test on CTR or conversion rate, your choice
  • Confidence level, p-value and z-score for every pairing
  • Sample-size guidance when the lead is not yet significant
  • Inline bar comparison — no chart library, no tracking
  • Creative-fatigue warning at 20% off a variant’s own peak
  • Copy as Markdown for a client update or Slack
  • CSV of every variant across every test
  • JSON export and import for backup and restore
  • Installable on your phone or desktop, works offline

“The maths here is the same two-proportion z-test that every A/B testing platform runs behind the scenes. The difference is that it works on the numbers your ad platform already gives you, in a file you own, without a seat licence or a data pipeline.”

Built by Mulkern AI Systems — practical tools for people who run marketing, not dashboards.

One file. One payment.

No subscription, no seats, no usage credits. You buy the tool and it is yours.

$19
One-time purchase
  • The complete tool, as a single self-contained file
  • Install it to your home screen and use it offline
  • Your data stays in your browser — no account to create
  • Use it on every client and every campaign you run
Buy Now →
Instant access after payment. No refunds — read what it does above before you buy.

Before you buy

Does it connect to Meta or Google Ads?

No. There are no API calls of any kind. You type or paste in the spend, impressions, clicks and conversions from whatever report you already look at. That is deliberate: no OAuth, no tokens to rotate, no access to your ad accounts, and it keeps working when a platform changes its API.

What statistical test does it run?

A two-tailed two-proportion z-test with a pooled standard error, on either CTR (clicks over impressions) or conversion rate (conversions over clicks). Sample-size guidance assumes 95% significance and 80% power. It compares the current leader against each other variant, one metric at a time.

Where is my data stored?

In your browser’s local storage, on the device you are using. Nothing is uploaded and there is no server to hold it. That also means clearing your site data will delete it, so export the JSON backup if a test matters.

How many variants can I compare?

As many as you like, though the statistics get less reliable the more you run at once: with enough variants one will look like a winner by chance alone. Two to four per test is the sensible range, and the tool says so in the verdict panel.

Does it work on my phone?

Yes. The comparison table collapses into stacked cards on narrow screens, and it installs to your home screen as an app that runs offline.