AI Blocks › Track your numbers
A/B test verdict maker
Give it your A/B test data. Get a split check, results with confidence ranges and a ship call.
A/B test verdict maker is a free AI skill for the job "Track your numbers". Add it to Claude, ChatGPT, Gemini or Codex as a skill, or paste it into any chat.
Free · No account · MIT licence · Made by Nimrod Fisher · Level: advanced · Thinking: high
Made by Nimrod Fisher. See the original on GitHub; we copied it as it is, with its licence.
What it does
Your A/B test ends with a clear ship, don't ship or keep running call, backed by the numbers. The skill confirms the test setup first: the question, who was split, the main metric and the guard metrics. Then it checks that traffic split the way it should, using a chi-square test. If the split is off, it stops, because the result cannot be trusted. Next it works out each version's rate with a 95 percent range, runs a z-test for rates or Welch's t-test for averages, and repeats the test on guard metrics. A drop in any guard metric blocks the launch. It finishes with the expected business impact and a filled-in report.
You give it
- Your test plan or hypothesis
- Data with user ID, version and outcome
- The main metric and any guard metrics
- The planned traffic split
- A buyer brief (optional)
You get
- A traffic split check
- Results with 95 percent ranges
- Guard metric checks
- A ship, stop or extend call
- A written test report
Use it when
- Your test finished and your team needs a yes or no.
- The result looks positive but you doubt it is real.
- A test has run for weeks with no clear winner.
- You want to size a test before you start it.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run code.