AI Blocks › Track your numbers
Experiment result analyst
Give it test numbers. Get the right test, sample size and whether the result is real, in plain words.
Experiment result analyst is a free AI skill for the job "Track your numbers". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Alireza Rezvani
Made by Alireza Rezvani. See the original on GitHub; we copied it as it is, with its licence.
What it does
A plain verdict on whether your test result is real or just noise, and whether to ship, hold or keep running. It matches the test to your data: a z-test for conversion rates, a t-test for averages, chi-square for categories. It reports the p-value, a confidence interval and an effect size, and asks two separate questions: is the result statistically significant, and is it big enough to matter? Before a launch, it calculates how many users each variant needs, given your baseline and the smallest change worth detecting. It also flags traps like peeking early and running many comparisons. Small Python scripts do the maths.
You give it
- Sample size per group
- Observed rates or averages
- Baseline and smallest change that matters
- Decision the result feeds into (optional)
- Python for the included scripts (optional)
You get
- Chosen test with reasons
- P-value, interval and effect size
- Required sample size
- Plain-language verdict
- Ship, hold or extend advice
Use it when
- Your A/B test finished and you need to know if the winner is real.
- You are about to launch a test and need the sample size.
- Someone shared a result and asked if it is significant.
- You suspect a test was stopped too early.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run code.