AI Blocks › Track your numbers
Statistical model checker
Give it a fitted Bayesian model. Learn if you can trust its results, and where it breaks, in plain words.
Statistical model checker is a free AI skill for the job "Track your numbers". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by PyMC Labs · Level: expert · Thinking: high
Made by PyMC Labs. See the original on GitHub; we copied it as it is, with its licence.
What it does
Before you quote a Bayesian model's results, you learn whether they deserve trust. The check covers three separate questions: did the sampling run properly, does the model predict data like yours, and do its conclusions make sense for the claim you want to make. It counts divergences, reads R-hat and effective sample size, and looks at trace and rank plots for stuck chains. It then compares simulated data to real data, runs leave-one-out checks and flags unreliable points. It warns against common slips, such as treating comparison weights as probabilities. It works with ArviZ output, so you need Python and a fitted model.
You give it
- Your fitted model output in ArviZ form
- The data it was fitted to
- What you want to claim from the model
- Sampler settings (optional)
You get
- A health report on sampling and precision
- Notes from prediction checks
- Model comparison with uncertainty
- Likely causes of problems and what to try
- A plain-words verdict on what you can claim
Use it when
- Your sampler reported divergences and you do not know if they matter.
- You must pick between two models and want a fair comparison with honest uncertainty.
- A reviewer asks whether your posterior results are reliable.
- Your model fits the data it saw but you doubt it predicts new groups.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run Python code and read your saved model results.