Test suite auditor
Point it at a Python project. Get a list of weak, flaky, slow and useless tests, checked by mutation.
Test suite auditor is a free AI skill for the job "Test and fix code". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · CC BY 4.0 licence · Made by sickn33 · Level: advanced · Thinking: high
Made by sickn33. See the original on GitHub; we copied it as it is, with its licence.
What it does
A verdict on whether your Python tests would actually catch broken code, with the weak, flaky, slow and useless tests listed. A passing suite stops being a comfort and becomes a number you can trust. A pinned runner measures the suite. The key check is mutation testing: it makes small deliberate breaks in your code and sees whether any test fails. Tests that never catch a break are weak. It also reruns tests to find flaky ones, times each test to find slow ones, and spots tests that duplicate others without adding detection. Quick mode is a fast sample and deep mode is thorough. The skill reports only and changes none of your tests. It needs Python and pytest.
You give it
- Your Python project
- pytest set up
- Mode: quick or deep
- Time you can spend on a run (optional)
You get
- Mutation results showing what tests miss
- A list of flaky tests
- A list of slow tests
- Tests that add no detection
Use it when
- Your tests pass but bugs still reach production.
- The test run takes so long that people skip it.
- Some tests fail at random and you cannot say which.
- You want to prune tests that add cost but no safety.
- You let AI write tests and need to check their value.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run code and read your project files.