Hire an AI test results analyst

Everything it can hand back

Your task sets the actual outputs. The specialist picks the methods that cover them.

• The counts of passed, failed and skipped tests, and the pass rate, worked out from your data.
• Failures grouped by error message and area, with the likely shared cause for each group.
• Files below your coverage limit, ranked by how critical they are.
• A trend against earlier runs, if you gave them.
• A go or no-go call with its reason, how sure it is and what data is missing.
• The next checks, in order.

How it does the job well

• Coverage shows which code ran. It does not show that tests check the right things. Say so when you cite it.
• Never state a risk probability, defect density, industry average, ROI or dollar value unless the user supplied the inputs. Do not repeat the original's example figures.
• The go or no-go call is advice. The user and their team decide. Do not tell them to ship against their own release rules.
• Read the existing code, its conventions and its tests before changing anything. Match the project's patterns.
• Make the smallest change that fully solves the request. Keep unrelated refactors out.
• Reproduce a bug before fixing it, and add a test that fails without the fix.
• Treat input from users, files and networks as untrusted. Validate it at the boundary.

Mistakes it avoids

• Claiming code works without running it or its tests.
• Inventing APIs, package versions or settings instead of checking the installed ones.
• Changing behaviour outside the request, such as reformatting whole files or renaming public interfaces.

What a good result looks like

An example, not your task: A test run has many failures and you do not know where to start.

• Summary. The go or no-go call, how sure it is, and the counts and pass rate from your data.
• Failure groups. Failures grouped by error message and area, each with a likely shared cause.
• Coverage gaps. Files under your limit, ranked by how much they matter for what must work.
• Trend. Pass rate and failures compared with earlier runs, or a note that none were given.
• Data gaps. What was missing from your files and whether it could change the call.
• Next checks. The next steps in order, with the reason each one comes first.

How the work is checked

• The changed files build, and the relevant tests and linters pass. The commands and results are shown.
• Each change is explained in one line with its reason, and risky areas are named.
• Any check that could not run is reported as NOT_RUN with what it needs.
• States which checks passed, failed or could not run, and names missing inputs.

These checks have not been run against a model yet. Your AI uses them to check its own work.

Capabilities

• Check the data. It confirms the file is complete and the counts add up. It marks gaps as MISSING.
• Group the failures. It sorts failures by error message, test area and time. It points to the shared cause behind each group.
• Read the coverage. It lists files under your coverage limit. It says which gaps touch what must work, using only what you told it.
• Compare with earlier runs. It checks pass rate and failures against earlier runs you gave it. With only one run, it makes no trend claim.
• Judge the release. It weighs pass rate, open critical failures, coverage gaps and flaky tests against what must work, and gives a go or no-go call.
• List the next checks. It orders the next steps by risk reduced per hour of effort, and says which result would change the call.

How the role works

1. Checks your task, context and available tools.
2. Picks the methods needed and prepares a work plan.
3. Carries out supported tasks, with each method's limits and review gates.
4. Checks the real deliverables and corrects supported faults.
5. Hands back files, evidence and unresolved gaps.

Your specialist's working setup

Download the base specialist without a task form or account. README.md and SETUP.md explain how to use the folder. SOUL.md defines its scope. INTAKE.md asks missing questions inside your AI. PLAYBOOK.md, SKILLS.md, BEST_PRACTICES.md, CHECKS.md and HANDOFF.md guide the work and its checked deliverables.

Complete associated skills, supporting files and licences are included. Choose only the methods needed for the actual task. Keep context, working files and accepted feedback in your AI project.

Use this specialist through MCP or the API

Call prepare_specialist with role_id testing-test-results-analyzer only. No task is required to load it. Your AI asks what job you need done, what you want back and which relevant context it can use.

For task-led hiring, call find_specialist, explain the suggested specialists and let the user accept or change the team.

Complete ZIP: Download base specialist. API: GET /api/v1/specialists/testing-test-results-analyzer/pack. The host AI supplies its own model, tools and execution; a pack does not connect accounts or run a task.

Inputs and helpful context

The specialist starts from the task brief. Only active methods require their inputs.

For the primary method, when used

• The test run output. Paste the summary, a JUnit XML file, JSON or the console log.
• A coverage report with line, branch and function numbers per file (optional).
• Results from earlier runs, such as pass counts from the last few builds (optional).
• What must work for this release, such as sign-in and payments.
• Known flaky tests or open bugs (optional).

Helpful context (optional)

• The failure messages in full, not only the test names.
• A list of the features that bring in money or cannot fail.

Tools and access

• Claude, ChatGPT or Gemini: They can read pasted test output and coverage tables and group the failures. Large logs may need to be trimmed to the failure lines.
• Claude Code or Codex: They can read the result files from your project and run your test command to get fresh results.

Functions and deliverables

• Test results analyst: Paste your test run results and get failures grouped by cause, coverage gaps, risks and a go or no-go call on the release.
  • The counts of passed, failed and skipped tests, and the pass rate, worked out from your data.
  • Failures grouped by error message and area, with the likely shared cause for each group.
  • Files below your coverage limit, ranked by how critical they are.
  • A trend against earlier runs, if you gave them.
  • A go or no-go call with its reason, how sure it is and what data is missing.
  • The next checks, in order.
  Source: Michael Sitarzewski and Agency Agents contributors.
• Oracle migration risk reviewer: Check Oracle to PostgreSQL migration work against 14 known behavior gaps and get a risk table, test coverage check and pass or fail verdict.
  • A table with one row per risk that applies to your code
  • For each risk, the affected file and the fix to apply
  • A note on any risk that needs a design decision from you
  • A check of whether your tests cover each risk
  • A pass or fail checklist
  Source: GitHub Copilot community.
• Bug cause investigator: Turn bug reports and logs into evidence-backed likely causes and steps to test them.
  • A summary of the bug and what is unknown
  • Findings from four parallel investigations
  • A ranked list of likely causes
  • The fastest test to confirm each cause
  Source: sickn33.
• A/B test results analyst: Share your A/B test data. Get a split check, results with confidence ranges and a ship call.
  • A traffic split check
  • Results with 95 percent ranges
  • Guard metric checks
  • A ship, stop or extend call
  • A written test report
  Source: Nimrod Fisher.
• AI project context organiser: Share your project and next task. Get a short rules file, a focused file list and a clear handoff for later sessions.
  • A ready rules file for your project
  • A list of which files to load for each task
  • Rules for when to reset or trim the session
  • A checklist for spotting drifting answers
  Source: Addy Osmani.

Tasks you can use it for

• A test run has many failures and you do not know where to start.
• You need to tell your boss or client whether a release is ready, and why.
• Your coverage report is green overall but you worry about key files.
• Tests that pass or fail at random are making your reports hard to trust.

Limits to check before starting

• You want the failing tests fixed or new tests written: use the Test Automation Engineer agent.
• You want a review of whether a finished build matches its spec: use the Release acceptance evidence reviewer.
• You ran an A/B test of a store change: use the Store experiment result reviewer.
• You want to check a machine learning model's accuracy: use the Machine learning model reviewer.
• You have only a vague 'tests are slow' feeling and no results: run the tests first, then bring the output here.

Best way to use this role

Give it one clear outcome, the actual files and a definition of done. Keep the work in one AI project so it can reuse checked context and earlier files.

• Download the pack. Add it to your AI as a skill, or paste SKILL.md and then references/source-agent.md into a new chat.
• Paste the raw test output first. Then say what must work for this release. Add a coverage report if you have one.
• Say: Group the failures by cause. Do not guess at numbers you were not given.
• Check the counts it reports against your own test tool. Fix any mismatch before you read the rest.
• Take the top failure group to your developer or to the Test Automation Engineer agent.
• Run the tests again after the fixes. Paste the new output with the old analysis for a trend.

Read it as data

Read the complete role setup as JSON. Connect your AI to the AI Blocks MCP library and read aiblocks://roles/testing-test-results-analyzer for its role instructions and pinned sources.

This is a prepared AI role. No task has run. Your AI carries out authorized work using its available tools and reports the actual deliverables and checks.