Hire an AI support bot error reviewer

Everything it can hand back

Your task sets the actual outputs. The specialist picks the methods that cover them.

• A failure list with one row per failure, the chat IDs behind it and the approved source it breaks
• Likely causes, each marked as a guess until confirmed
• A draft fix request for each failure, ready for the person who maintains the bot
• Retest cases with the question, the expected facts, the allowed hand-off to a person and the pass test
• A status for each earlier finding, such as new, unchanged, seen fixed or untested

How it does the job well

• Check the current policy and the customer's order or account facts before replying.
• Answer the actual question in the first two lines, then explain.
• Match the customer's tone. Acknowledge the problem once, without grovelling.
• Hand refunds outside policy, legal threats and safety issues to a person.

Mistakes it avoids

• Promising refunds, dates or fixes the policy does not allow.
• Copy-paste replies that ignore what the customer said.
• Closing a ticket without a clear next step.

What a good result looks like

An example, not your task: Your support bot gave a wrong answer on cancellations, refunds or prices, and you want to know how often.

• Scope and limits. The chats reviewed, the date range and how they were picked, so rates are read as this sample only.
• Failure list. One row per failure with the chat IDs, what the bot said, the source it breaks and its status.
• Likely causes. Each cause with evidence for and against, marked as a guess until someone confirms it.
• Draft fix requests. One request per failure saying what to change, why, and any open question for the bot owner.
• Retest cases. A question to ask, the facts a right answer needs, the allowed hand-off to a person and a pass test.
• Missing inputs and checks. Each missing input and what it blocks, plus PASS, FAIL or NOT RUN for every check.

How the work is checked

• Every reply states the facts it relies on and is checked against the policy.
• Escalations name the reason and the owner.
• Repeated issues are grouped so the root cause can be fixed.
• States which checks passed, failed or could not run, and names missing inputs.

These checks have not been run against a model yet. Your AI uses them to check its own work.

Capabilities

• Compare answers with evidence. Uses your chats, approved sources and scope to mark each chat as resolved or not and the answer as right or wrong, citing the chat turn and the source. Pasted exports work offline. Hands related failures on as clusters.
• Separate symptoms from likely causes. Looks at when and on what topic the failures bunch up, and at any tickets you linked. It lists evidence for and against each cause, and labels unverified causes as guesses. Hands the ranked list on.
• Create retestable fix requests. Writes one issue per distinct failure, with the redacted question, what the bot said, the source facts it should have used, when to pass to a person, and a pass or fail case. Keeps evidence IDs and open questions.
• Track earlier findings. Merges repeats with your earlier list, and marks each item new, unchanged, seen fixed or untested. A changed article is not counted as a fix.

How the role works

1. Checks your task, context and available tools.
2. Picks the methods needed and prepares a work plan.
3. Carries out supported tasks, with each method's limits and review gates.
4. Checks the real deliverables and corrects supported faults.
5. Hands back files, evidence and unresolved gaps.

Your specialist's working setup

Download the base specialist without a task form or account. README.md and SETUP.md explain how to use the folder. SOUL.md defines its scope. INTAKE.md asks missing questions inside your AI. PLAYBOOK.md, SKILLS.md, BEST_PRACTICES.md, CHECKS.md and HANDOFF.md guide the work and its checked deliverables.

Complete associated skills, supporting files and licences are included. Choose only the methods needed for the actual task. Keep context, working files and accepted feedback in your AI project.

Use this specialist through MCP or the API

Call prepare_specialist with role_id review-support-bot-errors-with-retest-cases only. No task is required to load it. Your AI asks what job you need done, what you want back and which relevant context it can use.

For task-led hiring, call find_specialist, explain the suggested specialists and let the user accept or change the team.

Complete ZIP: Download base specialist. API: GET /api/v1/specialists/review-support-bot-errors-with-retest-cases/pack. The host AI supplies its own model, tools and execution; a pack does not connect accounts or run a task.

Inputs and helpful context

The specialist starts from the task brief. Only active methods require their inputs.

For the primary method, when used

• Redacted chat logs with an ID, a date and every turn of each chat (export from your bot tool)
• The approved help pages, policies and escalation rules the bot should follow, each with its effective date
• The review scope, meaning the date range, how the chats were picked, and the topics to cover
• Your earlier findings list, or the word NONE on a first run (optional)

Helpful context (optional)

• A short list of the questions customers ask most, so the review can check those first
• The date of your last bot or help page change, so failures can be lined up with it

Tools and access

• Claude: Handles a long set of chats plus several help pages in one run, and keeps the evidence IDs straight.
• Your bot tool's export: The chat logs come from your own bot tool, such as a Zendesk or Intercom export or a custom bot's log. The agent does not connect to your account.
• Botsonic, Docsbot or Landbot: It reads chat logs from your own account.
• Composio CLI: It links your AI to your chatbot account. Your AI must be able to run commands.
• Your help center articles, pasted in: They give the review a source of truth. Without them accuracy is not checked.
• Gorgias for ticket lookup; pasted notes also work: Live lookups need Composio CLI access and connected accounts for these services.

Functions and deliverables

• Support bot error reviewer: Check your support bot chats for wrong answers. Paste redacted chats and approved help text. You get a failure list, fix requests and retest cases.
  • A failure list with one row per failure, the chat IDs behind it and the approved source it breaks
  • Likely causes, each marked as a guess until confirmed
  • A draft fix request for each failure, ready for the person who maintains the bot
  • Retest cases with the question, the expected facts, the allowed hand-off to a person and the pass test
  • A status for each earlier finding, such as new, unchanged, seen fixed or untested
• Support chatbot conversation reviewer: Find where your support chatbot fails. Review recent chats to get missed questions, wrong answers and help articles to add or fix.
  • A count of chats resolved, partly resolved and failed
  • A table of failed chats with the question, the bot's reply and the issue
  • The topics asked about most that the bot could not answer
  • Made-up answers, set against what is true
  • Help articles to add and handover rules to set
  Source: Composio.
• Support issue cause investigator: Find the root cause of a support issue. Give it 3 or more related tickets and get the shared pattern, a likely cause, evidence and a fix plan.
  • A likely root cause, stated in specific terms.
  • An evidence table that shows what each ticket tells you.
  • An impact summary, with Unknown where your data has no number.
  • Fixes listed by priority and owner.
  • A temporary workaround agents can share.
  Source: Composio.
• Bug report writer: Turn logs, screenshots, or rough notes into a GitHub issue draft with steps and impact.
  • A GitHub issue draft with a one-line summary and steps to repeat
  • Expected and actual results side by side
  • A severity rating with a short reason
  • Placeholders for sensitive details such as project names and user IDs
  • A draft saved as a dated markdown file, if your AI can save files
  Source: sickn33.
• Brand profile builder: Share your website. Get a reusable profile covering your products, customers, voice and approved claims, with sources.
  • A folder of eight plain files, from a start file for your AI to a claims list and a change log
  • A tag on every fact, and every claim marked approved, pending or banned
  • A two-sentence diagnosis of your biggest gap, and a list of what's missing and who can answer it
  • A short slice for your next step, if you name one

Tasks you can use it for

• Your support bot gave a wrong answer on cancellations, refunds or prices, and you want to know how often.
• You changed a help article and want a list of questions to ask the bot to see whether it now answers correctly.
• You are about to hand a list of bot problems to the person who maintains the bot and want each one to have evidence.
• You review bot chats every month and want repeat failures tracked instead of found again from scratch.

Limits to check before starting

• You want the bot or your help pages edited. This reviewer only proposes; a person makes each change.
• Your chats still carry names, emails or order numbers. Remove them first. It works on redacted copies only.
• You have no approved help text to check against. It can sort topics and flag risks, but it cannot call an answer wrong.
• You want to find the questions your help pages miss. Use the Support help article writer.
• You want one transcript scored once. Use the Support chatbot conversation reviewer.

Best way to use this role

Give it one clear outcome, the actual files and a definition of done. Keep the work in one AI project so it can reuse checked context and earlier files.

• Export the chats from your bot tool and remove customer names, emails, phone numbers and order numbers. Keep a chat ID and date on each one.
• Gather the help pages and policies the bot is meant to follow. Put the date each took effect at the top of each.
• Start a chat in Claude, ChatGPT or Gemini. Attach the agent pack, or paste the skill, then attach your files. Write the scope in a few lines, and paste your earlier findings or NONE.
• Read the failure list first. Open two chat IDs from it and check that the evidence says what the row claims.
• Treat every cause as a guess until the person who owns the bot confirms it. Give them the fix requests, not the raw chats.
• After the bot is changed, run each retest case in the bot yourself. Paste the new answers back and run the agent again to mark what passed. It cannot run the bot for you.

Read it as data

Read the complete role setup as JSON. Connect your AI to the AI Blocks MCP library and read aiblocks://roles/review-support-bot-errors-with-retest-cases for its role instructions and pinned sources.

This is a prepared AI role. No task has run. Your AI carries out authorized work using its available tools and reports the actual deliverables and checks.