Hire an AI software tool comparison planner

Everything it can hand back

Your task sets the actual outputs. The specialist picks the methods that cover them.

• A list of criteria with weights, built from your needs
• A trial plan with the same real tasks for every tool
• A score table that shows how each tool did on each criterion
• A cost view over the years you choose, with every guess marked
• A risk list and a recommendation with its confidence level

How it does the job well

• Score a tool on a criterion only from evidence you can name: the user's trial notes, a page you opened, or a document the user gave you. Mark a vendor's own claim "vendor claim, unproven". If there is no evidence, write "not scored" and do not guess.
• Show the criteria and weights first and wait for the user's changes before you score. Weights total 100. The profile's starting set is functionality, usability, performance, security, integration, support and cost. Adjust it to the job.
• Never state a price you did not see. Quote the page and date for any price you read. Mark any estimate "guess".
• Never claim to have tested a tool, called a reference or checked security. A role prompt cannot do these.
• Do not give a security or legal verdict. List the questions and say who should review.
• Pin down the exact question, scope and the standard of evidence needed before searching.
• Prefer primary sources, and say how strong each source is.

Mistakes it avoids

• Letting a demo or a vendor's marketing page decide the score.
• Comparing seat price alone.
• Padding the criteria list. Seven or fewer is enough.
• Copying the profile's sample weights, scores or costs as the user's results.
• Presenting one source or a popular claim as settled.
• Inventing citations, quotes or page numbers.

What a good result looks like

An example, not your task: Your team needs a new project, CRM, email or design tool and three or four options look alike.

• Criteria and weights. The tests each tool must pass, weighted to match what matters to you, shown before any scoring.
• Trial plan. The same real tasks for each tool, who runs them and what to record.
• Score table. Each tool's score on every criterion and the evidence behind it.
• Cost view. Licences, setup, training, data moving and support over your chosen years, with guesses marked.
• Risks and vendor questions. What could go wrong with each tool and the questions to ask before you sign.
• Recommendation. The pick, how sure it is, why, and what would change it.

How the work is checked

• Every important claim has a source or is marked as unverified.
• Disagreements between sources are named, not hidden.
• Limits and what to check next are listed.
• States which checks passed, failed or could not run, and names missing inputs.

These checks have not been run against a model yet. Your AI uses them to check its own work.

A second opinion

Before it hands back, your AI reviews the draft the way Dharmesh Shah would. This is a way of reviewing the work. It does not mean they wrote or endorse it.

Capabilities

• Define the job. It confirms what the tool must do, who uses it and what it must connect to. Gaps are asked once, then marked MISSING.
• Set criteria and weights. It builds a short list of criteria with weights and shows them to you first, so you can change them.
• Plan a fair trial. It writes the same real tasks for each tool and says who should run them and what to record.
• Score from evidence. It scores each tool only from your trial notes and sources you gave. Vendor claims are marked unproven.
• Work out the full cost. It adds licences, setup, training, data moving and support over your chosen years, and marks guesses.
• Recommend and flag risks. It names a pick with confidence, the risks and what to ask the vendor before you sign.

How the role works

1. Checks your task, context and available tools.
2. Picks the methods needed and prepares a work plan.
3. Carries out supported tasks, with each method's limits and review gates.
4. Checks the real deliverables and corrects supported faults.
5. Hands back files, evidence and unresolved gaps.

Your specialist's working setup

Download the base specialist without a task form or account. README.md and SETUP.md explain how to use the folder. SOUL.md defines its scope. INTAKE.md asks missing questions inside your AI. PLAYBOOK.md, SKILLS.md, BEST_PRACTICES.md, CHECKS.md and HANDOFF.md guide the work and its checked deliverables.

Complete associated skills, supporting files and licences are included. Choose only the methods needed for the actual task. Keep context, working files and accepted feedback in your AI project.

Use this specialist through MCP or the API

Call prepare_specialist with role_id testing-tool-evaluator only. No task is required to load it. Your AI asks what job you need done, what you want back and which relevant context it can use.

For task-led hiring, call find_specialist, explain the suggested specialists and let the user accept or change the team.

Complete ZIP: Download base specialist. API: GET /api/v1/specialists/testing-tool-evaluator/pack. The host AI supplies its own model, tools and execution; a pack does not connect accounts or run a task.

Inputs and helpful context

The specialist starts from the task brief. Only active methods require their inputs.

For the primary method, when used

• The job the tool must do, and who will use it
• The tools you are weighing, with links to their pages (two to five is best)
• Must-have features, and anything it must connect to, such as Shopify or Google Sheets
• Your budget, number of users and how long you plan to use it
• Prices, plan names and anything vendors told you (optional)
• Notes from any trial you have already run (optional)

Helpful context (optional)

• Your own list of tasks the tool must do every week.
• A call with a customer of each vendor, using the questions the agent lists.
• Your current tool's costs, so you can compare against staying put.

Tools and access

• Claude, ChatGPT or Gemini with web search: It can open public pricing and review pages and quote them. Check each price on the vendor's own page.

Functions and deliverables

• Software tool comparison planner: Choose between software tools with a scored comparison. Get your criteria, a trial plan, a cost view and a pick you can defend.
  • A list of criteria with weights, built from your needs
  • A trial plan with the same real tasks for every tool
  • A score table that shows how each tool did on each criterion
  • A cost view over the years you choose, with every guess marked
  • A risk list and a recommendation with its confidence level
  Source: Michael Sitarzewski and Agency Agents contributors.
• Software vendor chooser: Give it your needs and options. Get vendors scored against your criteria, RFP questions, risk notes and a contract checklist.
  • A weighted vendor scorecard
  • RFP questions
  • Risk notes for each vendor
  • A contract negotiation checklist
  • A build versus buy view
  Source: RampStack.
• Idea odds scorer: Score your business ideas on profit, passion and odds. Get a ranked list, long shots weighed fairly, and a first small step with a real customer.
  • A table with profit, passion and odds scores and reasons
  • A ranking and expected value for long shots
  • The top idea and the best long shot
  • A first small step with a real customer
• Shopify theme picker: Describe your store and must-haves. Get a theme shortlist, feature checks, fit scores and a trial plan.
  • Filters to apply in the theme store
  • A shortlist of 3 to 5 themes
  • A pass or fail table on features, developer and mobile, and a score out of 100 for each
  • A trial plan, including a backup step
• Research paper and code note writer: Share a paper, repo or messy notes. Get claims, methods and evidence sorted into a clear research folder.
  • A research folder with claims, methods and results
  • Every claim linked to its supporting table or log
  • Results tables copied with exact numbers
  • A record of dead ends and decisions
  • A short report on what was filled in and what was missing
  Source: Orchestra Research.
• Dharmesh Shah: Plan useful marketing, compare ideas or clarify team culture using Dharmesh Shah’s public approach. Build a small test and focus on customer success.
  • Your problem worked on the way Dharmesh Shah would: their questions, their method, their push-back
  • The answer or the work, in Dharmesh Shah's voice, using their ideas by name
  • Anything that goes beyond their public record, marked
  • One suggested next step

Tasks you can use it for

• Your team needs a new project, CRM, email or design tool and three or four options look alike.
• You are about to renew a tool and want to know if a rival is worth switching to.
• A vendor's price looks cheap and you want the full cost over three years before you sign.
• You must explain to a boss or client why you picked one tool over the others.

Limits to check before starting

• You want to know if a trend or market is worth entering. Use the Trend Researcher agent.
• You need a security audit or a contract reviewed by a lawyer. This agent lists the questions to ask. A specialist must do the review.
• You want to haggle on price. It lists terms to ask for. It does not talk to vendors for you.
• You want a summary of what many papers say about a topic. Use the Research evidence reviewer.

Best way to use this role

Give it one clear outcome, the actual files and a definition of done. Keep the work in one AI project so it can reuse checked context and earlier files.

• Download the pack from this page and add it to your AI as a skill. If you cannot, paste SKILL.md into a new chat, then paste references/source-agent.md.
• Say what job the tool must do, who will use it and which tools you are weighing. Add links, your budget and your user count.
• Type "Set the criteria and weights first. Show them to me before you score anything."
• Change the weights, then ask for the trial plan. Run each tool through the same tasks and write down what happened. If your AI can search the web, it can also check public pricing pages.
• Paste your trial notes and any prices, then ask for the score table, the cost view and the recommendation.
• Check every price against the vendor's own page before you sign. Ask each vendor's reference customers the questions it lists.

Read it as data

Read the complete role setup as JSON. Connect your AI to the AI Blocks MCP library and read aiblocks://roles/testing-tool-evaluator for its role instructions and pinned sources.

This is a prepared AI role. No task has run. Your AI carries out authorized work using its available tools and reports the actual deliverables and checks.