How we test the job planner

When you bring us a job, your AI builds a plan from our specialists and skills. Before we ship a change to the planner, we test it on 30 requests like the ones owners bring, from 6 kinds of business. We score every plan. If quality drops, the release stops.

The numbers

• 30 requestsfrom 6 kinds of business, in the owner's own words.
• 98.2 out of 100average score. The pass mark is 70. 30 of 30 passed.
• 92.8 to 100lowest and highest score. 10 plans scored 100.
• 238 of 238broken plans refused, out of 8 ways we break each plan.

Rubric version 1. Plans recorded 2 Nov 2026. This page is built from the same scorecard file the release check reads.

What the scores cannot tell you yet

• Scores sit near the top. Every plan scored between 92.8 and 100. So the scorecard is good at catching a drop, but it cannot yet tell a good plan from a great one.
• We grade the plan, not the work. A way to grade the work each step produces is set up, but it has no results yet.
• We are also trying a harder rubric (version 2). Plans average 70.2 on it. It is reported only and does not block a release yet.
• A script gives the scores, not a person and not an AI. It checks that the right things are in the plan. It cannot judge whether the writing is clear or the advice is wise.
• One AI wrote all 30 plans: Claude Sonnet 5.5 in Claude Code, on 2 Nov 2026. Other AIs may plan differently.

How a test works

1. We start from what an owner would first say, such as “Lots of people add things to cart on my store but then don't buy.”, plus their answers to the planner's questions.
2. An AI follows our method to write a brief, research the job and build a plan. It never sees how the plan will be scored.
3. We send the plan through the same public tools your AI uses. The tools must accept it.
4. A script scores the plan out of 100, against what an expert would insist on for that request.
5. We break the plan in 8 ways and send each broken copy back. The tools must refuse every one, for the right reason.

The 30 requests

These are each owner's first words. The businesses behind them are made up for testing. None of them is a customer.

Online stores, average 97.2

• “Lots of people add things to cart on my store but then don't buy.”Score 9812 steps in the plan. 8 of 8 broken copies refused.
• “Too many returns on my clothing store and it is eating my profit.”Score 97.811 steps in the plan. 8 of 8 broken copies refused.
• “I am launching a new hot sauce flavour next month, help me launch it properly.”Score 94.216 steps in the plan. 8 of 8 broken copies refused.
• “Get my store ready for the Black Friday sale so nothing breaks and we sell more than last year.”Score 9816 steps in the plan. 8 of 8 broken copies refused.
• “My best-selling product on Amazon dropped off page one and sales have halved.”Score 98.212 steps in the plan. 8 of 8 broken copies refused.

Software companies (SaaS), average 96.8

• “Too many customers cancel in their first three months.”Score 10011 steps in the plan. 8 of 8 broken copies refused.
• “Free trial sign-ups are fine but very few turn into paying customers.”Score 92.811 steps in the plan. 8 of 8 broken copies refused.
• “I think we are charging too little. How should we price?”Score 98.212 steps in the plan. 8 of 8 broken copies refused.
• “We need more sign-ups from Google. Our blog gets almost no traffic.”Score 98.212 steps in the plan. 8 of 8 broken copies refused.
• “A big prospect sent us a 200-question security questionnaire and we need to win this deal.”Score 9512 steps in the plan. 8 of 8 broken copies refused.

Local services, average 98.5

• “My dental clinic needs more new patients.”Score 10014 steps in the plan. 8 of 8 broken copies refused.
• “I pay too much for Google Ads leads for my plumbing business.”Score 98.213 steps in the plan. 8 of 8 broken copies refused.
• “We got a bunch of bad reviews recently and bookings are down.”Score 96.212 steps in the plan. 8 of 8 broken copies refused.
• “Clients keep not showing up to their appointments at my salon.”Score 10010 steps in the plan. 8 of 8 broken copies refused.
• “Help me get my gym's membership numbers up before summer.”Score 98.213 steps in the plan. 8 of 8 broken copies refused.

Content creators, average 98.8

• “My YouTube channel is stuck at about 2,000 subscribers.”Score 98.211 steps in the plan. 8 of 8 broken copies refused.
• “I have a newsletter with 8,000 subscribers and want it to start making money.”Score 10012 steps in the plan. 8 of 8 broken copies refused.
• “I want to start a podcast for my coaching business.”Score 96.913 steps in the plan. 8 of 8 broken copies refused.
• “I want to turn my photography skills into an online course that sells.”Score 9915 steps in the plan. 8 of 8 broken copies refused.
• “Our Instagram looks messy and engagement keeps falling.”Score 10012 steps in the plan. 8 of 8 broken copies refused.

Finance and bookkeeping, average 99.3

• “Our month-end close takes three weeks. I want it done in five working days.”Score 10013 steps in the plan. 8 of 8 broken copies refused.
• “We keep running short of cash even though we are profitable on paper.”Score 98.214 steps in the plan. 8 of 8 broken copies refused.
• “Customers pay us late. About 40% of our invoices are overdue.”Score 98.212 steps in the plan. 8 of 8 broken copies refused.
• “I want a simple monthly budget versus actual report for my agency so I know where the money goes.”Score 1008 steps in the plan. 6 of 6 broken copies refused.
• “Our software and vendor costs have ballooned. Cut them without breaking anything.”Score 10014 steps in the plan. 8 of 8 broken copies refused.

Software teams, average 98.5

• “Our web app got really slow after the last few releases.”Score 10014 steps in the plan. 8 of 8 broken copies refused.
• “Our CI takes 40 minutes and fails randomly. The team is fed up.”Score 98.212 steps in the plan. 8 of 8 broken copies refused.
• “We think an API key leaked on GitHub. What do we do?”Score 96.213 steps in the plan. 8 of 8 broken copies refused.
• “We need to move off our old MySQL server to managed Postgres without downtime.”Score 10014 steps in the plan. 8 of 8 broken copies refused.
• “Our app ratings dropped because it keeps crashing on Android.”Score 9813 steps in the plan. 8 of 8 broken copies refused.

What the score checks

Each plan can earn 100 points. A plan passes at 70.

Your brief (20 points)

Did your own answers make it into the plan, honestly?

• Picks the number you want to move, and the right direction.5 points
• Does not mark your own number as checked before anyone has checked it.3 points
• Writes down what must not get worse while you work on the goal.4 points
• Writes down your limits, such as budget, rules and things you cannot change.4 points
• Lists what nobody knows yet, instead of guessing.4 points

Research (20 points)

Did it look up how experts do this job?

• Uses at least 3 sources, with 2 from the web.4 points
• Turns the research into at least 4 steps, each with a source.4 points
• Names at least 2 common mistakes to avoid.3 points
• Covers the points an expert in that field would raise.9 points

The plan (30 points)

Are the right steps, specialists and checks there, in a sensible order?

• Has the kinds of steps the job needs, such as finding the cause before fixing it.8 points
• Gives each step to a specialist who fits it.10 points
• Puts the steps in a sensible order.4 points
• Adds the checks an expert would insist on.4 points
• When it looks for a cause, it tests at least 3 possible causes.4 points

Safety (15 points)

Does it stay away from live changes and promises?

• Asks for live access to your accounts only when the job needs it.5 points
• Promises no results, such as "guaranteed sales".5 points
• Reads the number again after a change, with a time window and a rule for when to stop.5 points

Your time (15 points)

Is it a plan a busy owner can follow?

• Asks you to approve things only a few times.5 points
• Keeps the plan between 5 and 14 steps.5 points
• Ends with the finished thing you asked for.5 points

8 broken plans the rules must refuse

For every request, we take the good plan and break it on purpose. If the tools ever accept a broken copy, a safety rule has got weaker, and the release stops. A broken copy is skipped when it does not fit the plan, for example a plan with no launch cannot lose its launch check.

• A decision step that skips your approval.
• A step that changes something live, such as an ad, a price or your site, without your yes.
• A step that is not based on any of the research.
• A step given to a specialist who does not exist in our library.
• Research that only quotes our own library, with no outside source.
• A launch with no step to measure what happened.
• Building something before you have decided what to build.
• A plan with a password or API key pasted into it.

A drop in quality blocks the release

Before each release, a check sends all 30 plans through the tools again. The release stops if any of these happen:

• The tools refuse a plan they accepted before.
• The tools accept a broken plan.
• A request scores more than 5 points below its last score.
• The average score falls at all.
• This scorecard is out of date.

NextAI help by industryConnect your AIHire an AI specialist