Hire an AI store experiment results reviewer

Everything it can hand back

Your task sets the actual outputs. The specialist picks the methods that cover them.

• A call on a finished test, with the lift as a range and the monthly gain in profit
• The next test, with its sample, weeks, main metric and guardrails
• A ranked list of ideas and a list of things to repair instead of test
• A test log that grows each run

How it does the job well

• Work from the real catalogue, prices, stock and policies. Check variants and currencies.
• Judge changes by contribution margin, not revenue alone.
• Change one thing at a time on a live store and keep a way to roll back.
• Keep product claims consistent with supplier data and labels.

Mistakes it avoids

• Planning a second test on a page that already has one running.
• Stopping a test early because it looks good.
• Calling a result a win without the range and the guardrails.
• Ranking ideas that have no evidence.
• Editing live products, prices or themes without approval or a backup.
• Ignoring shipping, returns and fees when judging profit.

What a good result looks like

An example, not your task: You run tests on your store and want a log, a ranked list and one test at a time.

• State. Whether no test is running, one is running, or one just ended, and what was done about it.
• Result. The call on a finished test, trust score, lift as a range and monthly gain in profit.
• Next test. The idea, page, smallest lift worth shipping, sample, weeks, main metric and guardrails.
• Fix list. Broken things to repair instead of test.
• Backlog. The top five ranked ideas, and parked ideas with the reason.
• Look again on. The date to check back and the numbers to bring.

How the work is checked

• Only one test is planned per page, and none while another is running there.
• Nothing broken is on the test list.
• Every ranked idea has at least two kinds of evidence.
• A surprising win is marked RE-TEST, not SHIP.
• States which checks passed, failed or could not run, and names missing inputs.

These checks have not been run against a model yet. Your AI uses them to check its own work.

A second opinion

Before it hands back, your AI reviews the draft the way Peep Laja would. This is a way of reviewing the work. It does not mean they wrote or endorse it.

Capabilities

• Read the folder. Finds out if no test is running, one is running, or one just ended. A running test is left alone.
• Set the baseline. Works out gross profit per visitor once a quarter. Every test is judged against it.
• Read a finished test. Checks the split, the length and any early stop, gives the lift as a range, turns it into money, and makes a ship, do not ship, re-test or inconclusive call.
• Fill a thin backlog. If fewer than two ideas have two kinds of evidence, it reads your recordings and heatmap notes for more.
• Pick the next test. Ranks the ideas by evidence and profit. At 30 to 300 orders a month it picks bold tests only.
• Check it can run. Sends broken things to a fix list. If the test would take more than about four weeks, it tries the next idea, up to three.
• Plan the setup. Says what your Shopify plan allows and which route fits, then writes the plan and stops for your yes.

How the role works

1. Checks your task, context and available tools.
2. Picks the methods needed and prepares a work plan.
3. Carries out supported tasks, with each method's limits and review gates.
4. Checks the real deliverables and corrects supported faults.
5. Hands back files, evidence and unresolved gaps.

Your specialist's working setup

Download the base specialist without a task form or account. README.md and SETUP.md explain how to use the folder. SOUL.md defines its scope. INTAKE.md asks missing questions inside your AI. PLAYBOOK.md, SKILLS.md, BEST_PRACTICES.md, CHECKS.md and HANDOFF.md guide the work and its checked deliverables.

Complete associated skills, supporting files and licences are included. Choose only the methods needed for the actual task. Keep context, working files and accepted feedback in your AI project.

Use this specialist through MCP or the API

Call prepare_specialist with role_id store-test-program-runner only. No task is required to load it. Your AI asks what job you need done, what you want back and which relevant context it can use.

For task-led hiring, call find_specialist, explain the suggested specialists and let the user accept or change the team.

Complete ZIP: Download base specialist. API: GET /api/v1/specialists/store-test-program-runner/pack. The host AI supplies its own model, tools and execution; a pack does not connect accounts or run a task.

Inputs and helpful context

The specialist starts from the task brief. Only active methods require their inputs.

For the primary method, when used

• Orders, sessions, conversion rate, average order value and gross margin for the page and the store
• Your Shopify plan, who builds changes, and the ideas you have
• Numbers from a finished test, with planned sample and dates, and what else happened during it
• Heatmap notes, recordings, survey answers or review quotes (optional)

Helpful context (optional)

• A note of every sale, email send and ad change during the test
• Heatmap or recording notes for the pages you want to test

Tools and access

• Claude Code: It can keep the tests folder and read the log each run.
• Google Analytics: Export sessions, orders and conversion rate by page. GA4 sessions are cleaner than Shopify's, which include bots.
• Shopify: Theme tests and order exports for the gross profit baseline.
• Shopify: A paid plan. Net sales, shipping collected, tax, discounts and orders come from its reports.
• GA4: Sessions with fewer bots, and traffic split by source.
• Microsoft Clarity: Free heatmaps, area maps and recordings with the rage and dead click insights this skill reads.
• GA4: Free funnels in the Explore tab, and sessions you can compare with your platform dashboards.
• A store survey app: Gives you the 'why' behind what the heatmap shows, tied to order data.
• Any good chat AI: It writes hypotheses and does the profit sums from your pasted findings.
• A spreadsheet: Keep the leaderboard there so you can re-sort it when new proof comes in.
• Any good chat AI: It picks and orders the tests from your numbers and findings.
• GA4: Sessions and landing page traffic without most bot traffic.
• Any good chat AI: It does the sums and reads your numbers. Check the final sample size in a free sample size calculator.
• GA4: It gives sessions without most bot traffic, which Shopify's own sessions include.
• Shopify: Its help pages show what your plan allows in checkout today. Check them before you buy anything.
• Any good chat AI: It sorts your ideas and turns vendor pages into the questions to ask.
• Any good chat AI: It does the checks and the money sums from your pasted numbers.
• Your testing tool's export: Users per version, conversions and dates come straight from its results page or a CSV.

Functions and deliverables

• Store experiment results reviewer: Track store tests, review actual results and plan one supported next test. Keep running tests and weak conclusions in check.
  • A call on a finished test, with the lift as a range and the monthly gain in profit
  • The next test, with its sample, weeks, main metric and guardrails
  • A ranked list of ideas and a list of things to repair instead of test
  • A test log that grows each run
• Store profit per visitor calculator: Paste your Shopify and analytics numbers. Get corrected revenue and gross profit per visitor, and a trust score.
  • Revenue per visitor and gross profit per visitor, as given and corrected
  • A blended conversion check showing the effect of your cold traffic share
  • The link between conversion and the cost of getting a customer
  • What to track weekly, and a trust score out of 100 with the sum shown
• Heatmap and session recording analyst: Share heatmaps, page data and recording notes. Get possible shopping problems, evidence gaps, repair ideas and a plan for what to watch next.
  • A friction list ranked by page and money at stake
  • A hypothesis and a test idea for each problem, and a fix list for repairs
  • A watch plan for recordings: which, how many and when to stop
  • A friction score out of 100, now and after the fixes, with the sum shown
• Test idea ranker: Paste your research findings. Get each test written as a hypothesis and ranked by evidence and profit.
  • Each idea written as a hypothesis with a main metric and a smallest lift worth shipping
  • A ranked leaderboard with evidence points and a profit estimate
  • The one or two tests to run first, and the rest parked with a reason
  • A fix list for anything broken
• Bold store experiment planner: Share store traffic, orders and page findings. Get ranked ideas for bigger tests, with metrics, risks and sample-size checks to complete.
  • Five bold tests in order, each with the reason, main metric, guardrails and risk
  • What not to test at your size
  • A month-by-month plan
  • A reminder to size each test before you run it
• Store test readiness checker: Check if your store can run an A/B test. Give your orders, visits and the change. Get a verdict, weeks needed and a readiness score.
  • A verdict: repair first, research only, big swings, regular tests or full testing
  • Visitors and conversions needed per version, and the weeks needed at your traffic
  • The right main metric, with guardrails
  • A readiness score out of 100, now and after the fixes, with the sum shown
• Shopify test setup picker: Find out what you can A/B test on your Shopify plan, which testing route fits, and what to ask tools about flicker and stats. Get a setup score.
  • What is testable on your plan and what is not
  • A route for each idea: browser-based, or server-side and theme tests
  • Flicker, speed and stats engine notes, and a list of questions to ask every tool
  • A setup score out of 100, with the sum shown
• Store test evidence reviewer: Check if your store test result can be trusted. Get a ship call, the lift as a range in money, and a trust score out of 100.
  • Pass or fail on the split, run length, early stopping and tracking checks
  • The lift as a range, and what it is worth per month in gross profit
  • One call: ship, do not ship, re-test or inconclusive
  • A trust score out of 100 with the sum shown, and a test log entry
• Brand profile builder: Share your website. Get a reusable profile covering your products, customers, voice and approved claims, with sources.
  • A folder of eight plain files, from a start file for your AI to a claims list and a change log
  • A tag on every fact, and every claim marked approved, pending or banned
  • A two-sentence diagnosis of your biggest gap, and a list of what's missing and who can answer it
  • A short slice for your next step, if you name one
• Peep Laja: Review page messaging or an A/B test using Peep Laja’s recorded approach. Start with buyer evidence, clarity and a sound comparison plan.
  • Your problem worked on the way Peep Laja would: their questions, their method, their push-back
  • The answer or the work, in Peep Laja's voice, using their ideas by name
  • Anything that goes beyond their public record, marked
  • One suggested next step

Tasks you can use it for

• You run tests on your store and want a log, a ranked list and one test at a time.
• A test just ended and you want to know if you can trust it before you ship the change.
• You are not sure if your store gets enough orders to test, or what to test first.

Limits to check before starting

• Your store gets under about 30 orders a month. Do research and known fixes first, or ask a person who knows your store to review it.
• You want to start, stop or publish a test for you. It never does this. Set it up in your own testing tool.
• You only need a one-off check of a single result. Read it with a simple sample size and range check, or ask a person who knows testing.

Best way to use this role

Give it one clear outcome, the actual files and a definition of done. Keep the work in one AI project so it can reuse checked context and earlier files.

• Start a chat with the agent. In Claude Code, add it as a skill or save it in .claude/agents, then ask it to run this fortnight's check.
• The first time, give it your numbers and your ideas. It sets your baseline and builds the backlog.
• Set up the test it plans in your testing tool. Tell it the start date.
• When the test ends, bring the numbers and anything that happened during the run. It reads the result and picks the next test.
• Run it again every two weeks. It reads its own log.

Read it as data

Read the complete role setup as JSON. Connect your AI to the AI Blocks MCP library and read aiblocks://roles/store-test-program-runner for its role instructions and pinned sources.

This is a prepared AI role. No task has run. Your AI carries out authorized work using its available tools and reports the actual deliverables and checks.