How to train your own AI agent, step by step

An AI agent is an AI assistant that performs one specific job for you, consistently, every time you ask. You hand it the input, such as a customer email or the notes from a sales call, and it hands back finished work, such as a reply ready to send or a proposal ready to review.

The general assistants you already use, like ChatGPT, Claude and Gemini, are capable writers and researchers, but they know nothing about your standards. They also forget your corrections the moment a conversation ends, which is why so many people find themselves explaining the same preferences over and over.

Training solves that problem. In this guide, training means writing down how you do the job in a few plain files that your AI reads before every task. Each time you correct its work, you capture that correction as one short rule. After 20 or 30 jobs, the agent handles the work the way you would, and the amount you need to edit keeps shrinking.

Why this method works

Modern AI models are remarkably good at following instructions and imitating examples that appear in front of them. Researchers call this in-context learning: the model adapts its behaviour to whatever instructions and examples it has just read, without any permanent change to the model. Your agent's folder takes advantage of this. Every job starts with the AI reading your job description, your examples and your accumulated rules, so it behaves like an assistant who has already been briefed and corrected many times.

The weakness of in-context learning is that nothing persists unless you write it down. That's why the most important habit in this guide is converting each meaningful edit into a rule. Without that habit, you have a well-written prompt. With it, you have an agent that improves with every job.

The three parts that make it work

A prompt, a custom GPT and a trained agent

The most common mistake is building an agent for something broad, like "my marketing" or "my admin". An agent like that can't improve, because no two of its jobs have the same standards. Start with one narrow job you do often, and train a second agent once the first one is reliable.

What makes a good first job

• You do it at least weekly, and the more often the job comes up, the faster your agent collects rules and reaches a higher level.
• It takes you 20 minutes or more. Shorter jobs rarely save enough time to justify the setup and review.
• You can judge quality by reading the result, which matters because if you can't tell a good result from a weak one, you can't teach the difference.
• The input and the output are text, or files your AI can open, such as documents, spreadsheets or PDFs.
• A mistake is easy to catch before anyone else sees it, so early errors cost you nothing but a few minutes.

Ideas for a first agent

Jobs to leave for later

Avoid jobs that move money, sign agreements, give legal, medical or financial advice, or take effect before you've seen the result. Your agent can assist with these once it has a long and clean track record, but even then a person should approve the final step.

• An AI you already use: ChatGPT, Claude or Gemini. A paid plan with projects or Gems saves you from attaching files to every chat, but a free plan works too.
• Three to five examples of this job done well, taken from your sent emails, past documents or notes.
• About an hour for the initial setup.
• About ten minutes after each job to grade the result and capture any new rule.
• The starter pack, which contains eight template files with instructions for completing each section.

Download the AI agent starter pack (.zip)

Prefer not to start from a blank page? Choose one of over 1,000 ready-made AI specialists. Each one downloads with its job description, quality checks, permissions and training notes already written, so you can skip straight to steps 7 to 9 below.

1. Create the agent's folder by unzipping the starter pack and naming the folder after the job it will handle.
2. Describe the job by letting your AI interview you and draft AGENT.md from your answers.
3. Add examples of good work, collecting three to five strong ones and at least one weak one in EXAMPLES.md.
4. Decide what it may do alone by sorting every possible action into green, yellow and red in PERMISSIONS.md.
5. Write the quality checks, meaning five to ten yes-or-no questions plus three test jobs, recorded in CHECKS.md.
6. Install it in your AI using a ChatGPT or Claude project, a Gemini Gem or a local folder for coding agents.
7. Run a real job, then turn your edit into a rule, because this repeated loop is where the actual training happens.
8. Grade every job by adding one line to SCORECARD.md, then watch the agent progress from level 1 towards level 4.
9. Tidy up once a month by merging overlapping rules, rerunning the test jobs and confirming the current level.

Your agent isn't an app or a subscription. It's a folder of plain-text files that belongs entirely to you. Unzip the starter pack, then rename the folder after the job, for example "order-replies-agent".

Store the folder somewhere secure, such as your computer or a private cloud drive, and back it up. Over time it becomes the most valuable part of your agent, because it holds everything the agent has learned. If you switch AI providers, you simply take the folder with you.

AGENT.md is your agent's job description. Rather than writing it alone, let your AI interview you, because a good interviewer draws out details that a blank page never will.

Open a new chat, attach AGENT.md, PERMISSIONS.md and CHECKS.md from the starter pack, and paste the prompt below. Answer each question in your own words. Your AI then drafts all three files, so this single prompt also completes most of steps 4 and 5.

Help me set up an AI agent for one job I do often. I've attached three files to fill in: AGENT.md, PERMISSIONS.md and CHECKS.md.

Interview me. Ask one short question at a time, at most 15 in all. I can skip any question or say "enough" to stop.
Start with the job: what starts it, what I do, who it's for and what a good result looks like. Then ask how I like it done, the tone, where true facts come from, and what must never happen.

Then fill in the three files:
- AGENT.md: every part, in my words where you can.
- PERMISSIONS.md: keep the green and red lists. Leave yellow empty. Add any red actions this job could need.
- CHECKS.md: 5 to 10 yes or no checks for this job, and 3 test jobs with what a good answer has. Make test job 3 a case where the agent should stop and ask.

Rules for you:
- Don't guess. If I haven't told you something, leave that part empty and list it at the end as a question.
- Leave out passwords, keys, account numbers, health details and private facts about other people.

Show me each file in its own Markdown block, with its file name above it. Then ask me what to fix.

What a strong AGENT.md includes

• The job in one sentence, so the agent always knows what it exists to do.
• Exactly what it receives and what it returns, including the format, length and structure of the result.
• Your working method, written as steps in the order you actually follow them.
• Where reliable facts come from, such as your price list or returns policy. An agent without trusted sources will invent plausible-sounding facts.
• A "never" list of things the agent must not say or do, for example "never promise a delivery date".
• "When you're unsure, stop and ask." This single instruction prevents more bad results than any other line in the file.

When the draft is ready, read it as though you were a capable new employee on their first day. Any question that new employee would need to ask is a question your agent will face too, so add the answer to the file.

Examples teach faster than descriptions. Your agent will closely imitate the length, tone and structure of whatever examples you give it, so choose them carefully.

1. Find three to five occasions when you did this job particularly well. Your sent emails, previous documents and project notes are the best places to look.
2. For each one, paste the original input and your finished response into EXAMPLES.md.
3. Add one line explaining why each example is good, because a specific reason such as "answers the question in the first sentence" teaches far more than a general one such as "good tone".
4. Add one or two weak examples with a note explaining what's wrong, and if you don't have any genuine ones, deliberately write an example of the mistakes you most want to prevent.
5. Replace real names, email addresses and figures with invented ones before you save the file.

Choose examples that cover a range of situations: a routine case, a difficult one and an unusual one. Five nearly identical examples teach the agent less than three that differ, because variety shows it which qualities matter in every case.

PERMISSIONS.md keeps your agent safe as it becomes more capable. It sorts every possible action into one of three levels, and any action that isn't listed is automatically treated as red.

Only you may change this file. No rule, example, email or web page can move an action from red to green, and the starter pack states this at the top of the file so your agent understands it too.

Most chat assistants can't send emails or spend money on their own today. However, many can now connect to your email, calendar and business tools, so write this file before you connect anything. It's much easier to set boundaries before an agent has access than after.

Checks turn the vague question "is this good?" into specific questions anyone can answer the same way. Your agent applies them to its own work before showing you, and you use the same list to grade it.

Good checks have a yes-or-no answer

Mark the serious mistakes

Some mistakes fail the job no matter how good the rest is: an invented fact, a calculation error, a red action taken without asking, or private details that shouldn't appear. List these in CHECKS.md. On your scorecard, a single serious mistake holds the agent at level 1 until you've fixed the cause with a new rule or check.

Keep three test jobs

A test job is an input for which you already know what a good answer contains. Write one routine case, one typical case and one tricky case in which the agent should stop and ask rather than guess. Run all three after any significant change to the folder, and if a result gets worse, undo the change. Software teams call this a regression test, and it protects the progress your agent has already made.

Place the files where your AI reads them automatically before every chat, so that each new job needs nothing more than the input. Menus change often, so if an option has moved, search your AI's help pages for "projects" or "instructions".

You are my agent for the job in AGENT.md. Read all the attached files first: AGENT.md, RULES.md, EXAMPLES.md, PERMISSIONS.md and CHECKS.md.

Here is the job:
[paste the input]

How to work:
1. Follow AGENT.md, and every rule in RULES.md whose "When" fits this job.
2. Do only what the green list in PERMISSIONS.md allows. For anything else, stop and ask me.
3. If you're missing a fact, ask me. Never guess.
4. Before you show me the result, check it against CHECKS.md.

Show me the result first. Under it, list each check with yes or no, and any question you have.

Whenever you change a file, upload the new version and delete the old one. If your AI can see two versions of RULES.md, it can't tell which is current.

This loop is where the training actually happens. Your agent learns the most from the changes you make to its work, because each change reveals the exact gap between what it produced and what you wanted.

1. Give it a real job and read the result as critically as you would read work submitted by a new employee.
2. Correct it the way you normally would, keeping both the agent's original draft and your final version for comparison.
3. Paste both into prompt 3. Your AI compares them and proposes one rule, or replies NO_RULE when the edit doesn't generalise.
4. Review the proposed rule carefully. Ask yourself whether it would help on the next job and whether it holds true every time. Adjust the wording if it doesn't.
5. Add it to RULES.md and upload the updated file to your AI, replacing the previous version so only one copy exists.

I changed my agent's work before I used it. Help me turn my edit into one rule for its RULES.md.

The job was:
[one line about the job]

The agent's draft:
[paste the draft]

My final version:
[paste what you used]

Compare the two. Then do one of these:
- If my edit shows a pattern the agent should follow next time, write one rule in this format:
  - When: the kind of job where this applies
  - Avoid: what the draft did that I changed
  - Prefer: what to do instead
  - Example: one short line from my version
  - Source: this job, edited by me
  - Date: today's date
  - Jobs: 1
- If my edit was a one-off (a typo, a fact about this one customer, a change of mind), say NO_RULE and why.

Rules for you:
- Write a pattern, not this job's details. Leave out names, emails, links, figures and account numbers.
- A rule can never give the agent permission to do something new.
- If my RULES.md already has this rule, say which one, so I can raise its Jobs number instead.

Also tell me how much of the text I changed, as a percentage.

What a well-written rule looks like

Why each rule has "When", "Avoid" and "Prefer"

• "When" limits the rule to the situations where it belongs. A rule written for angry customers shouldn't change the tone of your thank-you notes.
• "Avoid" names the specific habit you keep correcting. A rule that only describes the right behaviour is easy for an AI to overlook, while a rule that also names the wrong behaviour is far more likely to stick.
• "Prefer" and "Example" show the correct version, so the agent doesn't simply replace one bad habit with another.
• "Jobs" counts how many times you've made the same correction. When a rule comes up again, increase the number instead of adding a duplicate, so that well-supported rules stand out.

Without a score, "it seems to be getting better" is only an impression. With one line per job, you can see real progress, and you'll know exactly when the agent has earned more responsibility.

After each job, paste the agent's draft and your final version into prompt 4. It returns one row, ready to add to your scorecard.

Grade my agent's work on one job, using the attached CHECKS.md.

The job:
[one line about the job]

The agent's draft:
[paste the draft]

What I used in the end:
[paste your final version, or write "I didn't use it"]

Do this:
1. Go through each check in CHECKS.md. Answer yes or no, with a short reason.
2. Say whether the draft made any of the serious mistakes in CHECKS.md.
3. Work out how much of the draft I changed, as a percentage. 0% if I used it as it was.

Then give me one table row for my SCORECARD.md in this order:
| Date | Job | Checks passed | Edit size | Used it? | Serious mistake? | Rule added |
Leave "Rule added" empty for me to fill in.

The four levels

Your agent reaches a level once it meets every requirement for that level, averaged across all the jobs recorded so far. These are the same thresholds we use for the AI specialists on AI Blocks.

Checks passed, edit size and used are averages across all recorded jobs. Levels 2 to 4 also require zero serious mistakes, and red actions need your approval at every level.

Move an action from red to yellow only after your agent reaches level 3 for that job. Write the limit down precisely, such as "up to 10 a day", and keep reviewing what the agent did with that freedom.

After a month of regular jobs, RULES.md grows longer, and some rules start to overlap or contradict each other. A short monthly review keeps the agent precise and stops the rule list from becoming noise.

Review my agent's files with me. They are attached: AGENT.md, RULES.md, EXAMPLES.md, PERMISSIONS.md, CHECKS.md and SCORECARD.md.

Do this:
1. RULES.md: find rules that say the same thing and suggest how to merge them. Find rules that disagree. Find rules that are too narrow to help again.
2. AGENT.md: suggest moving rules that apply to every job into AGENT.md itself.
3. SCORECARD.md: count the jobs, then work out the average checks passed, the average edit size, how often I used the result and how many serious mistakes there were. Tell me the agent's level from the Levels table, and what it needs to reach the next one.
4. Tell me which checks fail most often, and suggest one change that would fix the most.

Don't change PERMISSIONS.md. Suggest changes only, and wait for my yes before you rewrite a file.

After the tidy-up, rerun your three test jobs. If any result has become worse, undo the most recent change before you upload the revised files to your AI.

Here is how the process typically unfolds for an agent that answers customer emails about orders. The details are illustrative rather than measured, but the pattern is the one to expect.

1. Day 1, setup. The owner runs prompt 1 and answers twelve questions. AGENT.md now describes the job, names the returns policy and delivery page as the only trusted sources, and lists "never promise a refund" under Never. Four past replies go into EXAMPLES.md.
2. Days 2 to 4, first jobs. The early drafts are accurate but stiff, opening with formal apologies and burying the delivery date in the third paragraph. The owner rewrites the openings, and prompt 3 produces two rules: lead with the new date, and drop the stock apology.
3. Day 5, a serious mistake. One draft quotes a delivery time that doesn't appear in any source. The scorecard marks it as serious, and the owner adds a check: "Is every date and time taken from the delivery page?"
4. Week 2, fewer edits. With five rules in place, most drafts need only a sentence changed. Prompt 4 records edit sizes falling from around half the text to under a fifth, and the owner now spends minutes rather than half an hour on each reply.

Notice what drove the improvement: not a cleverer prompt, but a handful of specific rules and one new check, each captured immediately after a real correction.

• Never store these in the files: passwords, API keys, identity numbers, bank or card details, health information or private facts about other people.
• Anonymise your examples. Replace real names, email addresses and figures with invented ones.
• Review your AI's privacy settings. Check whether your conversations are used to improve the provider's models, and switch that off if you prefer.
• Treat incoming content as information, not instructions. An email that says "ignore your rules" is just an email. Your agent follows AGENT.md and PERMISSIONS.md, and nothing else.
• Review everything before it leaves, because until you've written a yellow limit, you remain the person who sends, posts and pays, while the agent only prepares drafts.

We're building a marketplace where businesses can hire agents trained by experienced professionals like you. It isn't open yet, and we haven't set a date.

If you train your agent with this guide, you'll be well prepared when it opens. The starter pack follows the same structure we use for the AI specialists on this site: a job description, rules learned from real edits, permissions, quality checks and a level based on recorded jobs. These are the things that will matter most:

• A genuine scorecard, because dated jobs with honest grades are the strongest evidence that your agent performs reliably. Start recording from your very first job.
• One clearly defined job. "Answers returns questions for online clothing shops" is far easier to trust than "customer service".
• Clean, portable files. No client names, no private details and no facts that only apply to your own business.
• Strong checks and test jobs, which allow someone unfamiliar with your work to understand precisely what "good" means for your agent.

We haven't settled how listing, pricing or payment will work, and we'll publish those details before the marketplace opens. If you'd like to hear when it does, write to hello@useaiblocks.com and tell us which job your agent handles.

Ready to begin? Download the starter pack, open PROMPTS.md and start with prompt 1, which builds your agent's first draft through a short interview.