Hire an AI LLM post-training planner

Everything it can hand back

Your task sets the actual outputs. The specialist picks the methods that cover them.

• Which training method fits, and why a simpler one isn't enough
• A gate record with a fixed comparison, one change under test and a promote or stop rule
• For a failed run, an incident report with a status, the likely cause and the smallest next test
• For a saved checkpoint, a release record with a file list, hashes and a load test
• A list of evidence to keep before you clean up or retry

How it does the job well

• Do not scale a run whose smoke or signal gate has not produced its evidence.
• Do not diagnose from one number, such as loss, reward, throughput or an exit code. A running task or a checkpoint folder is not a pass.
• After an unexplained failure, change one thing at a time.
• Do not register, resume or publish an incomplete checkpoint. It needs the expected files, a hash list and a clean load test.
• A correlation, routing count or reward rise does not prove quality or cause. Quality claims cite at least one held-out measure.
• Do not scale a run whose smoke or signal gate has not produced the promised evidence.
• Do not diagnose from one scalar such as loss, reward, throughput, or an exit code.

Mistakes it avoids

• Biased wording in job ads, reviews or interview questions.
• Plans with no owner, timing or way to check progress.
• Treating a template as legal advice.

What a good result looks like

An example, not your task: You want to fine-tune an open model on your own examples and don't know where to start.

• Experiment gate record. The target, fixed comparison, one change under test, required measures, and a promote or stop decision.
• Incident report. A status, observed evidence, failure class, smallest next test, stop condition and what to keep.
• Checkpoint release record. Expected files, save evidence, a hash list, a load test and a register or resume decision.
• Assumptions and gaps. What was assumed, what is MISSING and which checks were not run.

How the work is checked

• Criteria are written down before people are assessed.
• Every plan has milestones and owners.
• Sensitive steps are marked for human and legal review.
• States which checks passed, failed or could not run, and names missing inputs.

These checks have not been run against a model yet. Your AI uses them to check its own work.

A second opinion

Before it hands back, your AI reviews the draft the way Margaret Hamilton would. This is a way of reviewing the work. It does not mean they wrote or endorse it.

Capabilities

• Freeze the decision. It writes down the target, baseline, model, data version, test set and budget, so runs can be compared fairly.
• Name the failure before any retry. It sorts the problem into data, objective, reward, rollout, runtime, integrity or quality, and names a competing cause if there is one.
• Pick the weakest method that works. It starts with SFT on trusted targets. It moves to DPO only when pairs are intact, and to RL only when the reward is checked.
• Run the smallest gate. It moves through preflight, smoke, signal and controlled gates. Each gate has an evidence file and a stop rule.
• Check checkpoints. It lists the expected files, then asks for a hash list and a clean load test before you register or resume.
• Preserve and hand off. It names what to keep, such as hashes, configs and sample rows, then states what the test proves and what it doesn't.

How the role works

1. Checks your task, context and available tools.
2. Picks the methods needed and prepares a work plan.
3. Carries out supported tasks, with each method's limits and review gates.
4. Checks the real deliverables and corrects supported faults.
5. Hands back files, evidence and unresolved gaps.

Your specialist's working setup

Download the base specialist without a task form or account. README.md and SETUP.md explain how to use the folder. SOUL.md defines its scope. INTAKE.md asks missing questions inside your AI. PLAYBOOK.md, SKILLS.md, BEST_PRACTICES.md, CHECKS.md and HANDOFF.md guide the work and its checked deliverables.

Complete associated skills, supporting files and licences are included. Choose only the methods needed for the actual task. Keep context, working files and accepted feedback in your AI project.

Use this specialist through MCP or the API

Call prepare_specialist with role_id engineering-llm-post-training-engineer only. No task is required to load it. Your AI asks what job you need done, what you want back and which relevant context it can use.

For task-led hiring, call find_specialist, explain the suggested specialists and let the user accept or change the team.

Complete ZIP: Download base specialist. API: GET /api/v1/specialists/engineering-llm-post-training-engineer/pack. The host AI supplies its own model, tools and execution; a pack does not connect accounts or run a task.

Inputs and helpful context

The specialist starts from the task brief. Only active methods require their inputs.

For the primary method, when used

• The behaviour you want to change in the model, and what must stay the same
• The base model name and how you train now, such as SFT, DPO or GRPO (optional)
• Five to ten sample rows of your training data, with private details removed
• Your test questions and how you score answers
• Your GPU, storage and time budget
• For a failed run, the config, log lines and the numbers you saw (optional)

Helpful context (optional)

• Your last good run's config, to use as the baseline.
• A fixed set of held-out test questions.
• The tokenizer and chat template files.

Tools and access

• Claude, ChatGPT or Gemini with file upload: They can read configs and logs together and keep the evidence straight.

Functions and deliverables

• LLM post-training planner: Plan an SFT, DPO or RL fine-tune of a language model, or find why a run went wrong. Get fair tests, a stop rule and a checkpoint check.
  • Which training method fits, and why a simpler one isn't enough
  • A gate record with a fixed comparison, one change under test and a promote or stop rule
  • For a failed run, an incident report with a status, the likely cause and the smallest next test
  • For a saved checkpoint, a release record with a file list, hashes and a load test
  • A list of evidence to keep before you clean up or retry
  Source: Michael Sitarzewski and Agency Agents contributors.
• Language model trainer: Set up a language model fine-tuning run with TRL. Give it your base model, data and goal. You get a ready trl command and the reason for each setting.
  • The right training method for your goal
  • A ready trl command
  • Settings with a short reason for each
  • Next steps to check results
  Source: sickn33.
• Model training builder: Describe your data and goal. Get model training code that loads, checks and splits your data, trains, and saves checkpoints as it runs.
  • Data loading and splitting code
  • A reusable preprocessing step
  • A training loop with validation
  • Saved checkpoints
  • Training and test scores
  Source: Burhan Sebin.
• Language model context extension planner: Share your model, target text length and GPU limits. Get a method, Python examples and training steps.
  • The method that fits your model and why
  • The config values to switch it on
  • Code for loading the model with the new setting
  • A fine-tune plan, if your method needs one
  • Links to the papers behind each method
  Source: Orchestra Research.
• me.md builder: Answer a few questions, or let your AI use what it remembers. Get one private file that tells any AI who you are.
  • One me.md file with up to ten dated sections
  • A list of facts taken from memory, for you to check
  • A short note of what it removed for privacy
  • The two next steps, to save it and set it up
• Margaret Hamilton: Review a system failure plan using Margaret Hamilton’s recorded approach. Check handoffs, priorities, warnings and what happens when people make mistakes.
  • Your problem worked on the way Margaret Hamilton would: their questions, their method, their push-back
  • The answer or the work, in Margaret Hamilton's voice, using their ideas by name
  • Anything that goes beyond their public record, marked
  • One suggested next step

Tasks you can use it for

• You want to fine-tune an open model on your own examples and don't know where to start.
• Training loss goes down but the model isn't better on your test questions.
• A preference run (DPO) or reward run (RL) finishes, but answers aren't better or just get longer.
• A checkpoint saved and you need to know it is complete before you resume or publish.

Limits to check before starting

• You use a hosted model and only change prompts. Use the Prompt engineer.
• You want to find silent bugs in a small model's training code. Use the AI model training debugger.
• You want a working AI feature in an app, with retrieval and deployment. Use the AI feature planner.

Best way to use this role

Give it one clear outcome, the actual files and a definition of done. Keep the work in one AI project so it can reuse checked context and earlier files.

• Download the pack and add it to your AI as a skill. Or open a new chat and paste SKILL.md, then references/source-agent.md.
• Say what you want the model to do better, and paste your data samples, test questions and budget.
• For a failed run, paste the config and log lines and say what you saw. Remove keys, tokens and private text first.
• Read the Status line first. UNVERIFIED means the evidence isn't there yet, not that the run is fine.
• Run only the "Next Minimal Test". Change one thing, then paste the new numbers back.
• Don't scale up, register or publish a checkpoint until the gate says promote.

Read it as data

Read the complete role setup as JSON. Connect your AI to the AI Blocks MCP library and read aiblocks://roles/engineering-llm-post-training-engineer for its role instructions and pinned sources.

This is a prepared AI role. No task has run. Your AI carries out authorized work using its available tools and reports the actual deliverables and checks.