AI Blocks › Build and ship software
Phoenix AI app tracer
Give it your AI app. Get traces of every step, plus tests that score answers and spot failures.
Phoenix AI app tracer is a free AI skill for the job "Build and ship software". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Orchestra Research · Level: advanced
Made by Orchestra Research. See the original on GitHub; we copied it as it is, with its licence.
What it does
See every step your AI app takes, score its answers, and catch failures before users do. Phoenix is an open-source tool you run yourself, with no vendor lock-in. The skill covers setup and use. You install the packages, switch on tracing so each call and tool step is recorded, and open the interface to inspect them. You build versioned test sets, then run judge models that grade answers, and compare prompts or models through experiments. It also supports live monitoring in production. Data sits in SQLite or PostgreSQL. It needs Python and a running Phoenix server.
You give it
- Your AI app code
- A test set of questions (optional)
- Which framework you use
- What counts as a good answer
You get
- Traces of every call and step
- Judge scores on answers
- Versioned test sets
- Experiment results comparing prompts or models
Use it when
- Your chatbot gives odd answers and you need to see each step.
- You want to compare two prompts on the same test set.
- You need a regression check before shipping a change.
- You want to watch an AI app in production.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run Python code and open a browser.