AI Blocks › Build and ship software
Prompt attack detector
Point it at your AI app's inputs. Get a small filter that flags tricks and hidden instructions before they land.
Prompt attack detector is a free AI skill for the job "Build and ship software". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Orchestra Research · Level: advanced · Thinking: low
Made by Orchestra Research. See the original on GitHub; we copied it as it is, with its licence.
What it does
A small filter that checks text before it reaches your AI app and flags tricks like "ignore previous instructions" or commands hidden inside documents. Bad input gets caught before it does damage. It uses Meta's Prompt Guard, a compact classifier that labels text as normal, injection or jailbreak. You set a threshold, and anything over it is blocked or sent for review. Use it on user messages, and on third-party content such as web pages or search results that feed a retrieval system. It handles several languages and runs fast, including in batches. You need Python with transformers and torch, and model access on Hugging Face.
You give it
- The inputs or documents to screen
- Score threshold you want to use (optional)
- Python setup with the model downloaded
- Whether the text is user input or third-party data
You get
- Working filter code
- Label and score for each text
- Block or allow decisions
- Batch screening script
Use it when
- Your chatbot takes free text from the public and you worry about jailbreaks.
- Your app reads web pages or emails and someone could hide instructions in them.
- You want a quick check on documents before they enter your search index.
- You need a log of suspicious inputs for review.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run Python code.