AI Blocks › Build websites and web apps
AI crawler access editor
Say which AI crawlers to allow. Get your robots.txt updated so AI assistants can read your catalog. Changes your live theme.
AI crawler access editor is a free AI skill for the job "Build websites and web apps". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by 40rty AI · Thinking: low
Made by 40rty AI. See the original on GitHub; we copied it as it is, with its licence.
What it does
Your Shopify store's robots.txt updated so AI assistants such as ChatGPT, Claude and Perplexity can read your catalog. If the file blocks them or says nothing, people shopping through AI may never see you. The skill reads your theme's robots template and adds a clearly marked block of allow rules for the main AI crawlers. Running it twice adds no duplicates. You can change the crawler list, or use audit mode, which only reports. It needs the Shopify CLI signed in, with permission to read and write themes. It edits your live theme, so it defaults to a preview. Copy the theme first.
You give it
- Your store address
- A signed-in Shopify CLI
- Theme write permission
- Which AI crawlers to allow (optional)
You get
- A preview of the new robots file
- An audit of current crawler rules
- A marked, safe-to-rerun block of rules
- The live theme updated once you approve
Use it when
- You sell on Shopify and want AI shopping tools to see your products.
- You do not know whether your robots.txt blocks AI crawlers.
- A site report flagged a missing AI crawler rule.
- You want a preview of the change before it goes live.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can access your Shopify admin and change it.