AI Blocks › Build and ship software
Local model runner for laptops
Name your model and computer. Get it running on a Mac, a plain CPU or a non-NVIDIA graphics card.
Local model runner for laptops is a free AI skill for the job "Build and ship software". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Orchestra Research · Level: advanced
Made by Orchestra Research. See the original on GitHub; we copied it as it is, with its licence.
What it does
An AI model running on a Mac, a plain CPU or a non-Nvidia graphics card, with no CUDA needed. Name the model and your computer and get the commands that start it. It uses llama.cpp, which runs compact GGUF models from 1.5 to 8 bits. It builds for Apple Metal, AMD ROCm or plain CPU, then downloads or converts the model. You can chat in the terminal or start a server that speaks the OpenAI style, so existing apps can connect. On a CPU it can run four to ten times faster than standard PyTorch. It explains the quantization formats so you pick one that fits your memory.
You give it
- The model name
- Your computer and its memory
- Operating system
- Whether you want chat or a server
You get
- Install and build commands
- A model download or conversion step
- A chat command
- A server command and test request
- A quantization choice that fits
Use it when
- You have an M-series Mac and want a local model.
- Your graphics card is AMD or Intel.
- You want a local server your apps can call.
- You are deploying to a machine with no GPU.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run code on your computer.