AI Blocks › Build and ship software
Multi-GPU training setup helper
Give it a training script. Get it running on one GPU, many GPUs or several machines with a few added lines.
Multi-GPU training setup helper is a free AI skill for the job "Build and ship software". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Orchestra Research · Thinking: low
Made by Orchestra Research. See the original on GitHub; we copied it as it is, with its licence.
What it does
Your training script running on one GPU, several GPUs or several machines, with only a handful of added lines. The same script works in all three setups. The skill uses Hugging Face Accelerate. You create one Accelerator object, pass your model, optimizer and data loader through its prepare call, and swap the usual backward call for the Accelerate one. You can delete the lines that move data to the GPU by hand. Then one launch command runs it. The skill also covers answering a few setup questions to make a config file, and the options for splitting memory across GPUs for big models.
You give it
- Your PyTorch training script
- How many GPUs and machines you have
- Whether the model fits on one GPU
- Precision preference (optional)
You get
- The script with the added lines marked
- A launch command
- A config file for your setup
- Memory-saving options for large models
Use it when
- Your script runs on one GPU and you just got four.
- You want to try mixed precision without rewriting the loop.
- Training must run across two machines.
- Your model is too big for one GPU's memory.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run Python code.