AI Blocks › Build and ship software
Longer context stretcher
Name your model. Get a method and code to make it read much longer documents than it was trained for.
Longer context stretcher is a free AI skill for the job "Build and ship software". Give the instructions to an AI assistant that supports the required files and tools, then review its output.
Free · No account · MIT licence · Made by Orchestra Research · Level: expert · Thinking: high
Made by Orchestra Research. See the original on GitHub; we copied it as it is, with its licence.
What it does
A method, with working code, for making a language model read documents far longer than it was trained on. This is for people training or running models, and it needs Python and a GPU. You name your model and the target length, such as 32k, 64k or 128k tokens. The skill covers four techniques for encoding word position: rotary embeddings, YaRN, ALiBi and position interpolation. It shows how to switch each on in the common Hugging Face tools, and when a short fine-tune is enough instead of full retraining. It points to the original papers, so you can check the maths. It also notes how to use FlashAttention to keep memory use down at long lengths.
You give it
- The model name
- The context length you need
- Your GPU and memory limits
- Whether you can fine-tune (optional)
You get
- A recommended technique and why
- Code to apply it
- Fine-tuning steps if needed
- Paper references to check against
Use it when
- Your model fails on documents longer than its trained limit.
- You want to extend a Llama or Mistral model to a longer window cheaply.
- You are choosing between YaRN, ALiBi and position interpolation.
- You need to run a model on long inputs without running out of memory.
How to use it
- Add it to your AI once: download it and add the folder to Claude, ChatGPT or Codex as a skill.
- Ask for the job in your own words. Your AI opens the skill when it fits.
- It works best when your AI can run Python code on a computer with a GPU.