We found that the retrieval capabilities of long-context LLMs can be attributed to a small set of attention heads. Check out our new preprint!