Proposal: Implement RLM

[kkontosis]

The following paper:

[2512.24601] Recursive Language Models
Recursive Language Models
https://arxiv.org/abs/2512.24601

Seems to achieve impressive improvement in context management and context extension and quality improvement for long context of an LLM.

It uses a clever way to interact with an LLM involving tool use.

Here is the abstract:
Alex L. Zhang, Tim Kraska, Omar Khattab
We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy that treats long prompts as part of an external environment and allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt. We find that RLMs successfully handle inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of base LLMs and common long-context scaffolds across four diverse long-context tasks, while having comparable (or cheaper) cost per query.

A youtube video exists discussing this paper and how it works: https://www.youtube.com/watch?v=huszaaJPjU8

And by searching online it seems that there are implementations in github.

The first thing that came to my mind is that this could be a great addition in OptiLLM, so after not finding it in the existing issues, I suggest it as a proposed implementation.