@Marcel Reschke that’s an interesting approach, but is that necessarily true? That would imply that the context limit of the models dictates what’s actually clear or not. We’re engineering context to make them more suitable for the models to avoid mistakes. GSD addresses context rot by creating phases and established a workflow to start each phase in a new model. So, it’s not even about hitting the 65-70% context threshold. It’s about the fact that each phase should be instantiated in a fresh context. RLM would eliminate that need.

@Marcel Reschke that’s why I’m so keen on RLM. It literally solved the context rot problem on the hardest benchmarks. I don’t quite understand it yet, but essentially, it’s the same instance. No context rot. At least as far as I understand it.

Chase Ai talks RLMs. https://youtu.be/KvrJaGqHz14?si=0zkbEIflil4O6NWE