Less latent, better relay: compression for multi-agent collaboration in LLMs

In the current artificial intelligence ecosystem, multi-agent systems based on large language models (LLMs) have begun to explore forms of communication beyond discrete tokens. Traditionally, agents exchange textual messages, but this approach loses contextual nuances that can be critical for complex tasks. Recently, techniques such as full KV (key-value) cache relay have allowed preserving richer latent representations, but at the cost of high memory and bandwidth consumption. This raises a key question: is it necessary to retain all information to achieve effective communication? Research shows that, in reality, less can be more. Through adapted KV cache eviction methods, such as the novel Orthogonal BackFill (OBF), it is possible to inject a low-rank orthogonal residual signal from discarded states, maintaining only between 10% and 20% of the original states. Surprisingly, this compression not only reduces costs but achieves between 97% and 120% of the accuracy of full relay in mathematical reasoning, expert knowledge, and coding tasks. The lesson is clear: preserving the most useful information, rather than the largest amount, optimizes collaboration between agents. This finding has direct implications for the development of AI for businesses seeking to implement efficient and scalable AI agents. In this context, having a technology partner like Q2BSTUDIO allows integrating these innovations into practical solutions. For example, when designing custom applications for corporate environments, latent representation compression can be applied to reduce cloud infrastructure costs without sacrificing accuracy. The company also offers AWS and Azure cloud services that facilitate the deployment of these multi-agent systems in hybrid environments, ensuring scalability and security. Additionally, the ability to analyze and visualize the results of these agents through Power BI or through custom business intelligence services allows organizations to make data-driven decisions with greater agility. For those companies concerned about the integrity of their systems, Q2BSTUDIO also provides specialized cybersecurity to protect communications between agents and sensitive data. In summary, the evolution towards compressed latent communication in multi-agent systems is not only a scientific frontier but a real opportunity for companies to adopt artificial intelligence more efficiently and sustainably. If you wish to explore how to apply these techniques in your organization, you can learn more about AI for businesses and custom applications that we offer to transform collaboration between intelligent agents.