Sangam: Efficient diffusion LLM serving with the AR stack

Diffusion language models (dLLMs) represent a significant advancement in text generation, but their bidirectional attention architecture and the need for iterative denoising pose unique challenges for inference systems. Unlike traditional autoregressive models, dLLMs do not allow traditional key-value caching (KV caching), forcing the use of cache refresh techniques such as those proposed in Fast-dLLM or dKV-Cache. In this context, Sangam emerges as a serving system designed specifically for dLLMs, introducing a token budget deficit scheduler that prioritizes ongoing decodes and manages prefills indivisibly, achieving amortized stall-free scheduling. Additionally, its hybrid serving strategy allows offloading prefills to decode workers to avoid under-provisioning, optimizing latency according to workload. The practical implementation of these systems requires a robust and customized infrastructure, where companies like Q2BSTUDIO offer artificial intelligence solutions for businesses that range from custom application development to AI agent integration. Likewise, the efficient deployment of these models relies on AWS and Azure cloud services, ensuring scalability and performance. Cybersecurity, business intelligence with Power BI, and process automation complete a technological ecosystem that allows organizations to fully leverage the capabilities of dLLMs without compromising operational efficiency.