SLIM-RL: RL with random masking and risk budget for dLLMs

In the current landscape of generative models, diffusion large language models (dLLMs) represent a significant step forward by offering non-autoregressive text generation, with advantages in parallelism and control. However, adapting these models through reinforcement learning (RL) has posed a technical challenge: how to align training trajectories with actual inference without incurring prohibitive computational costs. Methods like TraceRL reconstruct the trajectory of each rollout, multiplying the number of required samples and limiting scalability. Faced with this scenario, SLIM-RL emerges, a proposal that eliminates the need to reconstruct the trajectory using a decoder with a tau risk budget. In essence, SLIM-RL bounds the commit risk at each step of the rollout, generating training data with fine-grained uncertainty control. During optimization, it uses a trace-free random masking objective, combined with variance reduction techniques such as sequence-level importance sampling and deterministic quadrature over masking levels, under a monotonically decreasing mask schedule that preserves the mean. The results are telling: with only 46% of the training samples used by TraceRL, SLIM-RL matches its accuracy on MATH500 with block size 16, and surpasses it by 6.32% on the same metric and by 11.05% on GSM8K with comparable dynamic sampling. Even with block size 4, a 4B model outperforms much larger competitors like LLaDA-8B and Dream-7B in mathematics, trailing only the autoregressive Qwen2.5-7B. In code tasks, it improves upon TraceRL by 4.20% on MBPP and 3.65% on HumanEval. Furthermore, the tau decoder transfers without additional training to other dLLMs, underscoring its generality. Behind these advances lies a practical vision: efficiency in data and compute resource usage is key to democratizing artificial intelligence in enterprise environments. In this regard, companies like Q2BSTUDIO are exploring how to integrate these innovations into artificial intelligence solutions for businesses that require both precision and scalability. The SLIM-RL approach aligns perfectly with the need to develop custom applications that leverage generative models without sacrificing performance. For example, in designing AI agents capable of reasoning over technical documentation or generating code safely, risk management during inference becomes critical. Additionally, this method's ability to work with risk budgets opens the door to integrations with cloud services like AWS and Azure, where inference cost must be strictly controlled. The reduction in training samples also facilitates the adoption of custom software in regulated sectors, where each model iteration must be audited. Even in the field of cybersecurity, controlled text generation via dLLMs can be used to simulate attacks or analyze logs, provided that the risk of bias or error is bounded. Finally, business intelligence benefits from more efficient models: an RL-aligned dLLM can summarize financial reports or feed Power BI dashboards with contextualized and reliable information. In short, SLIM-RL not only provides a significant technical improvement but also lays the groundwork for business intelligence services and other cognitive capabilities to be deployed in production with an optimal balance of precision, cost, and risk control.