This article details Direct Nash Optimization (DNO), a method designed to optimize LLMs using Nash equilibrium principles, addressing the challenges faced by traditional soft policy iteration. DNO replaces unstable and complex policy updates with a regression-based contrastive objective for stable batch training. The approach enjoys monotonic improvements and converges to the Nash equilibrium. A 7-billion-parameter LLM trained with DNO outperforms Mistral Large and earlier versions of GPT-4 on AlpacaEval 2.0. The paper highlights key design choices for developing iterative self-improving algorithms.
The Future of AI Model Training
Direct Nash Optimization (DNO): Method for optimizing LLMs using Nash equilibrium principles, replacing unstable updates with a regression-based approach for stable training. Converges to the Nash equilibrium and outperforms previous models on AlpacaEval 2.0.
A BREAK?
Play for a moment before you go
OUR SERVICES
How we can help you
Artificial intelligence
AI agents, chatbots, and intelligent assistants that automate tasks and serve your customers 24/7 to improve the efficiency of your business.
More info →Software Development
Software development for companies in Spain: web, mobile and desktop applications, intranets, e-commerce, SaaS and custom management platforms, with accessibility and good practices included.
More info →Cloud services
Migration, infrastructure, landing zones, networks, managed databases, scalable hosting and security in Microsoft Azure and Amazon Web Services so that your business grows according to the capacity you need.
More info →Cybersecurity and pentesting
Security audits, penetration testing and protection of applications, data and infrastructure on-premise and cloud. Ethical hacking and support in regulatory compliance (ENS, ISO 27001, GDPR) without promising certification.
More info →Business Intelligence
Operational reporting and predictive analytics with Power BI and Microsoft Fabric: we integrate your sources, design dashboards and KPIs and turn your data into informed decisions.
More info →Process automation
We automate repetitive tasks and connect your applications with n8n (open-source, self-hosted), Power Automate (Microsoft 365), Make or RPA, eliminating manual work. If you need AI agents, you'll find them in our AI hub.
More info →Training for companies
We train your teams in technology with criteria: web development, databases, Git, best practices and security, automation with n8n, artificial intelligence for companies and creation of AI solutions with Microsoft Foundry (formerly Azure AI Foundry).
More info →Code Auditing
We audit the code that you, your team or an AI create: we tell you what is good and what to improve, we secure it and make it ready for production, with evidence for ENS, ISO 27001, GDPR, NIS2 and DORA.
More info →Conversational avatars with AI
We create conversational avatars with AI – digital humans with a face and voice – for B2B cases: internal training, retail, events and customer service. They are not text chatbots or multimedia content; they are visual interlocutors with the knowledge of your company.
More info →Online Marketing and AI
Paid media, technical SEO, GEO/AEO, email marketing, landing/CRO and analytics: we capture useful demand and measure results without promising positions.
More info →AI Governance, Security, and Responsible Operation
We help inventory, control, and professionalize AI applications, automations, and agents without slowing down teams' ability to innovate.
More info →On-premise local AI: hardware, models, and operation
We implement on-premise local AI to reduce token costs and keep data at home: hardware sizing (NVIDIA GB10 / AMD), LM Studio stack → vLLM and integration with real processes.
More info →
Do you have a project in mind?
Tell us your vision and we'll turn it into a software solution. Whatever the scope, we make your idea real.