For years, the Product Requirements Document (PRD) served as the compass of every digital development. But in the era of intelligent agents, that compass is being rewritten. Xavi Amatriain, first chief AI and data officer at Expedia Group, put it clearly at VB Transform 2026: 'The new PRDs are the evals.' In other words, the traditional product specification shifts toward an exhaustive definition of evaluations — including red teaming, security tests, and behavior metrics — that are encoded before writing a single line of code. This paradigm shift doesn't just affect giants like Expedia; it redefines how any company should approach software development today, especially when integrating artificial intelligence components.
Amatriain's thesis is both provocative and practical. Instead of drafting static documents describing functionalities, he proposes that teams invest all their effort in designing the tests that will validate that the system behaves correctly. 'All your thinking is going to go into the evals,' he stated. This approach turns continuous evaluation into the engine of development, a concept that resonates strongly at a time when 66% of companies surveyed by VentureBeat already allow production deployments without human review or plan to do so within the next twelve months. Yet only 5% fully trust the automated evaluations that make that decision. The gap is clear: we need better evals, not just more of them.
For Amatriain, the problem is not solved by piling on barriers. 'The more artificial business rules and guardrails you put into the system, the worse off,' he warned. Guardrails, though necessary, introduce bias into the feedback loop, distorting what we learn from the user. The solution is not to eliminate them but to minimize their impact through intelligent design. Expedia has opted for three layers of governance: broad principles (culture and values), processes and tools that enforce them, and automation that executes them. This model materializes in agent release 'toll gates' — checkpoints calibrated to risk. If an agent is low-risk, it barely needs oversight; if it involves critical decisions (like recommending a flight), it requires evaluation rounds, red teaming, and security review.
The architecture also changes. Amatriain dismisses artificial general intelligence as a single monolith and bets on the composition of specialized agents. At Expedia, tools form skills, those skills assemble into sub-agents, and an orchestrator coordinates them. Each agent is scoped, making it easier to evaluate and lock down in isolation before integration. 'It's not about the model, it's about how you design the system,' he insisted. This systemic design allows, for example, a hotel recommendation agent to combine retrieval-augmented generation (RAG) with direct API calls, choosing between quick responses (average prices in Chicago) or longer processes (comparing real traveler reviews with what a hotel claims). And always, the final decision rests with the user: 'The agent can recommend, suggest, discuss with you, but you have to click. That's non-negotiable,' Amatriain stated, turning that norm into a native security decision.
Security, precisely, must be integrated from the start. 'When you need a guardrail, it's because you didn't think about it earlier,' he said. In his vision, design principles must include security as a requirement as basic as functionality. And the cycle doesn't end at launch: monitoring signals from production must feed back into the eval suite, creating an almost automatic loop of detection and correction. The urgency is real: 54% of companies have already suffered an agent security incident or near-miss, according to VentureBeat. Moreover, attackers will no longer be just humans. 'You're going to get threats from other external agentic systems, very powerful, that will poke at everything you do,' Amatriain warned.
This landscape demands a profound transformation in how software is built. Companies that want to adopt this model — evals as PRD, risk-based governance, specialized agents, and security by design — need technology partners who understand both theory and practice. This is where custom software becomes a key enabler. Q2BSTUDIO, with its experience in personalized software development, helps organizations integrate these lessons into their own systems, whether by creating microservices, implementing evaluation pipelines, or designing monitoring dashboards with BI. AI is not a product you buy; it's an ecosystem you build, and every piece — from the cloud on AWS or Azure to cybersecurity — must be orchestrated with precision.
Q2BSTUDIO offers services that go beyond code: consulting on agent architectures, implementing continuous evaluation systems, and securing cloud infrastructures. In a world where evals are the new PRD, having a robust testing and monitoring platform is as critical as having a good model. Process automation, data analysis with Power BI, and cloud governance integrate into a holistic approach that allows companies not only to react to failures but to anticipate them. Expedia's lesson is clear: don't wait for an agent to fail with a real customer to adjust your evaluations. Design, evaluate, monitor, and repeat. And do it with partners who master every layer of the technology stack.
