Taxonomy of authority sources in test oracles with LLM

The growing integration of large language models (LLMs) in the generation of test oracles raises a crucial question: where does a verdict obtain its authority? While previous literature classifies these oracles by their form or the technique employed, a more robust approach consists of examining the source of authority that legitimizes the decision. A recent systematic review (following PRISMA 2020 guidelines) analyzed 54 studies and revealed that nearly half of the oracles derive their authority from a formal specification, while the rest operate without it, relying on implicit knowledge or expected behaviors. This taxonomy not only helps to understand to what extent we can trust an AI-generated oracle, but also guides the design of more robust strategies. For example, a specification-based oracle can be verified through logical mechanisms, while one without specification requires adjudication by contrast with other models or historical data. The label 'LLM as judge' describes a mechanism, not a basis of trust, and its reliability depends on the application context. In business environments, where custom applications are developed and critical processes are managed, this distinction is vital. Q2BSTUDIO, as a software development and technology company, integrates artificial intelligence into its validation flows, combining AI for businesses with proven methodologies. Adopting AI agents that act as oracles requires understanding the sources of authority: do they rely on documentation, training data, or autonomous reasoning? Furthermore, the cybersecurity of these systems is a priority, as a poorly informed oracle can generate false positives or negatives. Our cloud services aws and azure provide the necessary infrastructure to run tests at scale, while the capabilities of power bi and business intelligence services allow monitoring the quality of decisions. Ultimately, the taxonomy of authority sources is not an academic exercise; it is a practical guide for building custom software that is verifiable, reliable, and aligned with business objectives.