Explainability has become a fundamental pillar for AI-based systems, especially in sectors where automated decision-making directly impacts safety, regulation, or user trust. However, integrating this attribute into the software development lifecycle is far from trivial. A recent study conducted at Daimler Truck —one of the world’s largest commercial vehicle manufacturers— has highlighted the real difficulties faced by requirements engineering (RE) teams when trying to capture, specify, and validate explainability requirements. Although the context involves autonomous driving and safety-critical systems, the lessons learned are applicable to any organization developing AI solutions that need to ensure their models are understandable and auditable.
The qualitative, multi-phase study involved eight Daimler Truck professionals working on defining requirements for explainable AI systems. Researchers used think-aloud protocols and moderated group discussions to observe how traditional RE techniques were applied during elicitation, specification, and validation. Preliminary results reveal recurring problems: conceptual ambiguity during elicitation (what exactly does 'explainable' mean for a truck operator?), lack of expressiveness and testability in specifications (how do you measure an adequate explanation?), and fragmented validation due to vague criteria and regulatory uncertainty. Essentially, standard RE practices —designed for functional and performance requirements— are not equipped to handle the subjective, contextual, and dynamic nature of explainability.
From a technical-business perspective, these barriers have direct consequences on final product quality. Poor explainability specification can result in AI models that, while accurate, are black boxes impossible to audit. This not only compromises user trust but also exposes the company to legal and compliance risks —such as GDPR in Europe or the upcoming AI Act. For a company like Daimler Truck, operating in an environment where a misinterpretation could have catastrophic consequences, explainability is not a luxury but an operational necessity.
What can other organizations learn from this case? The clearest lesson is that explainability must be treated as a first-class requirement from the very start of the project, not as a late addition. This implies rethinking elicitation techniques: instead of generically asking 'what does the user need to know?', it is better to perform cognitive task analyses, map decision processes, and define concrete scenarios where explanation is critical. Specification, in turn, should include quantifiable metrics such as explanation fidelity, completeness, or stability, just as with other software quality attributes. And validation requires the involvement of real stakeholders —operators, regulators, auditors— in usability and comprehension tests, not just automated checks.
For a software development company like Q2BSTUDIO, these lessons translate into an opportunity to design methodologies that natively integrate explainability into our solutions. For example, when developing custom software that incorporates AI modules, we work with clients to define transparency requirements from the outset: which decisions must be explained, to whom, with what level of detail, and in what format. We use AI explainability (XAI) tools that generate heatmaps, counterfactuals, or symbolic rules, and embed them into customized dashboards. Additionally, we support this process with cloud AWS/Azure platforms that enable scaling explanation storage and bias monitoring, along with cybersecurity layers that protect data and model integrity against adversarial attacks that could manipulate explanations.
Another relevant aspect is the intersection between explainability and BI / Power BI. Business intelligence dashboards often present aggregated results from AI models but lack context on how those conclusions were reached. Incorporating explanatory elements —such as feature importance charts or interactive decision trees— within Power BI allows analysts to trust the data and make informed decisions. In this regard, Q2BSTUDIO has developed solutions that connect explainable AI models with cloud data sources and generate dynamic reports that include both the outcome and the rationale. This is especially valuable in regulated sectors like banking or healthcare, where every prediction must be auditable.
Looking ahead, the Daimler Truck study also highlights the need for specific RE frameworks for AI systems. The academic community and industry are moving toward explainability requirement patterns, but practical adoption remains limited. From Q2BSTUDIO’s perspective, we believe the key lies in combining agile methodologies with user-centered design techniques, performing rapid iterations where stakeholders can experiment with explanation prototypes and refine requirements. All of this supported by cloud platforms that facilitate collaboration and versioning of RE artifacts.
Finally, we cannot overlook the role of AI agents. As autonomous systems —like Daimler’s vehicles— incorporate agents that make real-time decisions, explainability becomes even more complex. An AI agent negotiating traffic or adjusting speed needs to justify its actions not only to the human supervisor but also to other agents and systems. This requires new ways of specifying and validating explanatory communication between agents, an emerging field where RE practices must evolve. At Q2BSTUDIO we are already exploring explainable agent architectures based on language models and symbolic reasoning, integrated with cloud services to ensure secure deployment.
In conclusion, the Daimler Truck study offers a privileged window into the real challenges of requirements engineering for explainability. Far from being a theoretical problem, it is a practical barrier that can slow down the adoption of AI in critical environments. Companies that want to lead in this space must invest in methodologies, tools, and specialized talent. Q2BSTUDIO is committed to helping its clients overcome these challenges by offering custom software development, explainable artificial intelligence, cloud computing, and cybersecurity services, all with a pragmatic, business-value-focused approach. Explainability is not the destination; it is the path toward more trustworthy and responsible AI.
