Do LLMs know what they know? Measuring Metacognition with TDS

In recent years, large-scale language models (LLMs) have demonstrated an amazing ability to generate coherent responses, translate languages, and even hold near-human conversations. However, a critical question remains: do they really know what they know? That is, can they accurately assess the certainty of their own answers? This problem, known in psychology as metacognition, has become central to understanding the reliability of artificial intelligence. Traditionally, the assessment of confidence in LLMs relies on calibration metrics such as expected calibration error (ECE) or Brier score, but these measures mix two distinct capabilities: how much the model knows (Type 1 accuracy) and how well its confidence signal reflects that knowledge (Type 2 metacognitive sensitivity). To decompose these factors, researchers have turned to Signal Detection Theory (TDS), a classic methodology in psychophysics that allows analyzing the ability to discriminate between states – in this case, between correct and incorrect answers – using a continuous confidence variable, such as the normalized logarithmic probability at the token level.

By applying SDT, a Type 2 ROC curve is constructed that shows how model confidence relates to actual correction, revealing an uneven variance structure that simple calibration metrics fail to detect. To measure metacognitive efficiency in open-ended question tasks (where there is no Type 1 decision with two alternatives), a model-free measure called normalized metacognitive information (meta-I₂r) has been proposed. This approach has been tested in four popular LLMs (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3-8B-Base, and Gemma-2-9B-Instruct) with 224,000 factual question trials. The results show that metacognitive information varies more than twice between models and is inversely correlated with accuracy: the least accurate model is the one with the most informative confidence signal. However, with only four models, a confusion of difficulty of error cannot be ruled out, which is why we speak of coupling, not decoupling. In addition, the uneven variance structure is model-specific (z-ROC slopes between 0.81 and 1.18), metacognitive sensitivity is domain-dependent (strongest in Art and Literature for all), and temperature dissociates Type 1 accuracy from metacognitive information, which remains stable as precision changes.

These findings have profound implications for the development of AI-based applications. If an LLM expresses high confidence in incorrect answers, the systems that rely on it—from virtual assistants to data analytics platforms—can generate misleading results. Therefore, measuring and improving metacognition is a key step towards a more reliable and transparent artificial intelligence. In this context, companies such as Q2BSTUDIO offer AI services for companies that integrate confidence assessments and calibration into their models, ensuring that AI solutions are not only accurate, but also know how to communicate their limitations. In addition, custom application development allows these advanced metrics to be incorporated into real workflows, personalizing the interaction between the user and the AI.

From a technical perspective, the decomposition of trust using TDS offers a window into the internal architecture of LLMs. The variability in z-ROC slopes suggests that each model learns to weigh evidence differently, which could be related to the training process, architecture, or data. For engineering teams, this opens the door to optimizing not only accuracy, but trust signal quality. For example, by adjusting temperature or designing sampling strategies that preserve metacognitive information while improving accuracy. This is particularly relevant in sectors such as cybersecurity, where a false positive or negative can have critical consequences. Q2BSTUDIO's cybersecurity services, combined with AI agents trained to assess their own certainty, could revolutionize threat detection, delivering alerts with calibrated confidence levels.

From a business point of view, the metacognition of LLMs directly impacts automated decision-making. A business intelligence platform that uses a language model to generate reports must be able to indicate when its claims are uncertain. The business intelligence services offered by Q2BSTUDIO, integrated with tools such as Power BI, can benefit from incorporating these confidence metrics to enrich dashboards with reliability indicators. Similarly, the implementation of AWS and Azure cloud services allows these solutions to be scaled, while maintaining low latency and data privacy. The company also develops bespoke software that includes trust assessment modules, facilitating the adoption of metacognitively aware LLMs in corporate settings.

The aforementioned study confirms that temperature is a hyperparameter that dissociates accuracy from metacognitive information: while accuracy may vary, the quality of the trust signal remains stable. This suggests that developers can adjust the temperature to control the balance between creativity and truthfulness without losing the model's ability to self-regulate its uncertainty. In applications where high accuracy is required, such as the generation of legal contracts or assisted medical diagnosis, this finding is crucial. On the other hand, domain dependency indicates that not all subjects benefit equally from metacognitive enhancements; The models seem more aware of their knowledge in areas such as art and literature than in exact sciences, which could reflect biases in the training data.

In conclusion, the measurement of metacognition in LLMs using SDT represents a significant methodological advance. It overcomes the limitations of traditional calibration metrics and provides a finer view of how models manage their uncertainty. For companies looking to implement AI responsibly, understanding and optimizing this capability is just as important as improving accuracy. Q2BSTUDIO, with its expertise in custom software development, AI agents, and cloud solutions, is well-positioned to help organizations integrate these advanced assessments into their systems. Metacognition isn't just an academic topic: it's the key to building an AI that not only speaks fluently, but also knows when to shut up or admit its doubt.