Instruction leakage in world models: goal-free solution
Language-conditioned world models have become a central promise of modern AI. The idea is elegant: an agent receiving a textual goal should build a compact representation of its environment and use it to plan actions. Instead of enumerating every possible state, the model learns to rely on explicit references, such as the relative position of two objects. This ability would dramatically reduce the data required and allow generalization to new situations. However, recent research has uncovered a trap. A model can achieve outstanding accuracy in relation reading without perceiving the world. The model is transcribing the instruction, not interpreting the scene. This phenomenon is known as instruction leakage and has deep consequences for the evaluation and design of language-vision systems.
Instruction leakage occurs when the variable we are measuring is contained in the instruction itself. If we ask a model to predict the position of an object and tell it, for example, to leave the silver object in front of the orange cone, the model can produce an answer that matches the goal without looking at the image. High accuracy does not reflect understanding; it reflects a spurious correlation. This is a validation problem very similar to the one found in other machine learning domains: if a training feature acts as a shortcut, the model will use it even when it is not the one that really matters.
The experiment designed to demonstrate this problem is striking. A relation predictor trained to read anchors achieves 0.90 accuracy. On the surface, the system is correctly interpreting the scene. However, when the linguistic goal is removed, accuracy drops to 0.27, very close to chance. This indicates that the internal representation does not contain the visual information needed for the task; it depends entirely on the instruction. The definitive test comes with a counterfactual instruction: the model is shown a scene and a false order. In 94.5% of cases, the predicted anchors follow the false order, while only 2.3% correspond to the real scene. In other words, the model does not look at the world; it simply copies what the text says.
This behavior is not an isolated case. Running the same protocol on a widely used external benchmark such as BabyAI reveals the same leakage. In contrast, a forward-dynamics model based on Language-Table behaves honestly while the instruction names only the referents. When the instruction is modified to also include direction, leakage appears. This nuance reinforces the idea that the decisive factor is not model complexity or competition among predictors, but the presence of the answer in the instruction. Another relevant piece of evidence is that degrading the action does not increase leakage. If the model had learned to respond using the action as an alternative information source, it would fall back on the instruction when the action degrades. That does not happen, which rules out the competition-based explanation.
The proposed solution is clear and actionable. First, the goal should not be part of the world dynamics. Dynamics predict how the environment evolves from actions and states; the goal belongs to the planner, which uses it to evaluate costs and benefits. Keeping the goal out of the dynamics prevents the world model from using it as a shortcut. Second, it is necessary to supervise the read path of the learned representation. That is, the model must demonstrate that it can extract the relation from the scene, not only from the instruction. With this separation, the same model recovers 0.88 accuracy, identical with and without the goal. The leakage disappears.
This finding has implications far beyond the laboratory. Any system that combines a textual goal with perception can suffer from instruction leakage. A remote assistance system, a warehouse robot, or a customer service agent can show excellent metrics in offline tests and fail badly in production because they have learned to play with words, not to understand context. Therefore, the evaluation strategy must include counterfactuals, ablations, and goal-free tests. Reporting accuracy is not enough; we must know exactly what information the model is using.
At Q2BSTUDIO, we constantly apply this lesson. When we build custom software, we design the architecture to separate the linguistic input from the perception module and the decision module. We do not want an agent to learn to win in the test environment by memorizing the label. We want it to learn to solve the real problem. Our client experience shows that this approach reduces time-to-production and simplifies evolutionary maintenance: if the model fails, we know whether the failure lies in perception, planning, or the user interface.
Instruction leakage should also be part of the security audit of any Artificial Intelligence system. In cybersecurity, we evaluate whether an adversary can manipulate the output by injecting malicious text. The parallel is evident: if an instruction can completely override scene perception, an attacker can trigger dangerous behaviors without touching visual data. Detecting this requires robustness testing, red teaming, and careful design of the language-environment interface.
Similarly, in a Business Intelligence environment with Power BI, a metric can be contaminated by an irrelevant column that acts as a leak. A report can look insightful when it is actually repeating a classification already embedded in the data. The technical solution is the same: separate the data layer from the presentation layer, audit variables, and test the result in scenarios where the potential leak source is removed. In cloud AWS/Azure projects, this kind of data hygiene is especially important because distributed pipelines multiply the risk of a decoy variable traveling from one service to another undetected.
AI agents are a particularly vulnerable case. An agent receives natural-language goals continuously and operates in partially observable environments. If the world model is conditioned on the goal, the agent can solve the task without building a model of the environment. Recent research suggests treating the goal as an input to the planner, not as part of the dynamics. This design choice not only eliminates leakage, but also makes the system more modular and auditable. The cost function can be changed without retraining the transition network, which eases adaptation to new business processes.
In summary, instruction leakage is a valuable warning for the AI community and for companies adopting these technologies. A good metric result does not guarantee that the model has the knowledge we think it has. The only way forward is to design experiments capable of distinguishing genuine perception from transcription of the prompt. Separating the goal from dynamics and supervising the read path are concrete steps in that direction. At Q2BSTUDIO, we apply these principles in every AI, cloud, and automation project to build robust, transparent, and genuinely useful software.
