.jpg)
In the autonomous driving ecosystem, one of the most complex challenges is managing the uncertainty of agents that are not directly visible to the vehicle's sensors. Corners, parked vehicles, or even adverse weather conditions create hidden zones where pedestrians, cyclists, or other cars can appear suddenly. Traditional approaches often apply a uniform precautionary criterion for any occlusion, resulting in overly defensive maneuvers that penalize efficiency and comfort. Alternatively, they attempt to model the hidden space without truly evaluating how that information impacts route planning. To bridge this gap between perception and decision-making, approaches based on Visual Language Models (VLMs) have emerged, which not only detect the presence of hidden agents but also prioritize those whose behavior significantly alters the planned trajectory.
The key lies in measuring how much the vehicle's plan would change if the existence of a hidden agent were known. Using a planning KL divergence metric (PKL), the impact of each occlusion on the optimal route can be quantified from an informational standpoint. This ranking allows an expert VLM—such as OpenAI's latest generation models—to generate rich, structured annotations about the visual evidence and reasoning needed to integrate those agents into decision-making. The result is a much more relevant training dataset, teaching smaller models to focus on what is truly critical, enabling them to outperform even much larger zero-shot versions. In tests on the nuScenes dataset, the PKL-guided data selection strategy improved performance by up to 30% compared to random sampling.
Beyond the automotive domain, this principle of prioritization based on impact on the planner has applications in multiple fields. In mobile robotics, logistics, or even intelligent surveillance systems, the ability to quickly identify which environmental elements are truly decisive for action allows for optimizing computational resources and improving safety. This is where the custom application of artificial intelligence and AI agents becomes especially relevant. At Q2BSTUDIO, we develop custom software that integrates visual language models with planning engines, adapting them to each client's specific needs. Our experience ranges from implementing AWS and Azure cloud services to scale real-time video processing, to the cybersecurity required to protect communications between sensors and decision systems. Additionally, we combine these capabilities with business intelligence services like Power BI to monitor performance metrics and risk patterns in dynamic environments.
The enterprise AI we offer is not limited to implementing algorithms; we design complete architectures where computer vision and natural language processing work together. For example, in an autonomous fleet management system, a VLM can detect a pedestrian hidden behind a parked truck and, through causal reasoning, suggest an evasive maneuver. The vehicle not only reacts but anticipates. This type of solution requires multidisciplinary development, ranging from data ingestion to the orchestration of microservices in the cloud. In this context, our experts integrate artificial intelligence platforms with scalable and secure data pipelines, ensuring that every decision is backed by relevant information and processed with appropriate latency.
Current research shows that training smaller models with intelligently selected data can match or surpass massive architectures that lack that refinement. For companies looking to adopt autonomous driving or advanced automation technologies, this represents a clear opportunity: it is not always about having the largest model, but the best trained and contextualized one. Incorporating metrics like PKL divergence allows for building perception systems that are aware of decision-making, a qualitative leap over classical approaches. At Q2BSTUDIO, we accompany organizations on this path, developing custom applications that turn uncertainty into a manageable variable. From simulating complex scenarios to deploying visual language models, our team combines software engineering, data science, and strategic vision to enable the next generation of safe and efficient autonomous systems.
