The ability of an intelligent system to identify in an image which areas are suitable for performing a specific action —opening a door, grasping an object, sitting on a chair— is known as affordance grounding. This skill is fundamental in robotics, augmented reality, and autonomous systems, but has traditionally been complex to implement due to the visual ambiguity of images and the semantic similarity between actions. Recent advances in large-scale language and vision models (LVLMs) have shown that their internal attention maps encode implicit spatial information, even when the model only generates text. Researchers have developed a novel approach that selects, among all output tokens, the one whose attention map is dominantly activated over the relevant object, transforming that semantic signal into affordance heatmaps without the need for supervised training. This method, called TokAG, outperforms previous approaches based on weakly supervised learning, achieving significant improvements in benchmarks such as AGD20K and HICO-IIF.
The practical application of this technology goes beyond the laboratory: companies developing AI for businesses can integrate affordance grounding into visual inspection systems, robotic assistants, or mixed reality interfaces. At Q2BSTUDIO we offer custom applications that incorporate advanced perception capabilities, combining artificial intelligence with cloud services such as AWS and Azure to scale solutions securely. We also provide cloud services AWS and Azure that facilitate the deployment of vision and language models, as well as business intelligence services to analyze the data generated by these systems. Our experience in cybersecurity ensures that solutions are robust against threats. Thus, token-based affordance grounding becomes a practical and accessible tool for any organization seeking to equip its AI agents with more precise and efficient spatial understanding.
