.jpg)
The automatic generation of web applications from user interface (UI) screenshots represents a promising advance at the intersection of artificial intelligence and software development. However, recent efforts have mostly focused on visual fidelity, leaving aside a critical aspect: the ability to infer underlying interactive behavior. This gap has been addressed by the new UI2App benchmark, designed to evaluate how vision-language models can reconstruct not only the appearance of an application, but also its navigation logic, state management, and full functionality from a static image.
The study analyzes 327 screenshots grouped into 45 coherent sets of states, corresponding to functional web applications with multiple routes. The evaluation is structured across four dimensions: executability, navigation reachability, visual fidelity, and interaction inference. Precisely this last one, measured by the Interaction Inference Score (IIS), reveals a worrying gap: the leading model in visual fidelity barely achieves 7.5 out of 100 in IIS, falling far behind others that prioritize functional understanding. Complex interactions, such as maintaining state across pages, prove to be a widespread bottleneck, with half of the evaluated models scoring zero in that category.
These results have direct implications for the development of AI for businesses and for creating custom applications. While AI-based assistants can quickly generate visually appealing mockups, true utility lies in their ability to translate those images into functional and navigable products. In this context, having a technology partner that masters both artificial intelligence and custom software development becomes essential. Q2BSTUDIO offers precisely that combination: from requirements definition to the implementation of complete solutions integrating AI agents, AWS and Azure cloud services, and Power BI dashboards for business intelligence.
Cybersecurity also plays a fundamental role, as an automatically generated application must ensure data protection and transaction integrity. Companies looking to leverage process automation and AI for their workflows need a holistic approach that spans from cybersecurity to user experience optimization. Ultimately, UI2App highlights that interaction inference remains a major challenge for current models, reinforcing the need to combine the power of artificial intelligence with human expertise in enterprise software development.
