.jpg)
Artificial intelligence has transformed cybersecurity, offering tools that promise to detect vulnerabilities automatically. However, a recent academic study exposes a harsh reality: deep learning models and large language models (LLMs) that excel in academic benchmarks fail when faced with real vulnerabilities in the Linux kernel. This finding underscores the gap between theory and practice, a challenge that companies must consider when adopting AI solutions.
In the study, researchers trained representative models such as ReVeal and LineVul with classic datasets (Juliet, Devign, BigVul, and ICVul) and then evaluated them with VentiVul, a set of 20 real Linux kernel vulnerabilities fixed in May 2025. The results show that the models fail to distinguish vulnerable code from non-vulnerable code in the representation space, and their performance drops drastically. This reveals that training datasets, with homogeneous distributions and noisy labels, do not reflect the complexity of real code.
Why does this happen? The models learn superficial patterns rather than deep semantic features of vulnerabilities. Code representation through graphs or transformers does not adequately capture the context needed to detect subtle flaws. Additionally, the lack of diversity in training data limits generalization. For a software development company, relying solely on these models would be risky; a hybrid approach combining AI with expert human analysis is needed.
Effective cybersecurity requires realistic evaluations. At Q2BSTUDIO we offer cybersecurity and pentesting services that complement automated tools with manual testing and contextual analysis. Our team identifies vulnerabilities that AI models overlook, providing comprehensive protection for critical applications.
Every organization has unique needs. That is why we develop custom applications and custom software that securely integrate artificial intelligence. Artificial intelligence for businesses can enhance anomaly detection, but it requires quality data and a solid infrastructure. Our AWS and Azure cloud services enable deploying these solutions with scalability and performance.
Furthermore, continuous security monitoring benefits from Power BI dashboards that visualize key metrics. AI agents can automate preliminary vulnerability analysis, freeing experts for more complex tasks. At Q2BSTUDIO we integrate business intelligence and process automation to provide a comprehensive view of the security posture.
The analyzed study demonstrates that AI in cybersecurity still needs to mature. To bridge the gap between theory and practice, companies must rely on partners who understand both the technical fundamentals and operational realities. At Q2BSTUDIO we combine technological innovation with practical experience, helping organizations protect their digital assets effectively.
