The rise of Software Engineering (SE) agents has transformed how organizations approach the development of complex systems. These agents, based on large language models (LLMs), can understand vast codebases and perform engineering tasks with minimal human intervention. However, building these systems remains a poorly documented challenge. A recent study with 20 practitioners from 12 organizations and 80 survey respondents reveals that the development process of SE agents is changing radically: bottlenecks shift rather than disappear. Non-coding tasks such as requirements definition, coordination, review, and deployment become more prominent, while evaluating agent output becomes a central and novel activity. In this context, companies like Q2BSTUDIO, specialized in custom software development, are helping their clients integrate AI agents into their workflows, offering solutions that range from cloud infrastructure to cybersecurity.
The study identifies a seven-stage workflow that reflects the maturity of teams building SE agents. These stages include problem definition, agent design, implementation, evaluation, iterative refinement, deployment, and maintenance. Unlike traditional development, where coding was expensive and slow, implementation has become cheaper thanks to LLMs. This causes other phases, such as evaluation, to become the new bottleneck. Practitioners spend increasing time defining robust evaluation criteria and interpreting automated test results. Instead of writing code from scratch, developers supervise and correct agent outputs, requiring deep domain understanding and knowledge of the underlying model capabilities.
Another key finding is the shift toward evaluation-driven development. In this paradigm, specifications become versioned artifacts readable by both humans and agents. The iteration cycle is no longer about writing more code, but about refining test cases and acceptance criteria. Agent performance metrics are stored alongside training datasets, and every new model or agent version requires a full battery of evaluations. For companies adopting this approach, having tailored AI tools is essential. Q2BSTUDIO, for example, designs custom dashboards with Power BI that enable real-time monitoring of agent effectiveness, integrating data from multiple sources and facilitating evidence-based decision making.
The study also highlights six main challenges teams face. The first is unreliable evaluation signals: automated tests may not capture semantic correctness or desired behavior in complex contexts. To mitigate this, practitioners combine unit tests with human reviews and user simulations. The second challenge is comprehension debt: as agent-generated code outpaces developers' ability to understand it, maintenance risks accumulate. This forces documentation of every decision and detailed logging of interactions. The third challenge is behavioral changes introduced by provider-side model updates. When OpenAI, Anthropic, or any other provider updates its LLM, agents can fail without warning, requiring re-evaluation and adjustment. Other challenges include inference cost management, evaluation scalability, and integration with legacy systems.
How do organizations address these issues? Identified practices include building automated evaluation pipelines that run thousands of tests per commit, implementing continuous monitoring systems with detailed logs, and adopting modular architectures that allow swapping agent components without rewriting the entire system. There is also growing collaboration between development and operations teams (MLOps) to ensure agents are deployed safely and efficiently. Cybersecurity becomes critical since agents may access code repositories and databases; for this reason, Q2BSTUDIO offers cybersecurity services to audit access and protect digital assets.
On the infrastructure side, the cloud plays an essential role. SE agents require elastic environments to run massive evaluations and store large volumes of training data. AWS and Azure platforms provide the necessary services, from GPU instances for inference to scalable databases. Q2BSTUDIO helps companies design optimized cloud architectures, leveraging managed services to reduce operational overhead. Additionally, integration with Business Intelligence tools like Power BI enables visualization of agent performance and early anomaly detection.
In summary, building SE agents is not merely a technical problem but an organizational and methodological shift. Practitioners must adapt their processes to accommodate continuous evaluation, understanding model behavior, and managing uncertainty. Companies that master these dynamics will be better positioned to harness the potential of generative AI in software development. Q2BSTUDIO, with its expertise in custom software, artificial intelligence, cybersecurity, and cloud, stands as a strategic ally to face these challenges. From specification definition to production deployment, its comprehensive approach ensures that SE agents not only work but deliver real business value.
