The leap from a functional AI agent in a lab environment to a reliable production system is not a model-tuning problem but an architecture governance one. At Q2BSTUDIO, where we develop custom software and AI solutions, we have learned that a prototype that 'works' can hide serious risks: unaudited identities, uncontrolled tools, runaway costs, or unrecoverable failures. This article proposes an original production checklist, designed for technical teams and executives who want to deploy AI agents with enterprise guarantees.
What distinguishes a prototype from a production-ready system? The difference is not in the prompt, but in the control planes surrounding the model. A prototype demonstrates that an agent can complete a task; a production system must be able to explain who acted, what state was retained, which tools were used, how behavior was evaluated, what telemetry was captured, where approvals interrupted execution, how cost was bounded, how failures were contained, and who owns the service. These are eight axes that must be treated as separate control planes, not as afterthought patches.
Identity: who is really acting In production, an agent operates with at least three identities: the requester (person, application, or event), the agent itself (non-human workload identity), and the delegated identity presented to the tool or data source. They cannot be collapsed into a single service account. At Q2BSTUDIO, when integrating agents with cloud AWS/Azure, we require each identity to have a clear scope, short-lived tokens, automatic rotation, and audit logging. Without this separation, any security failure is impossible to trace.
State: designed, not accumulated State in a prototype is often a conversation history that grows unbounded. In production, you must separate conversation state, workflow state, durable memory, evidence, approvals, and artifacts. Each type needs a schema, owner, retention policy, and encryption. The model should only see a temporary view of the state needed for the current step, not the authoritative store. The ability to pause an agent, inspect its state, and resume exactly once without duplicated side effects is a maturity indicator.
Tools: production APIs, not toy functions Every tool the agent invokes must have a capability record defining its purpose, data classification, authentication method, input/output schemas, timeouts, retry limits, idempotency, and approval requirements. At Q2BSTUDIO we classify tools by consequence: read, propose, reversible change, high-impact change. The more authority a tool has, the less discretion the model should have. Tool arguments must be validated with deterministic code, and results must be treated as untrusted data, because an external document can contain malicious instructions.
Evaluations: from prompt to full workflow A model benchmark does not tell if the agent chose the right tool, respected approval boundaries, recovered from a timeout, or stayed within budget. Production evaluations must measure the complete workflow: deterministic code-level tests, offline evaluations with representative tasks, trace evaluations that inspect the path (not just the final answer), adversarial evaluations (prompt injection, memory poisoning), and online evaluations with real samples. Release thresholds must be specific: zero critical policy violations, zero incorrect high-impact tool calls, task success above 95%.
Observability: reconstruct every run Traditional monitoring sees the external request and perhaps one model call. An agent may have multiple model calls, retrievals, memory reads, policy decisions, tool calls, retries, approvals, and handoffs. The observable unit is the complete agent run. We recommend a root span per invocation, with children for each operation. Content capture should be disabled by default and enabled only with redaction and encryption policies. Low-cardinality records (agent version, model, policy, tool, environment, outcome) allow an on-call engineer to diagnose with a single run identifier.
Security: runtime enforcement Prompt instructions are not security controls. The threat model must cover the entire agent path: user input, retrieved documents, web content, memory, model output, tool metadata, arguments and results, inter-agent messages, sandbox execution, network egress, state stores, and logs. At Q2BSTUDIO, when implementing cybersecurity in AI systems, we apply controls like schema validation, egress limits, secret scanning, encryption in transit and at rest, and shutdown path testing. A manipulated agent with read-only access to public documentation is a lesser risk than one with shell access and production credentials.
Cost: per-run budget The cost of an agent is not the cost of a single model call, but the cost of the entire run: instructions, history, retrieved context, tool schemas, retries, validation calls, reasoning, output generation, and handoffs. You must set a budget at the run boundary: maximum model calls, tool calls, retries, elapsed time, parallel branches, external API consumption. The primary metric should be cost per successful task. A cheap model that causes retries or incorrect actions is more expensive. Model routing, prefix caching, context compaction, and stopping rules must be part of the runtime, not optional best practices.
Approvals: state machine Human intervention must depend on consequence, reversibility, authority, and confidence. Read-only actions can be automatic; proposals must be clearly marked as drafts; reversible changes require approval only when evaluation evidence is not strong; high-impact actions (financial, destructive, external) need mandatory approval with separation of duties. The workflow must pause without losing state, and upon resumption must revalidate authorization, policy, and approval expiry.
Failure handling: safety and intent Agent failures are not only model failures. They can be due to invalid structure, stale retrieval, corrupted state, tool timeout, expired approval, unavailable dependency, budget exhaustion, or blocked policy. The response must depend on the failure class: retry only known transient failures with exponential backoff, use idempotency keys, detect state loops, use circuit breakers, preserve checkpoints before high-impact operations, and divert unrecoverable work to a manual queue with complete evidence. Fallback paths must not change the security model without the same rigorous design.
Ownership: beyond the original developer The first production incident will quickly reveal who is not assigned. An agent crosses several domains: business, product, platform, identity, data, tools, model, observability, cost, and governance. Each domain needs an accountable owner, and a global service owner must be responsible for the end result. Delivery documentation must include architecture, inventory, evaluation results, dashboards, runbooks, escalation contacts, and shutdown procedures.
Staged rollout Full production authority should not be granted on the first release. A disciplined rollout expands traffic and authority in stages: shadow mode (without exposing output or executing side effects), read-only pilot, canary with approvals, bounded automation, and controlled scale. Each expansion changes the data, identity, tools, failure, cost, and ownership model. The release question is not 'does the demo still work?' but 'what new authority are we granting, what evidence supports it, and how quickly can we contain it if the evidence is wrong?'.
At Q2BSTUDIO we help companies build these control planes on BI/Power BI, cloud, and automation, integrating AI agents that not only work but can be governed, explained, and recovered. Production is not the end of the prototype; it is the beginning of a system that deserves enterprise trust.
