Production AI agent security preflight
- Before production, enforce least-privilege identity and scoped tool/API access; defend against prompt injection/unsafe inputs; map/protect sensitive data; validate every generated tool call and parameter; gate high-impact actions; and use tamper-resistant logs, monitoring, rate limits/timeouts, ownership, and a tested kill switch. Continuously red-team post-release.
- Use identity-aware RBAC and least-privilege access with per-tool and per-resource scopes.
- Place policy enforcement and validation between the agent and every downstream tool or API.
- Require explicit approval or deterministic policy gates for irreversible or high-impact operations.
- Use structured inputs, untrusted-content isolation, secret/PII filtering, and prompt-injection testing.
Before production, enforce least-privilege identity and scoped tool/API access; defend against prompt injection/unsafe inputs; map/protect sensitive data; validate every generated tool call and parameter; gate high-impact actions; and use tamper-resistant logs, monitoring, rate limits/timeouts, ownership, and a tested kill switch. Continuously red-team post-release.
Do not launch until permissions, tool validation, monitoring, incident response, ownership, and emergency shutdown have documented, tested evidence. AI agent security must cover autonomy, tools, APIs, memory, integrations, downstream systems, and external side effects—not only generated text. Agents need these controls because they can make decisions, invoke tools, access data, and change external systems.
Record each control as part of production readiness. For every control, record a pass/fail result, retained evidence, an accountable owner, and a lifecycle stage. The required depth should reflect the agent’s autonomy, accessible data, reachable systems, business impact, and potential blast radius.
1. Inventory, ownership, and blast-radius mapping
Start with visibility. An undocumented tool, data source, identity, or integration can bypass controls applied to the main agent.
- Inventory the agent and dependencies. Record the agent, model, prompts or configuration, tools, APIs, databases, data sources, vendors, integrations, dependencies, and reachable systems. Evidence: current inventory or registry. Owner: platform or security. Stage: pre-launch gate and recurring review.
- Assign ownership and identities. Give each production agent, integration, data source, and workflow a named engineering, security, or operations owner. Document the identities used to access them. Evidence: owner register and architecture record. Owner: engineering manager. Stage: pre-launch gate.
- Map autonomy and blast radius. Document what the agent can read, write, disclose, approve, delegate, or trigger, including downstream systems and external side effects. Where applicable, include MCP servers, sub-agents, and agent-to-agent communication paths. Evidence: data-flow, trust-boundary, and reachable-system map. Owner: application security. Stage: pre-launch and every material change.
2. Identity, authentication, and least privilege
Give the agent a controlled service identity rather than unrestricted access to a user or administrator account. Scope access to the user, workflow, resource, tool, and action. For implementation details, see our IAM guide for AI applications.
- Enforce strong authentication and identity-aware RBAC. Authenticate users, agents, tools, and services, and tie authorization decisions to an identifiable user or service identity. Evidence: authentication and authorization tests for allowed and denied workflows. Owner: IAM team. Stage: pre-launch and continuous.
- Apply least privilege with narrow scopes. Define per-tool and per-resource permissions, delegation boundaries, tenant boundaries, and workflow limits. Evidence: permission matrix plus tests showing unused or excessive scopes are denied. Owner: application owner. Stage: pre-launch and access review.
- Control credentials and prohibit unrestricted access. Store secrets through approved credential controls, rotate or revoke them independently, and never grant unrestricted administrative, shell, file-system, or database access. Evidence: secret-management record, revocation test, and restricted-execution policy. Owner: platform engineering and security architecture. Stage: pre-launch and continuous.
3. Data, privacy, secrets, and memory protection
Protect information before it enters context, while it is processed, and after it becomes persistent memory. Minimize the data available to each workflow. For data-leakage controls, use our LLM data leakage prevention checklist.
- Classify and minimize sensitive data. Identify personal, financial, health, proprietary, confidential, and other sensitive data flows, then provide only what the task requires. Evidence: data inventory, classification, and approved data-flow map. Owner: privacy and data owner. Stage: pre-launch and change review.
- Mask, filter, redact, encrypt, or exclude protected information. Apply controls before data enters prompts, retrieval, memory, tool payloads, or logs, and test for leakage through outputs and traces. Evidence: representative-data tests and encryption/access-control records. Owner: data security. Stage: pre-launch and continuous.
- Protect memory and provider handling. Isolate memory by user, tenant, and workflow where required; validate writes; preserve provenance and integrity; define retention and rollback; and review provider or vendor retention and training policies. Evidence: memory isolation, poisoning, deletion, rollback, and provider-policy reviews. Owner: application and data-platform teams. Stage: pre-launch and recurring review.
4. Prompt, input, retrieval, and output security
Treat user prompts, retrieved documents, uploaded files, web content, API responses, and tool responses as untrusted content. Prompt engineering and model selection can reduce risk, but neither is a complete security boundary. For output-specific safeguards, see our LLM output security checklist.
- Separate trusted instructions from untrusted content. Document which instructions are trusted, isolate external content, and prevent untrusted text from directly changing policy or authorization decisions. Evidence: architecture design and isolation tests. Owner: AI application team. Stage: pre-launch gate.
- Test direct and indirect prompt injection. Use authorized tests against user content, documents, retrieval results, web content, file inputs, and tool responses. Include data exfiltration, goal manipulation, unsafe instructions, memory poisoning, and unauthorized tool use. Evidence: adversarial test results and remediation record. Owner: application security. Stage: pre-launch and continuous red team.
- Validate retrieval and generated outputs. Check authorization, relevance, sensitive data, policy compliance, and safe handling before retrieved content, model output, or a proposed action is used downstream. Evidence: output-policy tests and blocked-action traces. Owner: application owner. Stage: pre-launch and runtime.
5. Tool, API, MCP, and downstream-action controls
Place policy enforcement between the agent’s decision and every external side effect. A plausible tool call is not necessarily a safe or authorized tool call. For MCP-specific hardening, see our MCP client security checklist.
- Validate every agent-generated call. Check the tool name, schema, parameters, payload, destination, resource, record change, output type, and downstream action before execution. Evidence: tests showing invalid, unauthorized, malformed, and out-of-scope calls are rejected. Owner: API platform. Stage: pre-launch and runtime.
- Use allowlists, bounded integrations, and sandboxing. Explicitly allow permitted tools, domains, resources, operations, and data fields where practical; isolate code, file, browser, or other high-risk execution. Evidence: allowlist configuration and isolation tests. Owner: security and platform engineering. Stage: pre-launch and continuous.
- Secure MCP and agent-to-agent paths where applicable. Authenticate collaborating agents or MCP servers, validate their identities, tool descriptions, inputs, outputs, permissions, and trust boundaries, and protect their communications. Evidence: communication and authorization tests. Owner: platform security. Stage: pre-launch and continuous.
6. Bounded execution and risk-based approval
Autonomy should be proportional to impact. Use human approval or deterministic policy gates for high-impact actions, not automatically for every low-risk action.
- Define high-risk actions and approval thresholds. Include financial transactions, destructive writes, mass deletion, privilege changes, sensitive-data disclosure, compliance-sensitive actions, customer-facing actions, and other irreversible operations. Evidence: approved risk register and action-policy matrix. Owner: product, risk, and business owners. Stage: pre-launch gate.
- Gate consequential actions. Require a recorded human approval or deterministic policy decision before a high-impact action executes, with the approver, identity, scope, and conditions captured. Evidence: approval and denial tests, including duplicate and partial-execution cases. Owner: product and security. Stage: pre-launch and runtime.
- Bound execution with rate limits, timeouts, retries, and circuit breakers. Stop loops, excessive volume, repeated failures, unexpected destinations, and unresponsive dependencies. Evidence: failure-path tests demonstrating safe behavior. Owner: SRE or platform engineering. Stage: pre-launch and runtime.
7. Adversarial, functional, and authorization testing
Test the complete action chain, not only the quality of the agent’s text. Use non-production systems and authorized credentials for adversarial work. For a safe adversarial workflow, follow our AI agent red-teaming guide.
- Run adversarial security tests. Cover prompt injection, data exfiltration, tool misuse, privilege escalation, memory poisoning, goal manipulation, unsafe retrieval, and indirect attacks. Evidence: repeatable red-team or automated test results. Owner: application security. Stage: pre-launch and continuous.
- Run functional and authorization tests. Confirm correct tool selection, valid arguments, permission boundaries, tenant separation, data access, approval behavior, and downstream state changes for each supported workflow. Evidence: test cases covering permitted and denied users, roles, tenants, and workflows. Owner: QA and IAM. Stage: pre-launch and regression.
- Test failure and integration paths. Exercise wrong tool selection, incorrect arguments, permission denials, timeouts, loops, partial data, conflicting records, unavailable approvers, incomplete workflows, API failures, memory failures, and queue or database errors. Evidence: staging results and documented safe outcomes. Owner: QA, integration engineering, and platform teams. Stage: pre-launch and regression.
8. Evaluation, lifecycle management, and release gates
A production-ready agent completes defined tasks reliably, uses tools correctly, stays within permission boundaries, and leaves an auditable record. Traditional QA alone is insufficient for nondeterministic, multi-step action chains. For a repeatable lifecycle, use our AI agent risk assessment guide.
- Define agent-specific acceptance criteria. Evaluate task completion, tool correctness, permission boundaries, data handling, system-state changes, failure behavior, and auditable outcomes—not only plausible responses. Evidence: approved evaluation plan and representative test set. Owner: product and engineering. Stage: pre-launch gate.
- Run staging, regression, and release-gate tests. Use fixed tasks, abuse cases, edge cases, failure paths, production-like integrations, and non-production data or credentials before each material release. Evidence: signed test results, known exceptions, residual-risk record, and go/no-go decision. Owner: QA and release engineering. Stage: every release.
- Version and control the agent lifecycle. Track the model, prompt, retrieval configuration, tools, policies, and deployment configuration; require review before promotion; preserve lineage; support replay of relevant historical inputs; and enable rollback to an approved version. Evidence: version history, approval record, lineage, replay results, and rollback test. Owner: AI platform and release owner. Stage: pre-launch and every change.
9. Runtime observability, monitoring, and auditability
Centralize telemetry so investigators can reconstruct what the agent received, attempted, approved, and changed. Protect audit evidence from unauthorized alteration and apply appropriate privacy and retention controls.
- Centralize traces and telemetry. Correlate relevant inputs, model interactions, retrievals, tool calls, parameters, decisions, approvals, identities, outputs, errors, and state changes to a workflow or request. Evidence: end-to-end trace from representative runs. Owner: observability team. Stage: pre-launch and runtime.
- Protect audit records. Apply controlled access, integrity or tamper resistance, defined retention, privacy handling, and evidence-preservation procedures. Record denied calls, failed calls, retries, exceptions, approvals, and downstream changes as well as successful actions. Evidence: audit-control review and completeness test. Owner: security operations. Stage: pre-launch and continuous.
- Monitor behavior and drift. Alert on unusual tool usage, data access, communications, action sequences, policy violations, configuration changes, and behavior outside approved workflows. Assign alert ownership and a review cadence. Evidence: dashboards, alert tests, and escalation record. Owner: SOC or operations. Stage: continuous.
10. Incident response, rollback, governance, and evidence
Assume the agent can be manipulated, misconfigured, or exposed to a failed downstream control. The response path must be faster and more reliable than the agent’s action path.
- Maintain a tested pause, override, and kill switch. An authorized operator must be able to pause or disable the agent and consequential tools, revoke credentials, and block integrations. Evidence: production-relevant exercise with a recorded result. Owner: operations or security. Stage: pre-launch and recurring test.
- Document incident response and recovery. Cover detection, escalation, containment, evidence preservation, communication, exception handling, rollback, restoration to a known safe state, and post-incident review. Evidence: approved playbook and exercised recovery path. Owner: incident-response lead and SRE. Stage: pre-launch and recurring exercise.
- Maintain governance and evidence readiness. Map controls to applicable security, privacy, and governance requirements without treating one framework or vendor checklist as definitive. Retain owners, pass/fail results, exceptions, review dates, vendor assessments, and remediation records. Evidence: control register and evidence repository. Owner: security governance. Stage: pre-launch and continuous.
Final production go/no-go rule
Do not launch if any of the following is untested or lacks documented ownership and evidence:
- Permissions: identity, authentication, RBAC, least privilege, credential controls, and delegation boundaries.
- Tool and action validation: schemas, parameters, destinations, outputs, approvals, and downstream state changes.
- Data protection: classification, minimization, masking, encryption, provider handling, memory isolation, retention, and recovery.
- Testing: functional, adversarial, authorization, regression, integration, staging, and failure-path coverage.
- Operations: tracing, protected audit logs, monitoring, alert ownership, rate limits, timeouts, circuit breakers, and incident response.
- Emergency response: a production-relevant pause or kill-switch test, rollback path, and named decision-maker.
After release, continue red-teaming and evaluating the agent. Review access, prompts, models, retrieval, tools, data, integrations, configuration drift, audit completeness, and reachable systems whenever the agent or its blast radius changes.






