2026 AI Vendor Security Assessment Checklist for Compliance Reviews

A useful AI vendor security assessment checklist covers data handling and privacy, model governance, access controls, AI-specific attacks, compliance, incidents, subprocessors, evidence, and decision gates.

Use this reusable AI vendor questionnaire for procurement, contracting, renewals, expansions, and material service changes. It supplements—not necessarily replaces—your conventional third-party-risk and vendor-security review.

Start with the risk tier

Match the review to the service’s data, access, autonomy, integration scope, and business impact. An internal assistant requires a different level of scrutiny than an AI system that processes sensitive information, affects customers, connects to business systems, or takes actions without approval. For a structured way to classify and prioritize these risks, see the AI security risk assessment guide.

How to use the checklist

Adapt your existing third-party-risk questionnaire with AI-specific sections alongside the baseline review. Send the structured questionnaire to the vendor and require documentary or demonstrable evidence for material controls.

For each control, record:

• Question, applicability, and risk tier
• Vendor response and evidence requested
• Evidence received and validation status
• Risk rating, residual risk, and evidence quality
• Remediation, compensating control, owner, and deadline
• Contractual requirement, exception, and approval status
• Decision date, approval authority, and reassessment trigger

Use the assessment before production deployment, at renewal, before expanding data or access, and after a material model, subprocessor, integration, or service change. A PDF, Excel worksheet, or existing TPRM workflow can deliver it; the content and decision record matter most. To determine whether the vendor review should be supplemented by technical testing, compare an AI security assessment versus penetration test.

1. Vendor and AI service profile

Capture the service context before reviewing controls. The same vendor can present different risks depending on its model, data, integrations, and autonomy.

Use case and service details

• What business process, user groups, and locations will use the service?
• Is it a hosted large language model, AI-enabled application, retrieval-augmented generation system, agentic system, or another AI architecture?
• Will it provide internal assistance, customer communication, recommendations, decisions, or autonomous actions?
• What data will be submitted, generated, retrieved, embedded, or logged?
• What is the service’s business criticality and expected customer exposure?
• What model, feature, integration, or processing changes are planned during the contract term?

Models, providers, and data flows

• Which models and versions does the service use, and who provides the underlying or foundation models?
• Where are prompts, inputs, outputs, embeddings, uploaded files, and logs processed and stored?
• Which cloud providers, data providers, subprocessors, extensions, or other fourth parties support the service?
• Which dependencies can change without customer approval, and how will the vendor provide change notifications?
• Can the vendor provide a current architecture or data-flow description?

2. Data privacy, training use, retention, and isolation

Ask about prompts, inputs, outputs, embeddings, uploaded files, retrieved content, fine-tuning data, evaluation data, and related logs—not only ordinary application records. For safeguards covering training and inference data, see the LLM data leakage prevention guide.

Data use and training

• Are customer prompts, inputs, outputs, embeddings, or other customer data used for model training?
• Are they used for fine-tuning, evaluation, quality improvement, service improvement, or product development?
• What restrictions or opt-out rights apply, and are they enforceable by contract?
• Do data-use settings differ between testing, support, and production environments?
• Is customer data combined with data from other tenants, customers, or models?
• What information is sent to the underlying model provider or other subprocessors?

Retention, deletion, access, and residency

• How long are prompts, inputs, outputs, embeddings, files, and logs retained?
• What happens when the customer requests deletion or terminates the service?
• Can the customer export its data, configuration, prompts, outputs, and relevant records in a usable format?
• Who can access customer data, for what purposes, and under what approval process?
• Is data shared with subprocessors, support teams, model providers, or other parties?
• Where is data processed and stored, and what residency or cross-border commitments apply?
• How is customer data isolated between tenants, models, environments, and projects?

Require the vendor to distinguish policy statements from technical enforcement. A yes-or-no answer without a data-flow explanation, contractual restriction, configuration detail, or supporting documentation is not automatically validated.

3. Model governance and assurance

Model governance should explain what the vendor operates, how it was evaluated, where it is limited, and how changes are controlled.

Documentation and transparency

• What are the model names, versions, providers, intended uses, and prohibited uses?
• Can the vendor provide model cards, technical documentation, and available information about training-data provenance?
• What known limitations, failure modes, dependency risks, and expected error patterns should customers understand?
• What performance information is available for the intended use case and data types?
• How does the vendor notify customers about model swaps, version changes, retraining, fine-tuning, or material behavior changes?

Testing, drift, updates, and human oversight

• What testing covers accuracy, reliability, fairness, demographic bias, explainability, and harmful or misleading outputs?
• How are evaluation datasets selected, maintained, and kept relevant to the use case?
• How does the vendor monitor model drift and performance degradation after deployment?
• Are updates tested and approved before release, and can customers delay or control material updates?
• Are consequential outputs reviewed by a qualified human before a decision or action?
• Can the vendor support decision auditing, including relevant input, output, model version, reviewer, and action records?

4. Security architecture and technical controls

Review baseline controls protecting the service and its connections. The required depth should reflect the deployment and risk tier. For a broader control baseline to compare against the vendor’s architecture, use the AI security controls checklist.

Identity, access, and integrations

• How are users authenticated and authorized?
• How are credentials, API keys, service accounts, and non-human identities issued, protected, rotated, and revoked?
• Which APIs, OAuth scopes, tools, plugins, or connected systems are available?
• Are permissions inherited from the source system, separately administered, or expanded by the AI service?
• Can access be limited by tenant, user, role, environment, document, action, or data category?
• What is the potential integration blast radius if the model, integration, credential, or connected account is compromised?

Protection, monitoring, and application security

• How is data protected during transmission and storage?
• How are service environments segmented, and how are production, test, and support access separated?
• How does the vendor manage vulnerabilities, dependencies, patches, and security testing?
• How are inputs validated and outputs filtered before reaching users or connected systems?
• Are rate limits, abuse monitoring, and protections against excessive or automated use available?
• What security, administrative, integration, and model events are logged?
• How can the customer review relevant logs, and how are they protected, retained, and exported for investigations?

AI-specific threats

• How is the service tested for prompt injection and attempts to override system instructions?
• How does it reduce the risk that untrusted retrieved content causes an unsafe response or action?
• How are data poisoning, adversarial inputs, output misuse, and relevant model-specific vulnerabilities assessed?
• Does the vendor conduct adversarial testing relevant to the deployed model and integrations?
• How are vulnerabilities reported, prioritized, remediated, and communicated to customers?
• What prevents an AI output from being treated as an authorized instruction without appropriate validation?

5. Retrieval, agents, and autonomy

Retrieval-augmented generation

Apply these questions when the service retrieves customer documents or other external content:

• Does retrieval inherit the source system’s permissions at the user and document level?
• Is sensitive content masked or filtered before embedding and prompt construction?
• Can the service attribute an answer to its retrieved sources?
• How are embeddings protected, separated, deleted, and prevented from being used across tenants?
• Are retrieval requests, source documents, permissions, and returned passages recorded in an appropriate audit trail?

Agentic and autonomous workflows

Use these questions when AI can call tools, modify records, send communications, approve transactions, or act beyond generating information:. For a structured view of risks in systems that can use tools and take actions, see the OWASP Top 10 for Agentic Applications.

• What actions can the system take, and which require explicit approval?
• Are permissions limited to the minimum necessary system, role, record, and operation?
• Are high-impact actions subject to approval gates or human review?
• Can the system operate in a sandbox or constrained environment before production access?
• Are rate, transaction, action, and time limits available?
• Can operators quickly disable the model, connection, workflow, or credential through a kill switch?
• Are action chains logged, including the trigger, model version, retrieved context, tool call, result, reviewer, and final action where applicable?
• What happens when the model is unavailable, uncertain, inaccurate, or detects an unsafe condition?
• Is there a manual fallback for critical processes?

6. Governance, compliance, supply chain, and contracts

Governance and accountability

• Who owns responsible AI, security, privacy, model risk, and incident response?
• Does the vendor maintain an AI system inventory, risk register, or impact-assessment process?
• What policies govern model development, deployment, human oversight, acceptable use, and change management?
• Which laws, regulations, contractual requirements, or frameworks does the vendor consider applicable?
• What evidence supports claims involving SOC 2, ISO 27001, ISO/IEC 42001, NIST AI RMF, GDPR, HIPAA, the EU AI Act, or other relevant requirements?
• What systems, locations, products, and periods are covered by each certification or audit report?
• Can the customer audit relevant controls or receive an independent report and supporting information?

Certifications are not guarantees. A certification or framework-alignment claim does not by itself prove safe model behavior, privacy for this deployment, or compliance with every applicable legal requirement. Review its scope, validity, and relevance to the service being purchased.

Subprocessors and dependencies

• Identify every foundation-model provider, cloud provider, data provider, subprocessor, extension, and material fourth-party dependency.
• What data and functions does each dependency receive?
• How does the vendor notify customers before or after a material dependency change?
• What contingency exists if a model provider, cloud service, or subprocessor becomes unavailable or unacceptable?

Incident response, resilience, and contractual safeguards

• How does the vendor define an AI-related security, privacy, integrity, availability, or model-behavior incident?
• When and how will the customer be notified, and what notification SLA applies?
• What investigation, evidence-preservation, forensic-support, and customer-cooperation procedures are available?
• Who is accountable for remediation, and how are root causes and corrective actions reported?
• What commitments address availability, monitoring, failover, redundancy, manual fallback, and degraded performance?
• How are model drift, retraining, and unsafe updates handled?
• Do the contract and order documents make material questionnaire answers enforceable?
• Do they address data-use restrictions, incident notification, service commitments, liability, audit rights, model-change notices, termination, data export, and lock-in?

7. Evidence and validation worksheet

Request evidence in proportion to risk. Distinguish vendor self-attestation from independently verifiable proof.

Label each response self-attested, documented, demonstrated, or independently validated. Record gaps rather than treating an unanswered question as a pass. Compare claims with the actual models, connections, permissions, and environments proposed for deployment.

8. Scoring and procurement decision gates

There is no universal scoring model for AI vendor risk. Use a documented method that fits the use case, risk tolerance, and evidence quality.

Recommended process

1. Mark each control applicable, not applicable, or requiring clarification.
2. Rate control risk as low, moderate, high, or critical.
3. Rate evidence quality as self-attested, documented, demonstrated, or independently validated.
4. Record residual risk after remediation and compensating controls.
5. Apply critical-control overrides. A failed or unanswered control involving customer-data use, isolation, excessive access, autonomous actions, incident notification, or a material dependency may prevent approval regardless of the aggregate score.
6. Assign an owner and deadline for every accepted gap.

For a simple comparison, score eight domains from 0 to 10: data and privacy; model governance; security and access; AI threats and autonomy; compliance and auditability; incidents and resilience; subprocessors and contracts; and evidence quality. Use the resulting 0–80 total only as a decision aid, not as a substitute for critical-control review.

These bands are an illustrative working rubric, not an industry standard. They must not override a failed critical control.

9. Pilot and deployment controls

For a conditional or pilot decision, limit exposure while the vendor demonstrates its controls:.

• Use test or sanitized data where feasible and prohibit unapproved sensitive data.
• Restrict users, integrations, credentials, API scopes, records, and autonomous actions.
• Require human review for consequential outputs and approval for high-impact actions.
• Define success criteria for accuracy, reliability, security, privacy, and operational performance.
• Monitor prompts, outputs, access, actions, errors, incidents, and model or configuration changes as appropriate.
• Set a time limit, rollback process, kill switch, fallback procedure, and production approval gate.

10. Ongoing monitoring and red flags

Reassessment triggers

Reassess periodically and after:

• A model swap, version change, retraining, or material behavior change
• A new foundation-model provider, cloud provider, or subprocessor
• Expanded data access, new integrations, or broader customer exposure
• New autonomous features, tools, or action permissions
• A security, privacy, availability, or model-behavior incident
• A significant change to the service, contract, regulatory context, or business criticality

Common red flags

• Undisclosed or unrestricted use of customer data for training or improvement
• Vague answers about retention, deletion, access, processing, storage, or tenant isolation
• Unknown models, model providers, subprocessors, or material data flows
• Missing model documentation, relevant testing, or limitations information
• Excessive permissions, unclear OAuth scopes, weak integration boundaries, or no action controls
• No credible response to prompt injection, poisoning, adversarial misuse, or output misuse
• No meaningful incident-notification commitment, investigation support, or response SLA
• Unworkable data export, termination, audit, model-change, or lock-in terms
• Reliance on a certification without showing its scope, validity, or relevance

Final approval record

Close the AI vendor security assessment with a written decision identifying the approved use case, data scope, integrations, autonomy level, evidence quality, unresolved gaps, compensating controls, contractual requirements, owners, deadlines, approval authority, and next reassessment trigger.

Use an evidence-based process for consistent procurement and ongoing vendor oversight. A checklist does not guarantee security, privacy, safe AI behavior, or legal compliance; the final decision must reflect the organization’s use case, risk tolerance, and ability to monitor the service after deployment.

Watch this in video

AI-Driven Third-Party Risk Management: Turning Vendor Data …