What Is an AI Vendor Assessment? A Practical Guide to Third-Party AI Risk

Organizations rarely build every artificial intelligence system themselves. They buy software with embedded AI, connect to general-purpose models through an API, license specialized models, use vendor data, adopt AI features inside systems they already own, and allow contractors to operate AI services on their behalf.

Each arrangement can create value. Each can also introduce risk that the customer cannot see directly. The model may change without warning. Training data may be poorly documented. Customer information may be retained or used to improve the product. Performance may look strong in a vendor demonstration but weaken in the customer’s population or operating environment. A critical feature may depend on another provider several layers down the supply chain.

An AI vendor assessment is the process for making those risks visible before the organization becomes dependent on the product.

What is an AI vendor assessment?

An AI vendor assessment is a structured, risk-based evaluation of a third party that provides an AI system, model, data source, platform, component, or AI-enabled service. It examines whether the product is suitable for its intended use, whether the vendor’s claims are supported by evidence, whether important risks can be controlled, and whether the customer will retain enough information and authority to govern the system throughout its lifecycle.

The assessment usually brings together procurement, business ownership, legal, privacy, cybersecurity, compliance, enterprise risk, data governance, technology, and the people responsible for AI oversight. Higher-risk uses may also require model validation, accessibility, human resources, safety, ethics, internal audit, or sector-specific expertise.

The question is not whether the vendor is trustworthy in general. The question is whether this product is trustworthy enough for this use, with these people, data, controls, and consequences.

Why an ordinary vendor review is not enough

Existing third-party risk programs give AI governance a valuable foundation. They already examine financial stability, cybersecurity, privacy, service continuity, legal terms, subcontractors, and regulatory compliance. Those questions still matter.

AI adds risks that conventional questionnaires may not reach:

• Performance varies by context. A model can perform well in one population, language, workflow, or data environment and poorly in another.
• Outputs are probabilistic. The same or similar input may not always produce the same result.
• Models and services change. A vendor may update the model, safety filters, training data, integrations, or terms after approval.
• The system may be difficult to explain. The customer may not receive enough information to understand why a result occurred or to contest it.
• Data rights may be uncertain. Questions can arise about training data, generated content, confidential information, personal data, and intellectual property.
• Human reliance changes the risk. A tool described as advisory may function like an automated decision when employees routinely accept its recommendation.
• The supply chain may be layered. The vendor may rely on a model provider, cloud platform, data supplier, plug-in, or subcontractor that the customer never selected directly.

NIST addresses this directly in the AI Risk Management Framework. Its Govern function calls for policies covering third-party software, data, hardware, and supply-chain issues, including transparency, testing, instructions, intellectual-property concerns, auditability, incident response, and contingency planning. Its Manage function calls for third-party resources and controls to be monitored and documented, not merely reviewed once.

What counts as an AI vendor?

The review should not be limited to companies that advertise themselves as AI vendors. Relevant third parties include:

• Software providers that have added generative or predictive AI features.
• General-purpose model and API providers.
• Cloud platforms hosting models, data, or AI workloads.
• Recruiting, healthcare, education, financial, marketing, or case-management systems that use automated scoring or recommendations.
• Data brokers, labeling providers, and synthetic-data suppliers.
• Consultants or contractors who build, configure, or operate AI for the organization.
• Open-source models or components incorporated into a vendor product.
• Subprocessors and subcontractors supporting the vendor’s AI service.

Start with the intended use, not the questionnaire

The same product can be low risk in one use and high risk in another. A language model used to brainstorm internal meeting titles is different from the same model used to rank job applicants, draft clinical guidance, determine eligibility, investigate fraud, or communicate legal rights.

Before sending questions to the vendor, document:

1. The business problem and expected benefit.
2. The exact decision or workflow the system will influence.
3. Who will use the system and who will be affected by its outputs.
4. The data the system will receive, create, infer, retain, or share.
5. Whether outputs affect rights, safety, employment, health, education, credit, public services, or other significant interests.
6. The role of human review and whether the reviewer can realistically disagree with the system.
7. The harm that could follow from error, bias, misuse, unavailability, or unauthorized disclosure.
8. The alternatives available if the product is rejected or fails.

This use-case description becomes the basis for risk tiering, evidence requests, testing, contract terms, approval, and monitoring.

Use risk tiers to determine the depth of review

Tier	Typical characteristics	Review approach
Lower risk	Internal productivity, approved tools, nonsensitive data, no meaningful decision about a person, easy human verification, and limited consequences if wrong.	Basic registration, security and privacy checks, acceptable-use controls, standard contract review, and confirmation of data handling.
Moderate risk	Customer-facing content, operational recommendations, important business processes, confidential data, integrations, or outputs that require skilled human review.	Expanded documentation, use-case testing, legal and compliance review, defined performance limits, monitoring, change notification, and incident terms.
Higher risk	Meaningful influence over employment, education, healthcare, credit, insurance, housing, public benefits, safety, law enforcement, legal rights, vulnerable populations, or essential services.	Enhanced impact and risk assessment, independent testing or validation, strong evidence requirements, senior approval, formal human oversight, audit rights, strict change control, continuous monitoring, and an exit plan.
Prohibited or unacceptable	The use violates law, policy, contract, organizational values, or risk appetite, or the vendor cannot provide minimum evidence for a consequential use.	Reject or stop the use. A commercial need does not cure an unacceptable risk.

Risk tiering should consider both the product and the proposed use. A respected provider does not lower the impact of a harmful decision. A small vendor does not automatically create high risk if the use is limited and the controls are strong.

The eight-step AI vendor assessment

1. Identify the product and supply chain

Document the legal vendor, the product, the AI features, the underlying models, data providers, hosting environment, subprocessors, plug-ins, and critical integrations. Determine whether the vendor developed the model, fine-tuned another model, or simply provides access to a third party.

2. Classify the intended use and inherent risk

Assess the risk before new controls are applied. Consider affected people, sensitivity of data, decision significance, scale, autonomy, reversibility, visibility, and the consequences of failure. Identify legal or policy triggers that require enhanced review.

3. Request evidence, not promises

Ask the vendor for current documentation that supports its claims. Sales presentations and broad responsible-AI principles are not enough for a consequential use. The required evidence should match the risk tier.

4. Evaluate the vendor’s governance and controls

Review how the vendor manages AI risk internally. Look for named ownership, risk assessment, documented testing, security and privacy controls, incident management, complaint handling, model and data change processes, subcontractor oversight, monitoring, and governance that extends beyond product marketing.

5. Test the product in the customer’s context

Vendor testing does not replace customer testing. Evaluate performance with representative data, users, languages, edge cases, accessibility needs, adversarial inputs, and realistic workflows. Compare the results to documented thresholds and to a non-AI alternative where appropriate.

6. Negotiate contract protections

Convert important controls into enforceable obligations. The contract should address data use, confidentiality, security, documentation, changes, incidents, audit or assurance, subcontractors, intellectual property, service continuity, deletion, transition support, and responsibility for failures.

7. Decide and document

The decision may be approve, approve with conditions, pilot within limits, defer pending evidence, or reject. Record the evidence reviewed, unresolved limitations, compensating controls, owner, risk acceptance, monitoring plan, conditions, expiration, and events that require reassessment.

8. Monitor, reassess, and prepare to exit

Track product performance, incidents, complaints, vendor changes, control effectiveness, new subcontractors, legal developments, and concentration risk. Reassess at renewal and after material changes. Maintain a practical plan for data return or deletion, replacement, continuity, and safe decommissioning.

What evidence should the vendor provide?

Evidence area	Examples	What it helps establish
System documentation	System card, model card, intended-use statement, architecture summary, integration guide, and operating instructions.	What the system is designed to do, how it works at a useful level, and where its limitations begin.
Performance and testing	Evaluation methods, datasets, results by relevant subgroup, error analysis, robustness testing, red-team findings, and known failure modes.	Whether claims are supported and whether performance is relevant to the customer’s context.
Data governance	Data sources, provenance, rights, quality controls, retention, customer-data practices, training and fine-tuning rules, and deletion procedures.	Whether data is lawful, suitable, protected, and used as expected.
Privacy and security	Privacy impact material, encryption and access controls, security testing, incident response, vulnerability management, penetration testing, and assurance reports.	Whether systems and information are protected and failures can be detected and managed.
Responsible-AI controls	Risk or impact assessments, fairness testing, explainability approach, human-oversight design, abuse prevention, content controls, and complaint processes.	Whether risks to people and society are treated as operating issues rather than principles alone.
Lifecycle management	Release notes, model versioning, change policy, monitoring approach, rollback capability, decommissioning plan, and support commitments.	Whether the customer can remain in control after the original approval.
Supply chain	Underlying models, open-source components, data providers, subprocessors, hosting providers, geographic locations, and dependency changes.	Which parties and components can affect the service and where hidden concentration or transfer risk exists.
Independent assurance	Relevant certifications, audits, conformity assessments, validation reports, or qualified third-party reviews.	Whether important claims have been examined by someone other than the sales team or product owner.

No single document proves that a product is safe or compliant. Certifications and audit reports may support the assessment, but their scope, date, exclusions, testing depth, and relationship to the proposed use must be examined.

Core questions to ask an AI vendor

Do not confuse the questionnaire with the assessment

A completed questionnaire is one source of information. The assessment is the organization’s evaluation of that information against the use case and risk appetite.

Reviewers should identify claims that need verification, contradictions between documents, missing evidence, unclear ownership, outdated reports, broad exclusions, and conditions the customer would need to impose. For higher-risk uses, the organization may need a product demonstration, access to technical specialists, customer testing, references from comparable deployments, independent validation, or a limited pilot.

A vendor’s refusal to disclose proprietary model details does not automatically end the review. Some information may legitimately remain confidential. The question is whether the vendor can provide enough documentation, testing, assurance, contractual accountability, and operational visibility for the customer to manage the risk. For a consequential use, “trust us” is not a control.

Test the system in the real workflow

Testing should reflect how the organization will actually use the product. That includes normal cases, difficult cases, misuse, foreseeable workarounds, and the pressures employees face in real operations.

A practical evaluation may examine:

• Accuracy, reliability, false positives, false negatives, and uncertainty.
• Performance for relevant groups, languages, disabilities, and operating conditions.
• Hallucination, unsupported claims, harmful content, or unsafe recommendations.
• Prompt injection, data leakage, unauthorized access, and other security threats.
• Whether explanations are understandable and useful to the intended reviewer.
• Whether human reviewers notice errors, have enough time, and possess the authority to intervene.
• Whether logs and records support investigation, challenge, appeal, and audit.
• Whether the product creates a measurable improvement over the existing process.

Pass and fail thresholds should be defined before the test where possible. Otherwise, a team that wants the product may rationalize disappointing results after the fact.

Contract protections that matter

Legal counsel should tailor contract language to the organization, jurisdiction, sector, and risk. Common areas for AI-specific terms include:

• Permitted use of customer data: whether information may be retained, reviewed by humans, used for training, or shared.
• Confidentiality and security: controls, incident notice, cooperation, evidence preservation, and remediation.
• Documentation and instructions: information needed to use the system safely and comply with customer obligations.
• Performance commitments: defined service levels or product claims where the risk justifies them.
• Model and product changes: notice, impact information, customer testing time, consent where necessary, version control, and rollback.
• Subcontractors and underlying models: disclosure, approval rights, flow-down obligations, and change notice.
• Intellectual property: rights in inputs and outputs, infringement claims, training-data issues, indemnity, and acceptable use.
• Audit and assurance: access to relevant reports, testing results, records, and additional evidence after incidents or material changes.
• Legal and regulatory cooperation: information and support needed for assessments, notices, investigations, complaints, and regulators.
• Human oversight and controls: features and information the customer relies on to review or intervene.
• Suspension and termination: the right to stop unsafe or noncompliant use without being trapped by commercial terms.
• Exit and continuity: data portability, deletion, transition support, retention of necessary records, and contingency arrangements.

Approval outcomes and conditions

An assessment should end with a clear outcome:

Outcome	When it fits	What to document
Approve	Evidence and controls are sufficient for the intended use and risk tier.	Owner, approved use, baseline configuration, monitoring, renewal, and change triggers.
Approve with conditions	Risk is acceptable only with additional restrictions or actions.	Conditions, responsible owners, deadlines, validation, and consequences if unmet.
Limited pilot	Evidence is promising but the organization needs controlled testing.	Scope, data limits, users, duration, success criteria, monitoring, and stop conditions.
Defer	Important documentation, testing, contract terms, or controls are incomplete.	Missing items, responsible parties, and what is required for reconsideration.
Reject	Risk is prohibited, unsuitable, unsupported by evidence, outside appetite, or cannot be controlled.	Decision basis, alternatives considered, and whether future reconsideration is possible.

Approval should identify the exact use. A system approved to summarize internal documents is not automatically approved to score employees or answer questions from the public.

Red flags that deserve attention

• The vendor cannot identify the underlying model, material subcontractors, or important data sources.
• Marketing claims are precise, but testing methods and results are unavailable.
• The vendor promises the system is unbiased, fully explainable, or completely compliant.
• Customer data may be used for training by default, and opting out is unclear or ineffective.
• The vendor can materially change the model or data practices without notice.
• The product offers no useful logs, version record, incident path, or rollback mechanism.
• Human oversight exists only on paper, with no information or authority needed to intervene.
• The contract disclaims all responsibility while restricting the customer’s ability to test or audit.
• The vendor has no clear process for complaints, harmful outputs, vulnerabilities, or significant incidents.
• The vendor resists reasonable questions by calling all AI information confidential.

A red flag does not always require rejection. It requires an explanation, evidence, compensating controls, restricted use, stronger contract terms, or escalation. Several red flags together may show that the customer cannot govern the product responsibly.

Monitoring after purchase

Third-party AI risk changes over time. Monitoring should match the product’s importance and risk. It may include:

• Performance, error, drift, bias, override, and complaint measures.
• Vendor notices, release notes, model versions, and changed subprocessors.
• Security vulnerabilities, privacy events, service outages, and AI incidents.
• Changes to terms, data use, retention, intellectual-property protections, or pricing that affect control.
• New regulatory classifications or obligations.
• Overreliance, workarounds, unauthorized uses, and use beyond the approved scope.
• Vendor financial condition, acquisition, strategic change, or loss of critical personnel.
• Concentration risk created by dependence on one model, platform, or infrastructure provider.

Material changes should trigger reassessment. Examples include a new model, a changed use case, new personal or sensitive data, new autonomy, new integrations, a different hosting location, altered training practices, a significant incident, or evidence that performance no longer meets the approved threshold.

Build the exit plan before you need it

A high-risk or operationally important AI service needs a contingency plan. The organization should know how it will continue critical work if the vendor fails, the product becomes unsafe, access is suspended, a regulator intervenes, the vendor changes direction, or the contract ends.

The exit plan should address:

• Alternative manual processes, vendors, models, or systems.
• Export of data, configurations, prompts, records, logs, and documentation.
• Secure deletion and proof of deletion.
• Retention of records needed for audit, complaint, investigation, or legal obligations.
• Transition time, assistance, cost, and responsibility.
• Communication to users and affected people.
• Safe decommissioning without losing oversight of past decisions.

NIST’s AI RMF specifically connects third-party risk with contingency planning and safe decommissioning. Exit is not merely a procurement term. It is a governance control.

A practical approach for smaller organizations

A small nonprofit, school, public agency, or business may not have separate AI, model-risk, privacy, procurement, and security teams. It can still perform a disciplined assessment.

For a lower-risk tool, document the use, confirm data handling, review security and privacy, identify the underlying provider, check whether customer data trains the model, test representative tasks, restrict sensitive uses, name an owner, and record the decision.

For a consequential use, obtain qualified legal, security, privacy, and technical help. A small organization should not lower its standard because it has fewer staff. It should narrow the use, choose a better-documented product, use independent expertise, or decide not to automate the decision.

How the assessment fits leading frameworks and regulation

The NIST AI RMF treats acquisition and third-party resources as part of AI governance across the full lifecycle. Govern 6 calls for policies addressing third-party AI systems, data, transparency, testing, instructions, intellectual property, procurement, supply-chain risk, and contingency processes. Manage 3 calls for third-party resources and their controls to be regularly monitored and documented.

ISO/IEC 42001 applies to organizations that develop, provide, or use AI, including AI supplied by third parties. Its management-system approach connects leadership, responsibilities, risk management, lifecycle controls, transparency, performance evaluation, monitoring, and continual improvement. An assessment can provide evidence that those responsibilities are being applied to procurement and vendor oversight.

The European Union AI Act assigns different responsibilities to providers, deployers, importers, distributors, and other actors. The customer’s obligations depend on its role, the system, the use, and the applicable implementation date. Official European Commission guidance emphasizes documentation across the AI value chain and, for relevant high-risk uses, operating instructions, monitoring, human oversight, input-data quality, incident response, and information for affected people. Organizations should obtain legal advice for their specific role and jurisdiction rather than relying on a vendor’s general statement of compliance.

CISA’s software-acquisition guidance provides an additional foundation for evaluating supplier and software-assurance risk. AI review should connect to that existing procurement discipline, not operate as a separate island.

The bottom line

An AI vendor assessment is not designed to eliminate all third-party risk. Its purpose is to make the intended use, evidence, limits, controls, responsibilities, and remaining risk clear enough for an accountable decision.

A strong assessment answers five questions: Do we understand what the system does? Is there credible evidence that it works for our use? Can we control the risks that matter? Will the vendor give us the information and rights needed to govern it? Can we monitor it, respond when it changes, and leave safely if necessary?

If the organization cannot answer those questions, it is not ready to buy the product, no matter how impressive the demonstration looks.