Overview

Rutger Bakker is the head of enterprise architecture at a Dutch insurance group. When his team deployed a third-party AI underwriting tool two years ago, they ran a thorough security review of the vendor's infrastructure. Penetration testing, SOC 2 report review, data processing agreements - the works. Fourteen months later, the vendor disclosed that a machine learning library in their own software stack had been compromised through a malicious update. The attacker had not touched the vendor's infrastructure at all. They had walked in through a dependency three layers down the supply chain.

Rutger's team had assessed the vendor. Nobody had assessed what the vendor depended on.

What Supply Chain Risk Means for AI Systems

The term supply chain security comes from software development, where it refers to vulnerabilities introduced through external dependencies - open-source libraries, third-party services, build tools, and cloud infrastructure that your organization did not write or control but relies on.

In AI systems, the supply chain is longer and more complex than it is in traditional software. A typical enterprise AI deployment might depend on: a foundation model from a major AI provider; a middleware layer from a specialist vendor; open-source machine learning frameworks; cloud infrastructure from a hyperscaler; data connectors to internal and external sources; and monitoring and logging tools from yet more vendors.

Each dependency is a potential entry point. A compromise, failure, or policy change at any point in that chain can affect the security, performance, or reliability of your AI system - even if your organization's own practices are flawless.

The Three Main Vectors

Supply chain risk in AI systems tends to manifest through three paths.

Compromised model weights or training data. If a third-party model has been tampered with - either at the provider level or through a supply chain attack on the provider - the model may produce systematically manipulated outputs. This is sometimes called a backdoor attack: the model behaves normally in most cases but produces specific outputs when triggered by certain inputs. This type of attack is difficult to detect through standard testing because it is designed to evade it.

Vulnerable dependencies. The model runs on code. That code has dependencies. Those dependencies have their own dependencies. A malicious or compromised package anywhere in that stack can expose your system. The attack vector Rutger encountered - a compromised library update - is sometimes called a supply chain injection.

Third-party data and API exposure. Many AI systems pull in real-time data from external sources or expose outputs through APIs. A vendor whose data feed you rely on could expose you to data quality failures, adversarial data injection, or credential exposure. APIs you expose to partners are their entry point to your AI system's capabilities.

Assessing Third-Party AI Vendors

Vendor security assessment for AI systems needs to go beyond what is typical for general software procurement. The standard questions - "Do you have a SOC 2 report? Do you encrypt data in transit and at rest?" - are necessary but not sufficient.

Add these to your assessment framework:

Training data provenance. Where did the training data come from? Was it collected with appropriate permissions? Is it documented? For a model that will be used in a regulated domain, the provenance of training data can affect both compliance and liability.

Model update and versioning practices. How does the vendor manage model updates? Do they notify you when the underlying model changes? Can you pin to a specific version? A model update that changes output behavior without notice is both a security and a reliability risk.

Dependency inventory. What open-source libraries and third-party services does the vendor's system depend on? Do they have a software bill of materials - a SBOM (pronounced "ess-bom") - that documents these dependencies? If not, that is a gap worth naming in contract negotiations.

Incident history. Has the vendor had security incidents? What was disclosed, when, and what remediation was taken? Vendors who have had incidents and handled them well are often better security partners than vendors with no disclosed history.

Contractual Security Requirements

Security requirements that are not in the contract are wishes, not commitments. At minimum, AI vendor contracts should specify:

• Notification obligations. The vendor must notify you within a defined timeframe - typically 24–72 hours - of any security incident that could affect your data or systems.
  - Right to audit. You retain the right to conduct or commission security audits, with reasonable notice.
  - Subprocessor disclosure. The vendor must disclose significant subprocessors - the parties they in turn depend on - and notify you of changes.
  - Data deletion. Clear terms for what happens to your data on contract termination, including timelines and verification.
  - SLAs with teeth. Service level commitments should include meaningful remedies, not just the vendor's right to try to fix the problem.

Enterprise legal and procurement teams are often ahead of AI teams on standard contract language. If you are new to this, start with your legal department's standard SaaS terms and then layer on AI-specific additions.

Managing Concentration Risk

If your organization's AI capabilities depend heavily on one vendor - one foundation model provider, one cloud platform, one data source - you have concentration risk. A single vendor's outage, price change, policy shift, or security incident becomes your organization's crisis.

Managing concentration risk does not mean avoiding all major vendors. It means being conscious of the dependency and planning for it.

Practical approaches include: maintaining architectural abstraction layers so that swapping a model provider does not require rebuilding your entire system; using more than one cloud region or provider for critical AI workloads; keeping documentation of what each vendor provides and what would need to change if that vendor were unavailable; and avoiding vendor-specific API patterns that would make migration expensive.

Monitoring and Maintaining Oversight

Third-party security is not a one-time assessment. Vendors change. Their security posture evolves. Their own supply chains shift. Your monitoring needs to evolve with it.

Practical monitoring for third-party AI risk includes:

• Subscribing to vendor security bulletins and change notifications. Most enterprise vendors offer this. If yours does not, ask for it as a contract term.
  - Tracking publicly disclosed vulnerabilities in AI frameworks and libraries your vendors use. The National Vulnerability Database and equivalent resources publish these.
  - Reviewing vendor security reports annually - SOC 2, ISO 27001, or equivalent - and noting changes from the prior period.
  - Maintaining an AI system inventory that maps which vendors support which capabilities. When a vulnerability is disclosed, you need to know quickly whether you are affected.

The audit trail Rutger's team built - documenting which third-party models were used where - turned out to be the most valuable artifact when the vendor disclosed the library compromise. They could determine within a few hours which systems were potentially affected and which were not. Organizations without that inventory spent days in uncertainty.

Key Takeaways

• Your security posture includes your vendors' security posture. A well-secured system can still be compromised through a third-party dependency. Assess the supply chain, not just the vendor's front door.
  - AI supply chains are deeper than traditional software supply chains. Foundation models, middleware, ML frameworks, data connectors, and cloud infrastructure are each potential entry points.
  - Ask for a software bill of materials. An SBOM documents what a vendor's system depends on. Without it, you cannot know what you are inheriting.
  - Put security requirements in the contract. Notification timelines, audit rights, subprocessor disclosure, and data deletion terms are commitments only when contractually specified.
  - Manage concentration risk deliberately. Heavy dependence on a single vendor is a strategic vulnerability. Architectural abstraction makes it easier to adapt when a vendor's situation changes.
  - Maintain an AI system inventory. When a vulnerability is disclosed, knowing quickly which systems are affected is the difference between hours and days of uncertainty.
  - Third-party security monitoring is continuous, not annual. Vendors change, frameworks get patched, and new vulnerabilities emerge. Build a monitoring rhythm, not a one-time review.