Introduction

Enable leaders to establish policies and processes for managing risk when the organization uses third-party AI/models, SaaS AI services, or integrates third-party AI into products.

At the Strategic Leadership level, you are setting the direction for AI adoption and governance across the organization. You need to balance innovation with risk management, establish frameworks that enable responsible AI use, and ensure that the organization's AI strategy aligns with its broader governance objectives.

This lesson is designed to be accessible to professionals at all experience levels while providing the depth needed for practical application. Whether you are encountering these concepts for the first time or building on existing knowledge, the material ahead will strengthen your ability to navigate AI governance challenges with confidence and competence.

Core Concepts

Practical Use Cases

Scenario 1: Financial Services Firm Evaluating Cloud-Based Credit Scoring AI

A Credit Risk Officer at a bank is evaluating a cloud-based AI service for credit risk assessment. Process:

• Vendor Assessment: Evaluates vendor (major cloud provider). Key findings:
• - Reliability: Vendor financially stable with 99.99% uptime SLA
• - Security: SOC 2 Type II certified; encryption; data residency in U.S.
• - Transparency: Vendor provides model performance metrics; some transparency about methodology
• - Fairness: Vendor tested for fairness; provides disparity ratios by demographic
• - Support: 24/7 support; escalation path to senior engineers
• Contract Negotiation: Negotiates specific requirements:
• - Data Protection: Data encrypted in transit and at rest; data residency in U.S.
• - Compliance: Vendor warrants compliance with FCRA, UDAP; bank right to audit
• - SLAs: 99.95% uptime SLA; <500ms latency SLA
• - Exit: 60-day notice for termination; data export in standard format
• - Liability: Vendor liable for breach of confidentiality or non-compliance; cap on liability
• Integration & Testing: Before deploying in production:
• - Test vendor AI performance on bank's data; validate accuracy claims
• - Test data handling; verify encryption and security controls
• - Document vendor's AI methodology, performance, limitations
• - Establish monitoring and escalation paths
• Ongoing Governance:
• - Monthly monitoring of performance vs. SLAs
• - Quarterly review of accuracy on bank's portfolio
• - Annual relationship review with vendor; discuss roadmap and changes
• - Monitor cost and usage; watch for unexpected escalation

Scenario 2: Tech Company Using Open-Source AI Libraries

A Data Science Team at a tech company is using open-source ML libraries (TensorFlow, PyTorch). Governance:

• Assessment: Lightweight assessment for open-source:
• - Community health: How active? How many contributors?
• - Security: Any known vulnerabilities? Security update process?
• - License: What's the license? Compatible with our IP strategy?
• - Documentation: Sufficient documentation? Community support?
• Requirements:
• - Include open-source in software inventory
• - Document version used
• - Monitor for security updates; establish update process
• - Understand license implications
• Ongoing Governance:
• - Monitor open-source security advisories
• - Plan upgrades to maintain current versions
• - Track if dependencies change or project maintenance wanes

Scenario 3: Healthcare Organization Using Third-Party Diagnostic AI

A Chief Medical Officer at a hospital is evaluating a third-party diagnostic AI service for radiology. Governance approach:

• Vendor Assessment: Comprehensive assessment due to patient safety risk:
• - Clinical Validation: What validation data? Performance in relevant population?
• - Transparency: Can hospital understand how AI makes recommendations?
• - Fairness: Tested on diverse patient populations? Performance disparities?
• - Liability: If AI causes patient harm, who is liable?
• - Compliance: FDA approval if required? Privacy compliance?
• Contract Requirements:
• - Liability: Vendor liable for patient harm from AI error
• - Clinical Support: Vendor provides clinical validation data and support
• - Data Protection: Patient data protected; secure transmission; audit trail
• - Audit Rights: Hospital right to audit vendor's systems and controls
• Integration & Testing:
• - Pilot with sample of patient cases
• - Compare AI recommendations to radiologist interpretations
• - Track false positives/negatives
• - Assess integration with hospital workflows
• Ongoing Governance:
• - Continuous monitoring of AI accuracy vs. radiologist interpretations
• - Patient safety monitoring; escalation for unusual cases
• - Periodic comparison to benchmarks; alternative vendor evaluation
• - Annual contract review

Anti-Patterns & Misuse Risks

Anti-Pattern 1: No Third-Party AI Governance - Organization uses third-party AI services without assessment or governance - Risks unknown; no contract protections; no monitoring - Risk: Vendor failure, security breach, compliance violation affects organization - Fix: Establish third-party AI assessment and governance process

Anti-Pattern 2: Assessment But No Ongoing Governance - Vendor assessed and approved initially - No ongoing monitoring; vendor changes; security vulnerabilities emerge - Risk: Risks materialize post-deployment; organization unaware - Fix: Establish ongoing monitoring and periodic vendor re-assessment

Anti-Pattern 3: Overly Restrictive Contracting - Negotiation so tough that vendor exits negotiation - Impossible to find third-party AI meeting organizational demands - Risk: Can't use beneficial technology; innovation stalls - Fix: Risk-tier requirements; essential requirements vs. nice-to-haves; negotiation strategy

Anti-Pattern 4: Insufficient Transparency Demands - Organization can't understand what vendor's AI does or how it works - Can't assess fairness or risks - Risk: Black-box dependency; can't address concerns if problems emerge - Fix: Include transparency requirements in assessment and contracts

[Practical Tip]

As you work through these concepts, consider how each one applies to your current role. Think of a specific scenario from your recent work where this concept would have been relevant. Building these mental connections between theory and practice is the fastest way to internalize new knowledge and make it actionable in your daily responsibilities.

Human Judgment Checkpoints

• Third-Party AI Governance Checkpoint:
• - Do you have a process for assessing third-party AI before deployment?
• - Are contracts negotiated to protect organization?
• - Is ongoing monitoring in place post-deployment?
• - Are alternative vendors evaluated periodically?
• Risk Assessment Checkpoint:
• - Have you assessed the risk level of third-party AI (high/medium/low)?
• - Are governance requirements proportionate to risk?
• - Are mitigations in place for identified risks?

Traceability & Defensibility Considerations

Third-Party AI Documentation: - Maintain vendor assessment documentation - Keep copies of executed contracts - Document ongoing monitoring and escalations - For any issues with vendor's AI, maintain incident log and resolution

[Practical Tip]

As you work through these concepts, consider how each one applies to your current role. Think of a specific scenario from your recent work where this concept would have been relevant. Building these mental connections between theory and practice is the fastest way to internalize new knowledge and make it actionable in your daily responsibilities.

Responsible AI & Control Considerations

Third-Party AI for Responsible AI: - Assessment should include fairness, transparency, and responsible AI dimensions - Contracts should require vendor compliance with responsible AI standards - Monitoring should include bias and fairness metrics

Practice & Reflection Prompts

• Third-Party AI Inventory: What third-party AI services, models, or libraries does your organization use? Which are critical vs. non-critical?
• Assessment Process Design: Design a vendor assessment process for your organization. What's most important to assess? What level of rigor for different risk levels?
• Contract Framework: For third-party AI critical to your business, design contract requirements. What must be in contract to protect organization?
• Monitoring Plan: For each critical third-party AI service, design ongoing monitoring plan. What metrics? What frequency? Who monitors?

[Practical Tip]

As you work through these concepts, consider how each one applies to your current role. Think of a specific scenario from your recent work where this concept would have been relevant. Building these mental connections between theory and practice is the fastest way to internalize new knowledge and make it actionable in your daily responsibilities.

Terms & Glossary

• Vendor Assessment: Evaluation of third-party AI vendor across reliability, security, transparency, fairness dimensions
• Contract Requirements: Negotiated terms protecting organization (data protection, SLAs, liability, exit terms)
• SLA (Service Level Agreement): Vendor commitment to performance levels (uptime, accuracy, latency)
• Ongoing Monitoring: Post-deployment tracking of vendor performance, security, compliance
• Vendor Risk: Risk that third-party vendor fails, changes, or creates security/compliance problem

Links to Related Lessons

• Chapter 3, Lessons 1-2: Third-party AI governance operationalizes acceptable-use policy and standards
• Chapter 4: Third-party AI compliance and performance is part of governance metrics
• Chapter 5, Lesson 3: Cross-functional team (legal, IT, risk, business) involved in third-party AI governance

Detailed Examples

The following examples illustrate how the concepts from this lesson play out in real-world oversight scenarios. Each example is designed to help you recognize similar situations in your own work and respond with appropriate professional judgment.

Example 1: Third-Party AI Assessment Framework

``` THIRD-PARTY AI VENDOR ASSESSMENT FRAMEWORK [Organization]

Use this framework to assess third-party AI services, SaaS AI platforms, or purchased models.

VENDOR INFORMATION

Vendor Name: [Name] Product/Service: [Description] Service Type: SaaS Cloud Service Purchased Model API/Integration Licensed Software AI Use Case: [What will we use this AI for?] Risk Level: High Risk Medium Risk Low Risk

ASSESSMENT DOMAINS

• VENDOR RELIABILITY & VIABILITY
• - Vendor background: [Years in business, funding, financial stability]
• - Uptime/reliability history: [Historical uptime %, incident history]
• - Roadmap: [Vendor's product roadmap; planned changes]
• - Viability risk: [Is vendor likely to be around in 5 years?]
• Risk Rating: Low Medium High
• SECURITY & DATA PROTECTION
• - Data encryption: [In transit? At rest?]
• - Access controls: [How does vendor control access to our data?]
• - Certifications: [SOC 2? ISO 27001? HIPAA? GDPR?]
• - Data residency: [Where is data stored? Jurisdiction?]
• - Breach response: [Process for handling security incidents?]
• Risk Rating: Low Medium High
• DOCUMENTATION & TRANSPARENCY
• - Model documentation: [What documentation provided about the AI?]
• - Explainability: [Can organization understand how AI works?]
• - Performance metrics: [Accuracy, precision, recall provided?]
• - Training data: [Info about training data provided?]
• - Limitations: [Known limitations disclosed?]
• Risk Rating: Low Medium High
• PERFORMANCE & ACCURACY
• - Performance metrics: [Vendor-provided accuracy metrics]
• - Validation: [Can organization validate on our data?]
• - Benchmarking: [How does performance compare to alternatives?]
• - SLA: [What uptime/performance guarantees?]
• - Monitoring: [Can organization monitor performance?]
• Risk Rating: Low Medium High
• FAIRNESS & BIAS
• - Fairness testing: [Has vendor tested for bias?]
• - Protected characteristics: [Which tested?]
• - Fairness metrics: [Disparity ratios, equalized odds, etc.?]
• - Known biases: [Disclosed known biases?]
• - Bias monitoring: [Ongoing bias monitoring?]
• Risk Rating: Low Medium High
• SUPPORT & ESCALATION
• - Support availability: [Hours? Response times?]
• - Escalation path: [Can organization escalate critical issues?]
• - Technical expertise: [Can vendor provide technical support?]
• - SLA violations: [How are SLA violations remedied?]
• Risk Rating: Low Medium High
• REGULATORY COMPLIANCE
• - GDPR compliance: [If applicable]
• - HIPAA compliance: [If applicable]
• - SOC 2 attestation: [Type I or II?]
• - Other regulatory requirements: [Specific to our industry/jurisdiction]
• - Compliance certification: [Relevant certifications obtained?]
• Risk Rating: Low Medium High
• CONTRACTUAL TERMS & CONDITIONS
• - Data protection terms: [Acceptable?]
• - Liability terms: [Cap on liability? Indemnification?]
• - Exit terms: [Notice period? Data portability? Transition support?]
• - Audit rights: [Organization right to audit vendor?]
• - IP ownership: [Who owns outputs? Can organization reuse?]
• Risk Rating: Low Medium High

OVERALL ASSESSMENT

Overall Risk Rating: Low Medium High

Recommended Action: Approve for use; proceed with integration Approve with conditions: [Specific conditions/mitigations] Escalate for further evaluation Reject; recommend alternative

Key Risks Identified: [List material risks identified during assessment]

Mitigation Strategy: [How will organization mitigate identified risks?]

Monitoring Plan: [How will organization monitor vendor post-integration?]

Approvals: Risk Lead: [Name, Date] Compliance Lead: [Name, Date] Legal: [Name, Date] Security: [Name, Date] Governance Office: [Name, Date] ```

Example 2: Third-Party AI Contract Requirements Checklist

``` THIRD-PARTY AI CONTRACT REQUIREMENTS CHECKLIST [Organization]

Use this checklist when negotiating contracts with third-party AI vendors.

DATA PROTECTION & SECURITY Data encryption in transit (TLS 1.2 or higher) Data encryption at rest (AES-256 or equivalent) Access controls: MFA, RBAC, audit logging Data residency: [Specified geographic location] Sub-processor management: Organization approval required for changes Breach notification: Within 24 hours of discovery Data segregation: Organization data isolated from other customers Vendor undergoes annual SOC 2 Type II audit

COMPLIANCE & AUDIT Compliance warranty: Vendor warrants compliance with [GDPR, HIPAA, CCPA, SOC 2, etc.] Audit rights: Organization can audit vendor systems [frequency: annually/quarterly] Compliance documentation: Vendor provides compliance certificates, attestations Regulatory cooperation: Vendor cooperates with regulatory requests/investigations Security incident notification: Vendor notifies if breach affects organization data

PERFORMANCE & SERVICE LEVELS Service uptime SLA: [Target %: 99.9%, 99.99%, etc.] Uptime measurement: [Third-party monitoring, vendor reporting] Uptime remedies: [Credits, service improvements if SLA violated] Performance SLA: [Latency, throughput, accuracy targets if applicable] Support levels: [24/7, business hours, response times] Escalation process: [How to escalate critical issues]

INTELLECTUAL PROPERTY & OWNERSHIP Output ownership: [Organization owns outputs; can reuse, modify, distribute] Licensing: [License granted to use vendor's IP within service] Open-source: [Vendor discloses open-source components; complies with licenses] Confidentiality: [Vendor does not use organization data or outputs for own purposes]

TERMINATION & TRANSITION Termination right: [Either party can terminate with notice period: 30/60/90 days] Data export: [Vendor provides organization data in standard formats] Data deletion: [Timeline for deleting data after termination: 30 days] Transition support: [Vendor provides reasonable cooperation for transition to new vendor] Termination fees: [What are costs for early termination?]

LIABILITY & INDEMNIFICATION Liability cap: [Cap on vendor liability, e.g., 12 months of fees] Indemnification: [Vendor indemnifies organization for IP infringement, breach of warranty] Consequential damages: [Vendor not liable for indirect/consequential damages] Insurance: [Vendor carries appropriate insurance; proof of insurance]

CHANGE MANAGEMENT Material changes: [Vendor notifies of material changes; organization can terminate if not acceptable] SLA changes: [Cannot reduce SLAs without consent] API/interface changes: [Notice period for breaking changes] Pricing changes: [Notice period; cap on increases]

DOCUMENTATION & TRANSPARENCY Model documentation: [Vendor provides documentation of AI methodology, training data, performance] Explainability: [Vendor provides explanations for AI outputs] Fairness testing: [Vendor tests for bias; provides fairness metrics] Known limitations: [Vendor discloses known limitations, biases, performance gaps] Updates: [Vendor provides notification of model updates; testing before deployment]

REGULATORY COMPLIANCE AI regulation compliance: [Vendor complies with AI regulations (EU AI Act, etc.)] Transparency requirements: [Vendor meets transparency/disclosure requirements] Consumer protection: [Vendor complies with consumer protection laws]

GENERAL TERMS Entire agreement: [Contract represents entire agreement] Dispute resolution: [Jurisdiction, venue, arbitration if applicable] Term: [Initial term, renewal terms, notice for non-renewal] Confidentiality: [NDA terms for protecting confidential information]

NEGOTIATION NOTES [Record what was negotiated, what was accepted, what was rejected, rationale]

Contract Approval Risk: [Name, Date] Legal: [Name, Date] Compliance: [Name, Date] Finance: [Name, Date] Governance: [Name, Date] ```

Putting It Into Practice

Strategic leadership requires translating these concepts into organizational capabilities and governance frameworks:

• Set clear expectations: Establish organizational standards for AI use that are specific enough to guide behavior but flexible enough to accommodate evolving capabilities.
• Build governance infrastructure: Ensure that committees, reporting lines, and escalation procedures are in place to support responsible AI adoption at scale.
• Champion responsible innovation: Balance the drive for AI-enabled efficiency with the imperative for risk management, ethical use, and stakeholder trust.
• Prepare for the future: Stay informed about emerging AI capabilities and regulatory developments. Position your organization to adapt proactively rather than reactively.

Key Takeaways

• Third-party AI requires different governance: Can't inspect/modify vendor AI; requires vendor assessment and contracts
• Assessment should be risk-tiered: High-risk third-party AI needs comprehensive assessment; low-risk can be lighter
• Contracts protect organization: Clear data protection, compliance, SLAs, and exit terms are critical
• Ongoing monitoring is essential: Vendor performance, security, compliance changes must be monitored post-deployment
• Documentation enables defensibility: Assessment, contracts, and monitoring create trail of risk management
• Transparency is key: Must understand vendor's AI enough to assess risks and monitor effectiveness

As you continue through this credential program, you will build on the foundation established in this lesson. Each subsequent lesson adds new dimensions to your understanding and expands your capability to work effectively with AI in oversight roles.