The Hook

Your CISO schedules a meeting with IT leadership about the AI platform you're evaluating. She asks: "Where is our data going? Who has access? What happens if the vendor gets hacked? Can we audit them? What's their incident response SLA?"

You realize you haven't thought about security deeply. You picked the platform because it had good demos and good accuracy, not because you analyzed its security posture. You didn't ask about data residency. You didn't look at their certifications. You didn't consider subprocessors.

Now you're in contract negotiations, and your CISO and legal team want:

• SOC 2 Type II certification
• HIPAA compliance (if healthcare data is involved)
• Data processing agreement (DPA) meeting GDPR requirements
• Data residency guarantees (data must stay in specific regions)
• Encryption in transit (TLS 1.2+) and at rest (AES-256)
• Right to audit vendor controls
• Incident notification procedures with specific SLAs
• Subprocessor list and approval rights

The vendor responds: "Yeah, we support most of that, but there are caveats. Some features require data in our cloud infrastructure, not yours. Encryption at rest is available but encryption in transit isn't available in all regions. We have 500+ subprocessors in our ecosystem, and you inherit all of them. We can provide SOC 2 Type I, but Type II takes 18 months. DPA is available but costs extra."

This is where vendor evaluation gets real. Security is easy to ignore until you're in a breach. Then it becomes the most important thing. This lesson teaches you what to actually evaluate, how to prioritize requirements, and what red flags signal "don't choose this vendor." As IT leadership, you need to understand vendor security evaluation as thoroughly as your CISO does.

Purpose: Making Security a Gate, Not an Afterthought

Security and compliance must be gates, criteria that a vendor must meet before you choose them. If a vendor can't meet your security requirements, you don't choose them, no matter how good the AI accuracy is. This upfront rigor prevents expensive problems later.

Why This Matters for IT Professionals

One vendor security decision can cost your organization millions. A breach of 1M customer records can cost $10M+ in incident response, notification, lawsuits, and lost trust. One bad vendor choice can be catastrophic. The difference between average IT leaders and strong IT leaders is whether they evaluate security rigorously upfront or discover problems after deployment.

Core Concepts: Four Dimensions of Vendor Evaluation

Vendor evaluation for AI platforms requires assessing four dimensions systematically. Missing any dimension creates risk.

Dimension 1: Vendor Security Posture - How Secure Is Their Infrastructure?

Your vendor's security maturity determines the baseline security of your deployment. A vendor with poor security practices will have breaches. You need to know how secure they are.

Key certifications to assess:

• SOC 2 Type II: This is the gold standard for enterprise vendors. It's a third-party audit confirming the vendor's security controls and testing them over time (at least 6 months). Type II is important, Type I is insufficient for most use cases.
• ISO 27001: An information security management system standard. Good to have but less important than SOC 2.
• FedRAMP (if US government work): Required for government contractors. Extremely rigorous.
• Industry-specific certifications: HIPAA compliance for healthcare, PCI-DSS for payment processing, etc.

Key practices to assess:

• Penetration testing (do they regularly test for vulnerabilities?)
• Incident response plan (tested, documented, with clear SLAs)
• Employee background checks and vetting
• Access controls (who can access customer data, and how is it logged?)
• Security updates and patch management
• Data backup and disaster recovery

Red flags:

• No SOC 2 certification (major warning sign)
• SOC 2 Type I only (insufficient)
• Won't share penetration testing results
• No documented incident response plan
• No employee background check policy
• Vague answers about access controls

Dimension 2: Data Handling and Residency - Where Does Your Data Live?

Where your data is stored, how it's protected, and who can access it are fundamental to security and compliance.

Key questions to assess:

• Physical location: Where are servers physically located? (EU, US, Canada, etc.)
• Data isolation: Is your data isolated from other customers? (Yes, usually, but confirm)
• Multi-tenancy architecture: Are multiple customers' data mixed? (Should be logically isolated at minimum)
• Default encryption: Is data encrypted in transit (TLS 1.2+) and at rest (AES-256)?
• Encryption keys: Who holds the encryption keys? Can you hold your own? (Customer key management is more secure)
• Data deletion: If you request deletion, will your data be completely removed? (Critical for GDPR)
• Backup retention: How long are backups kept? When are they purged?
• Data residency options: Can you choose where data is stored? (Some vendors offer regional options)

Red flags:

• Data shared across customer accounts (security nightmare)
• No encryption in transit
• Vendor holds all encryption keys and won't give you options
• No clear data deletion process
• Backups retained indefinitely
• Only one geographic location, and it doesn't meet your requirements

Dimension 3: Compliance and Legal Agreements - What Does the Contract Commit To?

Different regulations require different vendor commitments. These commitments must be in writing.

Critical agreements:

• Data Processing Agreement (DPA): Required for GDPR compliance. Vendor commits to data protection requirements.
• Business Associate Agreement (BAA): Required for HIPAA compliance. Vendor commits to healthcare data protection.
• Service Level Agreements (SLAs): Specifies uptime, performance, incident response times
• Security requirements: Specific commitments about encryption, access controls, audit logging

Red flags:

• Vendor won't sign DPA (you can't use them with EU data legally)
• No BAA available for healthcare data
• No clear SLAs for incident notification
• Compliance requirements are "coming soon" instead of now
• Vendor claims compliance but won't put it in writing

Dimension 4: Subprocessors and Third-Party Risk - Who Else Has Your Data?

Vendors rarely process data entirely by themselves. They use cloud providers, analytics services, backup providers, etc. Each is a potential risk.

Key assessment:

• Subprocessor list: Does vendor provide complete list? Is it regularly updated?
• Subprocessor vetting: How thoroughly does vendor vet their subprocessors?
• Your approval: Can you approve/reject specific subprocessors?
• Audit rights: Can you audit subprocessor security?

Red flags:

• Won't provide subprocessor list
• Unwilling to share how they vet subprocessors
• 100+ subprocessors (too many to manage)
• No process for you to approve or reject subprocessors
• Subprocessor list changes frequently without notification

Why This Matters: The Cost of Choosing Wrong

Many companies choose AI platforms based on features and cost, then discover security/compliance issues that make the platform unsuitable. At that point, they've already invested in implementation and switching costs are high.

Organizations that succeed evaluate security upfront as a gate: "If this vendor can't meet our compliance requirements, we don't choose them, no matter how good the model is."

Core Concepts: The Four-Dimensional Security Evaluation

Key Insight: Evaluate on Four Dimensions

Dimension 1: Vendor Security Posture

How secure is the vendor's infrastructure and processes?

Evaluate:

• SOC 2 Type II - Third-party audit confirming vendor's security controls (required for most enterprises). Type II is important, Type I is less rigorous.
• ISO 27001, Information security management system (good to have, shows vendor takes security seriously)
• Penetration testing, Does vendor regularly test for vulnerabilities? (Ask for reports)
• Incident response plan, What happens if vendor gets hacked? How do they notify you?
• Employee background checks, Do they vet employees with access to customer data?
• Access controls, Who can access customer data? Is access logged and monitored?

Red flags:

• No SOC 2 certification (major red flag)
• SOC 2 Type I only (insufficient)
• No penetration testing or unwilling to share results
• No documented incident response plan
• Hundreds of employees with data access
• No background check policy

Key Insight: Data Handling and Residency

Where does your data live? Who can access it?

Evaluate:

• Data location, Where are servers physically located? (EU, US, on-prem?)
• Data isolation, Is your data isolated from other customers? (Yes for most, but some vendors share resources)
• Multi-tenancy, How are multiple customers' data separated? (Logical or physical isolation?)
• Default encryption, Is data encrypted in transit (TLS/SSL) and at rest (AES-256)?
• Encryption keys, Who holds the keys? Can you hold your own keys? (Customer key management is more secure)
• Data deletion, If you want data deleted, will it be completely removed? (Important for GDPR/CCPA compliance)
• Backup retention, How long are backups kept? When are they purged?

Red flags:

• Data shared across customer accounts (security nightmare)
• No encryption in transit
• Vendor holds all encryption keys (you can't decrypt without them)
• No process for data deletion
• Backups retained indefinitely
• Data stored in countries with weak data protection laws

Key Insight: Compliance Certifications and Agreements

Different regulations require different approaches.

GDPR (EU, UK, and more)

• Personal data of EU residents must be protected
• Vendor must sign Data Processing Agreement (DPA)
• Must be able to demonstrate "data minimization"
• Must have "right to be forgotten" (delete data if requested)
• Requires consent for data collection

Vendor evaluation:

• Does vendor offer DPA as standard? (Cost to add it? Timeline?)
• Can they support data deletion/anonymization workflows?
• Do they have EU data centers? (Nice to have, not always required)
• What's the compliance certification status?

Red flags:

• Vendor won't sign DPA
• No data deletion capabilities
• Data goes to US by default
• No transparency about subprocessors

HIPAA (Healthcare)

• Patient medical data must be protected
• Vendor must sign Business Associate Agreement (BAA)
• Requires encryption, access controls, audit logs
• Requires incident notification

Vendor evaluation:

• Does vendor offer HIPAA-compliant services? (Not all do)
• Do they sign BAA?
• Can they demonstrate HIPAA compliance? (Documentation?)
• What's the audit schedule?

Red flags:

• Vendor says "we can make it HIPAA-compliant" (no. They either are or aren't)
• No BAA offered
• No audit logs
• No incident notification process

SOX/FINRA (Financial Services)

• Financial data must be protected and auditable
• Requires strict access controls and audit trails
• Requires segregation of duties

Vendor evaluation:

• Can vendor demonstrate audit controls?
• Do they segregate duties?
• Can they support your audit schedule?

PCI-DSS (Payment Card Data)

• Requires specific security controls for payment card data
• Encryption in transit and at rest
• Access logging and monitoring

Vendor evaluation:

• Can vendor certify PCI-DSS compliance?
• Do they have recent audit results?

CCPA (California)

• Residents have right to know what data is collected
• Right to delete personal data
• Right to opt-out of data sales

Vendor evaluation:

• Can they support data access requests?
• Can they support deletion requests?
• Do they sell or share data?

Key Insight: Subprocessor and Third-Party Risk

Vendors often use third-party services. Each third party is a potential risk.

Evaluate:

• Subprocessor list, Does vendor provide list of all third parties with data access?
• Subprocessor vetting, How do they choose third parties? Are they vetted for security?
• Your approval, Can you approve/reject specific subprocessors?
• Right to audit, Can you audit subprocessors if needed?

Example: A vendor uses AWS, Stripe, Datadog, and five other services. That's eight entities with potential data access. Each needs evaluation.

Red flags:

• Won't provide subprocessor list
• 100+ subprocessors (too many to manage)
• Using shady services with poor security reputations
• No process for you to reject subprocessors

Practical Workflow: Systematic Vendor Security Evaluation

When evaluating an AI vendor, follow this structured process to ensure you don't miss critical security gaps:

Phase 1: Pre-Evaluation Qualification

• Define your security requirements based on the data you'll process:
  - Which compliance frameworks apply? (GDPR for EU data, HIPAA for health data, SOX for financial data, etc.)
  - Which regulations govern your industry?
  - What are your data residency requirements?
  -
  What encryption standards are non-negotiable?
• 
  Create your security gate criteria, requirements that must be met:
• SOC 2 Type II certification minimum
  - DPA availability (if handling EU data)
  - BAA availability (if handling health data)
  - Specific data residency regions required
  -
  Encryption in transit AND at rest required
• 
  Screen vendors against gate criteria immediately:
• Does vendor meet your minimum security requirements?
  - If not, eliminate them without wasting time on POC or negotiation

Phase 2: Deep Dive Assessment

• Request security documentation:
  - Full SOC 2 audit report
  - Compliance certifications (HIPAA, ISO 27001, FedRAMP, etc.)
  - Complete subprocessor list
  - Data residency options documentation
  - Encryption implementation details
  -
  Incident response plan
• 
  Conduct detailed security assessment:
• Have your CISO/security team review SOC 2 report for controls relevant to your use case
  - Verify subprocessor list, identify any that concern you
  - Map vendor security commitments to your compliance requirements
  -
  Identify gaps or limitations
• 
  Get customer references from your industry and ask about their security experience

Phase 3: Negotiation and Contract

• Ensure agreements include what you need:
  - Data Processing Agreement (DPA) for GDPR
  - Business Associate Agreement (BAA) for HIPAA
  - Service Level Agreements with specific incident response SLAs
  -
  Clear security commitments
• 
  Document limitations and assumptions:
• What are the caveats or limitations? (e.g., "encryption in transit not available in region X")
  - What controls remain your responsibility (shared responsibility model)?
  - What happens if vendor changes their security posture?

Phase 4: Pre-Deployment and Ongoing

• Verify security setup before going live:
  - Confirm encryption is enabled
  - Confirm data is stored in correct regions
  - Set up and test audit logging
  -
  Test incident notification procedures
• 
  Establish vendor security monitoring:

How will you stay informed of vendor security updates?
- What will you monitor to ensure vendor continues meeting requirements?
- How often will you reassess vendor security?

Practical Use Cases: Security Evaluation in Action

Use Case 1: Evaluating a Healthcare AI Vendor

Scenario: Hospital system wants AI for patient risk prediction. Considering several vendors.

Compliance Requirements:

• HIPAA-mandatory (patient data)
• Data residency: must stay in hospital's cloud account or on-premises
• Encryption: all data encrypted at rest and in transit
• Audit logs: all access must be logged and auditable

Vendor Evaluation:

Vendor
HIPAA BAA
Data Residency
Encryption
Key Management
Red Flags

Vendor A
Yes
Customer cloud
Yes (at rest)
Vendor keys
Partial encryption, shared infrastructure

Vendor B
Yes
On-premises only
Yes (both)
Customer keys
More work to implement

Vendor C
"Coming soon"
Required cloud
Partial
Vendor only
Not ready for HIPAA

Decision: Vendor B (on-premises, customer-managed keys, BAA in place)

Implementation:

• Deploy model in hospital's own Kubernetes cluster (on-premises)
• Data never leaves the hospital's infrastructure
• Hospital manages encryption keys (maximum control)
• Audit logs stored in hospital's SIEM
• Result: HIPAA-compliant, zero compliance risk

Use Case 2: Evaluating a Vendor for EU Operations

Scenario: Company operates in EU and US. Data residency required for EU customers.

Requirements:

• EU data must stay in EU (GDPR)
• US data can be in US
• Need single platform for both regions

Vendor Evaluation:

Requirement
Vendor Status
Action

EU data center
Yes, in Frankfurt
Use Frankfurt region

EU DPA
Yes, standard
No extra negotiation

US data center
Yes, in Virginia
Use Virginia region

Data isolation
Not great (multi-tenant)
Acceptable risk for this use

Customer keys
No (vendor holds keys)
Vendor holds keys (some risk)

Decision: Vendor works, but negotiate:

1. DPA signed upfront (not custom negotiation)
2. EU data physically in EU (verify in writing)
3. Data deletion capabilities (for GDPR compliance)
4. Incident notification SLA (within 24 hours if breach detected)

Examples: Practical Evaluation Tools

Example 1: Security Evaluation Checklist

Before signing a contract with AI vendor, require:

Vendor Security Basics

• [ ] SOC 2 Type II certification (or equivalent audit)
• [ ] ISO 27001 certification (preferred)
• [ ] Penetration testing results (shared under NDA)
• [ ] Documented incident response plan (tested at least annually)
• [ ] Employee background check policy (documented)

Data Handling

• [ ] Data isolation between customers (confirmed in writing)
• [ ] Encryption in transit (TLS 1.2+)
• [ ] Encryption at rest (AES-256)
• [ ] Data deletion process (timeline: 30 days?)
• [ ] Backup retention policy (how long backups kept?)

Access Controls

• [ ] MFA required for admin systems
• [ ] Least privilege access (employees only access data they need)
• [ ] Audit logs (who accessed what data, when?)
• [ ] Regular access reviews (quarterly? annually?)

Compliance

• [ ] DPA (if handling EU data)
• [ ] BAA (if handling healthcare data)
• [ ] Data residency options (which regions supported?)
• [ ] Right to audit (can you audit their controls?)

Subprocessors

• [ ] Subprocessor list (provided? regularly updated?)
• [ ] Approval process (can you approve/reject?)
• [ ] Audit rights (can you audit them?)

Incident Response

• [ ] Notification SLA (how fast do they tell you if breached?)
• [ ] Breach investigation (do they investigate? share findings?)
• [ ] Credit monitoring (do they offer if data breached?)

Example 2: Data Residency Requirements by Industry

Industry
Regulation
Requirements
Vendor Must Provide

Healthcare
HIPAA
Data in US (generally)
US data center, BAA

Finance
FINRA/SOX
Data in jurisdiction
US/EU data centers

EU Operations
GDPR
Data in EU
EU data center, DPA, deletion

China Operations
Data Localization Law
Data in China only
China data center (hard to find)

Government Contractor
FedRAMP
Special compliance
FedRAMP-certified cloud

Oil/Gas
Industry-specific
Varies
Varies (ask customer)

Anti-Patterns in Security Evaluation

Anti-Pattern 1: Ignoring Security Until Late Negotiations

You've picked the vendor, implemented a POC, and now legal asks about security. The vendor says "we can look into that" but won't commit.

Better approach: Make security a gate. If vendor can't meet your security requirements, eliminate them early.

Anti-Pattern 2: Assuming Vendor Compliance Means You're Compliant

Vendor is SOC 2 certified, so you're good, right? Not necessarily. Their SOC 2 might not cover your specific requirements.

Better approach: Verify vendor compliance with your specific requirements, not just generic certifications.

Anti-Pattern 3: Not Understanding Shared Responsibility

You think vendor is responsible for all security. Actually, you share responsibility. If your users' passwords are weak, that's your responsibility.

Better approach: Understand shared responsibility model. Vendor does some, you do some.

Anti-Pattern 4: Forgetting About Subprocessors

Vendor is secure, but they use 10 subprocessors you never knew about. One gets hacked, and your data is exposed.

Better approach: Get subprocessor list. Understand which subprocessors have data access. Evaluate their security.

Anti-Pattern 5: Not Defining Data Deletion Procedures

You want to delete data, but vendor says "we'll delete it eventually." It stays on backups for a year. Now you're in GDPR violation.

Better approach: Define deletion procedures upfront. How soon must data be deleted? What about backups? When are backups purged?

Examples: Vendor Security Assessment in Practice

Example 1: Assessing a Vendor for EU Financial Data

Company profile: European financial services company managing customer accounts and transactions. Must comply with GDPR and industry financial regulations.

Security requirements identified:

• GDPR DPA required (EU customer data)
• EU data residency required
• Encryption in transit and at rest
• Annual SOC 2 Type II audit
• Right to audit vendor

Vendor assessment:

1. Request SOC 2 report: ✓ Vendor has Type II certification
2. Request DPA: ✓ Available as standard
3. Data residency options: ✓ Frankfurt data center available
4. Encryption: ✓ AES-256 at rest, TLS 1.2+ in transit
5. Audit rights: ✓ Right to audit included

Decision: Proceed with vendor, but include in contract:

• DPA signed before deployment
• EU data residency in Frankfurt specified
• Incident notification within 12 hours
• Annual right to audit

Example 2: Assessing a Vendor for Healthcare Data

Organization profile: Healthcare provider managing patient data. Must comply with HIPAA.

Security requirements identified:

• HIPAA BAA required
• Encryption in transit and at rest
• Audit logging for all data access
• SOC 2 Type II or equivalent
• Data residency in US

Vendor assessment:

1. HIPAA compliance: ✗ Vendor says "coming soon"
2. BAA available: ✗ Not currently offered
3. SOC 2: ✓ Type II certification present
4. Encryption: Partial (at rest yes, in transit only in some regions)
5. Audit logging: ✓ Available

Decision: Do not use vendor for HIPAA data. Find alternative vendor that:

• Has current HIPAA compliance
• Offers BAA as standard
• Has encryption in transit and at rest in all required regions

Human Judgment Checkpoints for Vendor Security

Checkpoint 1: Does the vendor meet your security gate criteria?

Before investing time in POC or deep evaluation, confirm vendor meets minimum requirements. If not, eliminate them.

Checkpoint 2: Can you explain the vendor's security posture clearly?

If you can't explain their key security controls, certifications, and limitations in one page, you don't understand them well enough. Get clarity before signing.

Checkpoint 3: Are compliance agreements (DPA, BAA) included in the contract?

These must be signed before data goes to the vendor. Don't proceed without them if they're required.

Checkpoint 4: Do you have the right to audit?

If something goes wrong, you need the right to audit vendor controls. If they won't allow audit, that's a risk you should carefully consider.

Checkpoint 5: What's the incident response SLA?

If the vendor gets breached, how fast will they tell you? What will they do? Incident response SLAs should be in writing.

Key Takeaways

• 
  Make security and compliance a gate, not an afterthought. Eliminate vendors that can't meet your requirements before you invest time and resources in implementation. If a vendor doesn't meet your minimum security standards, no amount of great AI features makes them acceptable.
• 
  Know your regulatory requirements upfront. Different industries have different compliance needs. Healthcare has HIPAA. EU operations have GDPR. Finance has SOX. Know what applies to your organization before evaluating vendors.
• 
  Get compliance agreements signed before any data moves to the vendor. DPA (for GDPR), BAA (for HIPAA), and other compliance agreements should be included in your contract. Don't negotiate these after implementation starts.
• 
  Understand data handling practices thoroughly. Where is data physically stored? Who can access it? Is it encrypted in transit and at rest? Can you hold your own encryption keys? These are fundamental to your security posture.
• 
  Evaluate subprocessor and third-party risk. Vendors rarely process data alone. They use AWS, Google Cloud, backup providers, analytics services. Each is a potential risk. Get the list, understand it, and assess it.
• 
  Acknowledge shared responsibility. You're not delegating all security to the vendor. You share responsibility for secure implementation. The vendor secures their infrastructure; you secure your implementation, access controls, and data handling.
• 
  Document your security assessment. When you choose a vendor and approve their security posture, document your reasoning. What requirements did they meet? What limitations did you accept? Why did you make this choice? This documentation is critical for audits and future review.