Skill.re
AI-Powered Security Testing and Vulnerability Detection
Security is Everyone's Job, AI Makes It Easier
Security vulnerabilities hide in code. Developers miss them. Security audits happen annually and find months-old issues. By then, the damage is done. A single vulnerability can cost millions in breach response, legal liability, and customer loss.
Continuous security testing catches issues early. But doing it well requires expertise that not every team has. You need to understand OWASP Top 10, common attack patterns, cryptography, authentication flows, access control models. That's a lot to ask of everyone on the team.
This is where AI helps: identifying common vulnerability patterns, suggesting secure coding practices, and designing comprehensive security tests. More importantly, it democratizes security knowledge. Junior developers can write secure code with AI feedback. Security teams can focus on complex threats instead of reviewing basic code.
What AI is Good At
- Identifying common vulnerability patterns (SQL injection, XSS, broken auth, weak crypto)
- Scanning dependencies for known CVEs
- Suggesting secure coding alternatives with explanations
- Generating security test cases for attack scenarios
- Threat modeling and attack vector identification
- Security review of code and infrastructure configuration
- Explaining security concepts and best practices
- Detecting suspicious patterns (hardcoded secrets, weak hashing)
What it's not good at:
- Understanding your specific threat model and risk tolerance
- Making decisions about acceptable security risk
- Evaluating complex, multi-step attack chains
- Understanding business context of security decisions
- Discovering zero-day vulnerabilities (new, unknown exploits)
- Authenticating security scanning results (false positives are common)
You understand your threat model, your users, and your risk tolerance. The AI helps find vulnerabilities systematically and suggests improvements. Together, you build secure systems faster.
The Shift-Left Principle: Catch security issues during development, not in production. AI enables automated security testing in CI/CD pipelines. When developers get immediate security feedback, they learn and improve. Vulnerabilities are fixed before code reaches production.
Automated Security Scanning
Security scanning has four layers: static analysis (code review), dependency analysis (third-party libraries), secret detection (hardcoded credentials), and dynamic analysis (runtime behavior). AI helps with all four.
SAST (Static Application Security Testing)
SAST analyzes source code without running it. It catches common programming mistakes that lead to vulnerabilities.
"Review this code for security issues:
```python
def authenticate_user(username, password):
query = f\"SELECT * FROM users WHERE username = '{username}' AND password = '{password}'\"
result = db.execute(query)
return result is not None
@app.route('/api/user/')
def get_user(user_id):
user = db.query(f\"SELECT * FROM users WHERE id = {user_id}\")
return jsonify(user)
def hash_password(password):
import hashlib
return hashlib.md5(password).hexdigest()
API_KEY = 'sk-12345678901234567890123456789'
```
Look for: SQL injection, XSS, authentication bypass, insecure cryptography, hardcoded secrets, unsafe deserialization."
The AI identifies multiple issues: SQL injection in both queries (attacker can break out of the string), MD5 is cryptographically broken (use bcrypt), hardcoded API key (rotate it immediately), missing password hashing (plaintext comparison). It explains each issue and suggests secure alternatives.
Dependency Scanning
Your code depends on libraries. Those libraries have vulnerabilities. Tracking them manually is impossible, the AI does it automatically.
"Scan our dependencies for vulnerabilities:
requirements.txt:
```
django==3.2.0
requests==2.25.0
pillow==8.1.0
jinja2==2.11.0
```
Identify:
- Known CVEs in our dependencies
- Outdated packages with security patches
- Packages with moderate or high severity vulnerabilities
- Suggested safe versions
- Upgrade risk analysis"
The AI scans against CVE databases and identifies which packages need updating. It prioritizes by severity: a critical RCE vulnerability in a library you use is high priority; a low-severity issue in a rarely-used library is lower priority. It suggests safe upgrade paths.
Secret Detection
Hardcoded secrets are a common mistake. A developer adds an API key to code for testing, forgets to remove it, commits it. Now it's in version control history forever. The secret is compromised.
"Scan the codebase for accidentally committed secrets:
[scan all files]
Look for:
- AWS access keys (pattern: AKIA...)
- API keys and tokens
- Private keys (BEGIN RSA PRIVATE KEY)
- Passwords in connection strings
- Database credentials
- OAuth tokens
- Stripe/PayPal secrets"
The AI identifies secrets using pattern matching and entropy analysis. It finds obvious ones (API_KEY = "...") and subtle ones (credentials embedded in URLs or config objects). Each finding gets a severity rating and remediation advice.
Infrastructure as Code Scanning
Infrastructure configuration has security issues too: overly permissive security groups, unencrypted databases, exposed S3 buckets.
"Scan our Terraform/CloudFormation for security issues:
[infrastructure code]
Look for:
- Security groups allowing 0.0.0.0 (world-open)
- Unencrypted databases or storage
- Missing authentication requirements
- Overly permissive IAM roles
- Missing logging/monitoring
- Weak TLS versions"
The AI identifies misconfigurations before infrastructure is deployed. This catches problems in the planning stage, not after they're in production.
Security Testing
Scanning finds obvious issues. Testing verifies that security controls actually work.
Threat Modeling
Threat modeling is asking: "What are we protecting? Who wants to attack us? What are they trying to do? What can we do to stop them?" This frames all security work.
"Threat model our payment processing system:
Assets we're protecting:
- User payment data (card numbers, bank details)
- Transaction history (revealing user spending patterns)
- User personal information (name, address, email)
- System availability (if we're down, users lose money)
Threat actors:
- External attackers (want to steal data, commit fraud)
- Malicious insiders (have access, want to steal data)
- Competitors (want to understand our business)
- Script kiddies (running automated attacks)
- Organized crime (stealing payment data at scale)
Attack goals:
- Steal payment data (for fraud or sale)
- Modify transactions (send money to attacker)
- Expose user information (blackmail, identity theft)
- Disrupt service (competitive advantage, extortion)
- Escalate privileges (become admin, access everything)
For each threat/goal combination, what's the attack vector and what controls prevent it?"
The AI helps structure threat models and suggests relevant attack vectors based on architecture. It identifies which assets are highest value (payment data = high value, logs = low value) and which threats are most likely (external attackers = more likely than insiders).
Security Test Generation
Once you understand threats, you design tests that verify controls work.
"Generate security tests for user authentication:
Scenario 1: SQL Injection
- Attack: username = \"admin' --\" password = \"anything\"
- Expected result: Authentication fails (or succeeds but only for the attacking user, not admin)
- Test: Attempt login with injection payloads. Verify we don't gain unauthorized access.
Scenario 2: Brute Force
- Attack: Try 1000 password guesses per second
- Expected result: Account lockout after N failed attempts, or exponential backoff
- Test: Attempt 100 failed logins. Verify account lockout or rate limiting.
Scenario 3: Session Fixation
- Attack: Set user's session ID to a known value before they log in
- Expected result: Session ID changes after authentication
- Test: Set session ID, log in, verify session ID changed.
Scenario 4: CSRF (Cross-Site Request Forgery)
- Attack: User visits attacker's website which submits a request to your site using user's credentials
- Expected result: Request is rejected because CSRF token doesn't match
- Test: Submit request without CSRF token. Verify it's rejected.
Scenario 5: Unauthorized Access
- Attack: User tries to access another user's data
- Expected result: 403 Forbidden or 404 Not Found (don't leak whether resource exists)
- Test: User A tries to access User B's profile. Verify access is denied.
Scenario 6: Token Expiration
- Attack: Use an old, expired token
- Expected result: Request is rejected. User must re-authenticate.
- Test: Use token that expired yesterday. Verify request is rejected.
Generate test code that exercises all scenarios."
The AI generates comprehensive test code that exercises security controls. Each test is specific: it attacks a vulnerability, verifies the system rejects the attack, and measures response time (slow responses can leak information).
Secure Code Review
Not all code needs deep analysis, but critical code (auth, payments, data access) should be reviewed thoroughly.
"Review this code for security issues and suggest secure alternatives:
```python
def process_payment(user_id, amount):
# Get user from database
user = db.query(f\"SELECT * FROM users WHERE id = {user_id}\")
Call payment API
response = requests.post(
'https://payment-provider.com/charge',
json={'amount': amount, 'user_id': user_id},
verify=False # Ignore SSL verification
)
Store result
if response.status_code == 200:
db.execute(f\"INSERT INTO transactions VALUES ({user_id}, {amount}, 'success')\")
return True
else:
return False
```
Identify:
- Security issues (explain why)
- Secure alternatives (how to fix)
- Patterns (what should we do everywhere)
- Risk assessment (high/medium/low)"
The AI identifies multiple issues with explanations:
- SQL injection: User ID isn't sanitized, attacker can break query
2. SSL verification disabled: Enables man-in-the-middle attacks
3. No idempotency: If payment API succeeds but network fails, charging twice is possible
4. No logging: Can't audit payments or detect fraud
5. No input validation: What if amount is negative? 0? 999999999?
6. No error handling: Payment API error silently fails
For each, the AI suggests secure patterns and safe code.
Integrating Security into CI/CD
Scanning and testing are only valuable if they happen continuously. Manual security reviews happen quarterly. By then, issues are months old. Automated security in CI/CD catches issues immediately.
"Set up automated security testing for our CI/CD pipeline:
On every pull request:
1. Run SAST: Scan code for common vulnerabilities
- Fail the PR if critical issues found
- Warn on medium/low severity
2. Run dependency scan: Check for CVEs in libraries
- Fail on high/critical CVEs
- Warn on medium
3. Run secret detection: Look for hardcoded credentials
- Fail if secrets found
- Require rotation + removal
4. Run custom security tests
- Authentication tests
- Authorization tests
- Input validation tests
5. Generate security report and attach to PR
On merge to main (deployment):
1. Run all above checks again (harder failures)
2. Run penetration tests (specific to deployment)
3. Verify no secrets in deployment artifacts
4. Verify security headers are configured
5. Verify logging/monitoring for security events
Post-deployment:
1. Continuous dependency scanning (new CVEs discovered daily)
2. Runtime monitoring for suspicious behavior
3. Weekly security summary emailed to team"
The AI helps design this pipeline and generates the testing code. It becomes part of your development workflow, not a separate process.
Security Shift-Left Culture: Developers who get immediate security feedback learn. They improve their code. They build security intuition. Over time, your team naturally writes more secure code and needs less explicit security testing.
Key Insight
Security is a continuous process, not an annual event. Automated scanning catches obvious issues. AI-assisted security testing lets you design comprehensive threat models and verify that security controls actually work.
Understanding False Positives
Security scanners aren't perfect. They find real vulnerabilities and they produce false alarms (false positives). A false positive is code the scanner flags as vulnerable, but it's actually safe.
Example: The scanner flags any use of md5() as insecure (true for passwords, false for generating cache keys). You need to review findings and mark obvious false positives. Otherwise, you'll get alarm fatigue and ignore real vulnerabilities.
"We ran SAST on our codebase and got 47 findings:
- 3 SQL injections (real, need fixing)
- 12 hardcoded secrets (real, need rotation)
- 5 weak crypto (real, use bcrypt not MD5)
- 18 low-severity findings (review each, many might be false positives)
- 9 findings in test code (false positives, test code doesn't go to production)
Review each finding and classify:
- True positive: Real vulnerability, needs fixing
- False positive: Scanner flagged it incorrectly, can suppress
- Benign: Safe in context, but document why
Fix true positives. Suppress false positives with comments explaining why."
The AI helps you triage findings and decide which are real threats.
What to Do Monday Morning
- Scan your codebase for vulnerabilities right now. Ask the AI to review your code using SAST. Look for the most critical issues (SQL injection, hardcoded secrets, weak auth).
- Check your dependencies for known CVEs. Run a dependency scanner. Update or patch any packages with high/critical severity vulnerabilities immediately.
- Search for hardcoded secrets in your repo. Scan git history. Rotate any API keys or credentials you find. Remove them from code.
- Threat model your most critical service. Ask the AI to help structure a threat model. Identify highest-value assets and most likely attacks.
- Integrate security scanning into CI/CD. Make it automatic on every PR. Start with SAST and secret detection. Expand to dependency scanning and custom security tests.
FAQ
Q: Can AI find all vulnerabilities?
A: No. It finds common, well-known patterns well (injection, XSS, weak crypto, hardcoded secrets). It misses complex or novel vulnerabilities that require understanding business logic or attack chains. Use AI as a tool to catch obvious issues. Combine with professional penetration testing for deeper analysis.
Q: Should we do regular penetration testing?
A: Yes. Annual or quarterly professional penetration tests are essential. Automated scanning catches obvious issues. Pen testers find the clever ones (attack chains, social engineering, business logic bypasses). Both are needed: automation for continuous checking, professionals for deep analysis.
Q: How do we handle false positives?
A: Review each finding and classify it. Mark obvious false positives with comments explaining why they're safe. Keep a ratio of true positives to total findings high (ideally >70% true positives). If false positive rate is high, tune the scanner or add custom rules to reduce noise.
Q: What about zero-day vulnerabilities?
A: AI can't find zero-days (vulnerabilities nobody knows about). But you can reduce risk by: keeping dependencies updated, following security best practices, having good monitoring/logging to detect attacks, and practicing incident response. Defense in depth helps.
Q: When should security testing block deployment?
A: Critical vulnerabilities (SQL injection, hardcoded secrets, authentication bypass) should block deployment immediately. High-severity issues (weak crypto, missing authorization checks) should be reviewed and fixed before deployment. Medium/low severity issues should be tracked but don't necessarily block shipping.