Hiring managers for operator-builder roles in 2026 will not read your Coursera certificates, your LinkedIn skill endorsements, or your bootcamp credentials. They look at four artifacts and only four. (1) A deployed agent with a URL the interviewer can hit. (2) A documented eval set of 50+ cases with scores. (3) A governance write-up containing a Blast Radius classification and a Fundamental Rights Impact Assessment stub. (4) A postmortem of a real incident with a real root cause and a real remediation. If you have all four, you are in the top decile of applicants. If you have three, you get to the panel. If you have two, you get a screen. Zero or one and your resume goes in the pile with the bootcamp grads.

Why the Portfolio Replaced the Resume

Two structural shifts happened between 2023 and 2026. First, agent-builder skills became too verifiable to hide behind credentials. A hiring manager can read your eval set in fifteen minutes and know whether you understand the work. They cannot read a Coursera certificate and know anything similar. Second, agent-builder failures became too expensive to hire blind. A wrong hire at a $160K-$200K total comp level who deploys a bad agent into a production environment costs the company $300K-$500K in remediation, not counting the incident's direct cost. Hiring managers will not take that risk on a resume alone.

The shift is comprehensive. In April 2026, the YC Work at a Startup board's top-50 operator-builder JDs were scraped by a hiring analytics firm. Seventy-three percent of the JDs listed "deployed-agent portfolio" as a required or strongly preferred qualification. Forty-one percent listed "documented eval set" explicitly. Zero JDs mentioned Coursera or Udemy certificates as relevant signal. The market has spoken.

The portfolio is not a bonus. It is the qualification. The certificate is not a tiebreaker. It is decoration. The resume is the introduction to the portfolio, not a substitute.

Artifact 1: A Deployed Agent (URL + Interviewer Access)

The first and most important artifact is a live, deployed agent that the interviewer can use. Not a screenshot. Not a Loom video. A URL. With credentials. That works when they click it.

What counts as deployed

"Deployed" means the agent is running on real infrastructure, accessible by HTTP request, and not just a notebook in your local environment. The agent's tooling actually touches a real system. The kinds of agents that count:

• A Lindy or n8n workflow accessible via a webhook URL that processes a real input and produces a real output.
• A custom-built agent on Vercel, Render, or Cloudflare Workers, with a public URL.
• A Claude or OpenAI Assistants API integration with a small Next.js or Streamlit frontend.
• A Slack bot installed in a public Slack workspace the interviewer can join.
• An MCP server with a hosted endpoint the interviewer can connect to.

What does not count: a GitHub repo with no deployment, a Postman collection, a Notion doc describing what you'd build, a YouTube video, a Figma mock. Hiring managers in 2026 have learned to distinguish "I could build it" from "I built it" by requiring URL access.

What the agent should do

Three signals separate a strong portfolio agent from a weak one:

1. It solves a real problem. Not a toy "tell me a joke" agent. A real workflow: lead enrichment from a CRM ID, contract redlining from a PDF, customer health scoring from product usage data, support ticket triage from inbound text, code-review summarization from a GitHub PR URL, expense categorization from a receipt image. Real problems have edge cases.
2. It uses at least three tools. A single-LLM-call wrapper is a chatbot, not an agent. The portfolio agent should chain at least three tool calls (search-API, knowledge-store lookup, write-to-system) and demonstrate that you understand tool orchestration.
3. It has visible structure. The output should not be a wall of text. It should show the steps it took, the tools it called, and the data it referenced. This is observability at the user-facing layer and signals that you think about traceability.

Three real examples

The following are anonymized portfolios from candidates who landed GTM Engineer and Agent Operations Lead roles in Q1 2026:

Example A: The renewal-risk scorer

A candidate built a deployed agent at renewal-risk.fly.dev that takes a customer's HubSpot deal ID as input, pulls the contact, the deal stage, the last 30 days of product-usage signals from a public Mixpanel dashboard, and the last three support tickets, then produces a renewal-risk score (0-100), a one-paragraph rationale, and a recommended action. Reviewers spent twenty minutes playing with it. The candidate got six interviews from three job applications.

Example B: The PDF redliner

A candidate built a Streamlit app that accepts a PDF contract upload and an organization's standard playbook. The agent reads the contract, identifies clauses that deviate from the playbook, produces a redlined PDF with comments, and sends a Slack summary. The deployment cost about $40/month in OpenAI tokens. The candidate documented the cost openly. They were hired as an AI Solutions Architect at a legaltech firm.

Example C: The MCP server

A candidate built and published an MCP server for an internal API at their previous employer (with permission). The server exposes five tools, has a hosted endpoint at api.candidate-portfolio.com/mcp, and documentation. Reviewers connected it to Claude Desktop in real-time during the panel interview. The candidate was hired as an Agent Operations Lead at a developer-tools company.

The interviewer-access requirement

The biggest mistake candidates make: building a portfolio agent that requires their own credentials to run. The agent uses the candidate's personal OpenAI key, their personal HubSpot token, their personal Stripe account. When the interviewer clicks the URL, the agent works for forty seconds, then hits a rate limit or auth error. The signal is destroyed.

The fix: use a separate, throttled, monitored portfolio account with its own keys. Set a daily cost cap of $10-$20. Use observability so you can see when an interviewer is using the agent. Plan the portfolio environment as carefully as you would plan a production deployment, because it is one.

Artifact 2: The Documented Eval Set (50+ Cases, Scored)

The second artifact separates serious candidates from the rest. A portfolio agent without an eval set is impressive demo work. A portfolio agent with an eval set is production thinking.

What 50+ cases actually looks like

The eval set is a structured file (JSON, CSV, or a Google Sheet) with at minimum:

• Input: the exact prompt or input data passed to the agent.
• Expected output: the human-reviewed correct answer, or a category, or a structured object.
• Tags: what kind of case is this — happy path, edge case, adversarial, ambiguous.
• Most-recent score: did the agent get this right on the last run.
• Notes: for failed cases, why did it fail and what did you do about it.

Fifty cases is the floor. Hiring managers report that anything below 30 cases reads as a homework assignment; 50 to 100 reads as a serious effort; 200+ reads as production-grade. Most strong portfolio eval sets are between 60 and 120 cases.

The composition that impresses

Hiring managers look for a specific distribution:

• 50-60% happy-path cases (the agent should clearly succeed)
• 20-25% edge cases (typos, missing fields, ambiguous inputs)
• 10-15% adversarial cases (prompt injection attempts, attempts to extract system prompts, attempts to make the agent take unauthorized actions)
• 5-10% "should escalate" cases (the correct answer is "I cannot do this; route to a human")

An eval set that is 100% happy-path is a vibe check. An eval set that includes adversarial cases shows you have thought about production failure modes. An eval set that includes "should escalate" cases shows you understand HITL design.

How to score the eval set

Three scoring methods, in increasing rigor:

1. Exact match: the agent's output exactly matches the expected output. Works for structured outputs (categories, IDs, JSON shapes). Brittle for natural language.
2. Rubric scoring: a checklist of criteria the output should satisfy. "Mentions the customer's company name. Cites the right product version. Avoids hallucinated dates. Recommends a specific next action." Easier for natural-language tasks; can be human-scored or LLM-judged.
3. LLM-as-judge: a separate LLM (often a stronger model) scores the agent's output against the expected output. Reproducible, fast, but requires careful prompt design to avoid the judge's own biases.

A strong portfolio includes the score over time. "We added a memory layer in week 4. Eval set accuracy went from 71% to 84%. Adversarial robustness went from 60% to 78%." This is the language of someone who has done the work.

Tools named for eval sets in 2026

You do not need a fancy tool to do this, but reviewers will recognize the names: Braintrust, Promptfoo, LangSmith eval, OpenAI Evals, Inspect AI (Anthropic), Helicone evals, Langfuse evals, Phoenix (Arize). Pick one. Show that you have used it. Bonus: pick two and explain why you picked the one for this portfolio piece.

Artifact 3: The Governance Write-Up (Blast Radius + FRIA Stub)

The third artifact is the one most candidates skip. It is a 1-3 page document that names the agent's governance posture — what could go wrong, what authority it has, and what would happen in the worst case. It signals that you think about deployed agents the way a production engineer thinks about production systems.

Blast Radius classification

Blast Radius is a framework that classifies an agent's actions by reversibility and externality:

• Internal-only, reversible: updating a status in your own Airtable, writing to your own Notion. Worst-case is internal cleanup.
• Internal-only, hard-to-reverse: writing to a shared CRM that other reports depend on. Worst-case is data confusion across the team.
• External, reversible: sending a Slack message in a customer channel. Worst-case is embarrassment plus correction message.
• External, hard-to-reverse: sending an email to a customer. Worst-case is a public incident, customer churn, and a corrective email.
• External, irreversible: issuing a refund, closing a contract, posting to public social media. Worst-case is financial loss, legal exposure, or brand damage.

For your portfolio agent, the write-up should list every action the agent can take and classify it. If the agent only does Internal-only reversible actions, say so. If it does anything external, document the HITL gate (or the lack of one and why).

Fundamental Rights Impact Assessment (FRIA) stub

From August 2, 2026, EU AI Act Article 27 requires deployers of certain high-risk AI systems to conduct a Fundamental Rights Impact Assessment. Even if your portfolio agent is not high-risk under the Act, including a FRIA stub in your governance doc signals to hiring managers that you have read the regulation and know how to think about it.

A FRIA stub is a one-page section that addresses, briefly:

1. Who is affected by this agent's outputs? (Customers, employees, prospects, the public?)
2. What rights could be impacted? (Privacy, non-discrimination, access to remedy, fair treatment, freedom of expression?)
3. What is the worst-case impact and how is it mitigated?
4. What human oversight is in place?
5. How is the agent's identity logged and auditable?

For a customer-support triage agent: "Affected: customers. Rights: access to remedy, non-discrimination. Worst case: legitimate complaint misrouted to a low-priority queue, delaying response. Mitigation: 90-day audit of routing decisions, sample-rate review by CS lead, escalation if confidence below 0.7. Oversight: every triage decision logged with model version, confidence, and routing target; CS lead reviews daily."

This is not an aspirational document. It is a one-page artifact that signals: "I have read Article 27 and I can do the work."

What strong governance write-ups include

• The Blast Radius classification table with every action listed.
• The identity model: whose authority does the agent act under, where is it logged.
• The kill switch: how to stop the agent and the most recent test of it.
• The cost ceiling: per-run cap, daily aggregate, alert thresholds.
• The HITL gates: which actions are gated, what the response SLA is.
• The FRIA stub if relevant.
• A "known limitations" section: what the agent does poorly and why.

Artifact 4: The Postmortem (Real Incident, Real Root Cause, Real Remediation)

The fourth artifact is the one that separates the candidate who has actually shipped from the candidate who has only built. A postmortem documents a real production incident your agent had — or a near-miss in your testing process — and traces the root cause, the response, and the remediation.

Why this artifact matters most

Hiring managers know that every agent fails. The relevant signal is not "this candidate's agents never fail." The signal is "this candidate has seen a failure, diagnosed it competently, and improved the system." That signal is gold because most candidates have never had a failure visible enough to write up — they have not yet shipped at the level where things can go wrong.

If you have a real postmortem, even from a small failure, you are demonstrating that you have crossed the operational threshold. That alone moves your offer.

What the postmortem includes

The standard SRE postmortem template, applied to agents:

1. Summary: what happened, when, who was affected, severity.
2. Timeline: minute-by-minute, what the agent did, what humans noticed and when.
3. Root cause: the specific reason it failed. Not "the LLM hallucinated" but "the prompt did not constrain the agent to the user's tenant, so it queried records from another tenant."
4. Contributing factors: what made the failure possible or worse. Missing eval coverage, missing observability, missing HITL gate, missing cost cap.
5. Impact: how many records affected, dollar cost, customer-facing impact, regulatory exposure.
6. Response: what was done in the moment to stop and contain the damage.
7. Remediation: what changed afterward. New eval cases added, new HITL gate, new alert threshold, new identity scoping.
8. What we'd do differently: the lessons that generalize beyond this specific incident.

Three real portfolio postmortems

Postmortem A: The cost overrun

"On March 14, our lead-enrichment agent looped 47 times on a single input because the customer's company name was ambiguous and the agent kept retrying disambiguation queries. The single run cost $87. Aggregated across 18 affected runs that day, the cost was $1,560. Root cause: no per-run cost ceiling and no loop-iteration cap. Response: rate-limited the disambiguation tool, capped iterations at 7. Remediation: added a per-run $5 cost cap enforced at the runtime layer, added Helicone alerts on per-run cost spikes, added 8 eval cases covering ambiguous company names. What we'd do differently: load-test cost caps before scale-out, not after the incident."

Postmortem B: The wrong-tenant query

"On February 8, our customer-success agent returned product-usage data from Customer A to a user authenticated as Customer B. The user was a senior CS rep at a small SaaS who reported the bug within four hours. Root cause: the agent's tool layer did not enforce tenant scoping; the tool function accepted a customer_id argument and queried without checking that the caller had access. Contributing factors: no integration test for cross-tenant access, no eval case covering it. Impact: one customer's product-usage summary leaked; no PII exposed. Response: disabled the agent, audited logs for similar incidents (found zero), notified the affected customer. Remediation: rewrote the tool to enforce caller-tenant matching, added 12 cross-tenant eval cases, added principal logging to every tool call. What we'd do differently: assume cross-tenant exposure is the default failure mode and prove the opposite in eval before launch."

Postmortem C: The hallucinated SLA

"On January 23, our support-triage agent told a customer their SLA included 24/7 phone support. The customer's actual SLA included business-hours email support only. The customer escalated; account management resolved with a partial credit. Root cause: the agent had access to a stale snapshot of SLA documents in its knowledge store; the SLA had been re-negotiated 60 days prior. Contributing factors: no freshness check on knowledge-store documents, no escalation path when the agent's confidence in SLA details was low. Impact: $400 customer credit, two hours of CS team time, one tense email thread. Response: corrected the SLA document, sent the customer an apology with the corrected info, audited 60 days of similar responses (found two more). Remediation: added 'document_last_updated' to every knowledge-store entry, added an eval case for SLA-related questions, lowered confidence threshold for SLA-claim responses. What we'd do differently: treat freshness as a first-class metadata field, not an afterthought."

If you have not yet had an incident

If you genuinely have not had a real incident yet, you have two options. The first is to engineer a near-miss in your testing. Run your agent on a deliberately adversarial input set, find the most embarrassing failure, document it as if it were a near-miss in production, and write the postmortem. Most hiring managers will accept this if you label it accurately.

The second option is to write up a postmortem of a public incident from another company — with attribution — and explain what you would have done differently. This is the weakest of the four artifacts because it is one degree removed from your own work, but it is better than not having a postmortem at all.

What Hiring Managers Do Not Care About in 2026

Five things that operator-builder candidates spend time on that do not move offers:

• Coursera, Udemy, edX certificates. Zero signal. Hiring managers do not read them.
• Bootcamp credentials. Slightly negative signal in some hiring rooms because they correlate with "I learned the demo, not the production work."
• LinkedIn skill endorsements. Zero signal; everyone endorses everyone.
• "Used GPT-4 / Claude" listed as a skill. Zero signal; every applicant claims this.
• Generic tech blog posts. Mild positive signal only if the blog post is technical, specific, and recent. "10 ChatGPT prompts for productivity" is not signal.

What does move offers: the four artifacts. A deployed agent. A documented eval set. A governance write-up. A postmortem.

How to Build the Portfolio in 90 Days

The next lesson covers this in detail, but in summary:

1. Days 1-30: ship the deployed agent. Pick one workflow platform (Lindy or n8n if non-coding; Vercel + OpenAI Assistants if coding). Build one agent that solves one real problem.
2. Days 31-60: build the eval set. Write 50-100 cases. Score them. Run them on every change. Add adversarial and edge cases.
3. Days 61-75: write the governance doc. Classify Blast Radius. Write the FRIA stub. Document HITL gates, identity model, cost caps.
4. Days 76-90: create your postmortem. Either from a real incident or an engineered near-miss. Write it up using the SRE template.

By day 90 you have all four artifacts. By day 100 you have a portfolio site — a single landing page that links to the four artifacts with two-paragraph context each. Reviewers spend twenty minutes on the site, get a deep impression of your competence, and you move from "screen" to "panel" with three out of four hiring managers.

The Portfolio Site Itself

The portfolio site is not optional in 2026. It is the index page reviewers visit. Structure that works:

• Hero section: your name, one-line positioning ("Agent Operations Lead, 18 months production experience, building eval-set-first agents"), and links to the four artifacts.
• Artifact 1 (Deployed Agent): URL, what it does, three-paragraph explanation of the problem, the architecture, the cost.
• Artifact 2 (Eval Set): link to the eval set file, the most recent score, a chart of score over time.
• Artifact 3 (Governance): the 1-3 page write-up as a PDF or hosted page.
• Artifact 4 (Postmortem): the postmortem document.
• About: brief bio, named tools you use, work history (optional, can be on LinkedIn instead).

Hosting: Vercel, Netlify, GitHub Pages, or even a single-page Notion site. The medium does not matter; the content does.

Signals That Mark a Top-Decile Portfolio

Top-decile portfolios — the ones that get four interviews from four applications — share specific patterns:

• The agent is hosted on real infrastructure (Vercel, Render, Cloudflare) with a custom domain.
• The eval set has more than 100 cases and shows score progression over weeks.
• The governance write-up cites EU AI Act articles by number, not just by name.
• The postmortem includes a contributing-factors section, not just root cause.
• The site links to the candidate's GitHub with at least one repo of production-style Python or TypeScript.
• The eval set includes a section on the candidate's own scoring methodology and its biases.
• The candidate names specific tools they have used: LangSmith, Helicone, Braintrust, Promptfoo, Inspect AI, Phoenix.
• The candidate documents cost per run for the deployed agent and discusses cost-control tradeoffs.

The Interview Itself, After the Portfolio

The portfolio's job is to get you to the interview. Once you are there, the conversation shifts. Hiring managers will:

1. Ask you to walk them through the deployed agent live (10-15 minutes).
2. Ask you to explain a specific failure case from the eval set and what you did about it.
3. Ask you to describe a real on-call incident or scaling problem (if you have one).
4. Ask you what you would do differently with the deployed agent if you rebuilt it today.
5. Test you on technical depth: Python or SQL coding for GTM Engineer/Agent Ops Lead roles; workflow architecture for AI Solutions Architect/AIRops; cross-functional scenarios for AI Product Ops.

The portfolio carries the first half of the interview. The second half is the conversation. Both halves matter; the portfolio decides whether the second half happens.

Key Takeaways

• Four artifacts replace the resume in 2026 operator-builder hiring: a deployed agent with interviewer URL access, a documented 50+ case eval set with scores, a governance write-up with Blast Radius + FRIA stub, and a postmortem of a real incident.
• Coursera certificates, bootcamp credentials, LinkedIn endorsements, and "Used GPT-4" skill listings produce zero signal in operator-builder hiring. Seventy-three percent of top YC-board JDs in April 2026 required a deployed-agent portfolio; zero mentioned certificate credentials.
• The deployed agent must be hosted and accessible on real infrastructure with the interviewer's own access — not a screenshot, not a Loom, not a notebook. Three signals separate strong from weak: solves a real problem, uses 3+ tools, has visible structure in output.
• The eval set should be 50-120 cases with a specific composition: 50-60% happy path, 20-25% edge cases, 10-15% adversarial, 5-10% should-escalate. Score with exact match, rubric, or LLM-as-judge. Show score progression over time.
• The governance write-up classifies Blast Radius across all the agent's actions (internal-reversible to external-irreversible), includes a FRIA stub addressing affected parties and rights impacts, and documents identity model, HITL gates, cost ceilings, and known limitations.
• The postmortem is the artifact most candidates skip and the one that most reliably moves an offer. It demonstrates you have crossed the operational threshold — you have seen a failure, diagnosed it, and improved the system.
• Named tools matter: LangSmith, Helicone, Langfuse, Arize, Braintrust, Promptfoo, Inspect AI, OpenAI Evals, Phoenix. Pick a few, use them, name them in your portfolio. Reviewers verify by recognition.
• Build the portfolio in 90 days: days 1-30 deploy the agent, days 31-60 build the eval set, days 61-75 write the governance doc, days 76-90 create the postmortem. Day 100 host the portfolio site linking all four.
• Top-decile portfolios share specific patterns: custom domain hosting, 100+ eval cases, EU AI Act citations by article number, contributing-factors sections in postmortems, and named-tool literacy that reviewers verify.
• The portfolio's job is to get you to the interview; the interview converts. Hiring managers spend the first half walking through your portfolio live, the second half on technical depth. The four artifacts decide whether the second half happens.