Helicone occupies a category of one in 2026: agent observability you can install in five minutes by changing a base URL. No SDK to import. No decorator to wrap. No instrumentation library to configure. You point your OpenAI or Anthropic client at oai.helicone.ai/v1 (or the Anthropic equivalent), add an auth header, and every prompt, response, latency reading, and token count flows into the Helicone dashboard automatically. This lesson covers the proxy pattern in detail — what it gives you, what it doesn't, and when the trade-off of "no tool-call detail, no multi-step trace tree" is the right call (often) versus when you need the deeper SDK-level instrumentation we covered in Lesson 1 (also often).

The Proxy Pattern, Explained

The architecture is simple enough to fit in a paragraph. Helicone sits between your application code and the LLM provider's API. Your app calls Helicone; Helicone records the call, forwards it to OpenAI (or Anthropic, or any compatible endpoint), receives the response, records the response, and returns it to your app. End-to-end latency overhead is small — a few milliseconds in the same region, low double digits across regions — and the application code never knows the difference, because the Helicone endpoint speaks the OpenAI API protocol verbatim.

The five-minute install looks like this in Python:

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    api_key=OPENAI_API_KEY,
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    },
)

That's it. Every client.chat.completions.create() call from this client is now logged in Helicone. You log into your Helicone dashboard, see the request, see the response, see the latency, see the cost, see the user metadata you tagged. Five minutes. No code changes anywhere else in your application.

What you get out of the box

Proxy-level capture covers a surprising amount of ground:

• Full request and response logging. Every prompt, every completion, every system message. You can scroll through the last 1,000 calls, search by content, filter by model.
• Latency per call. Time-to-first-token (TTFT) and total duration. Plotted over time. Sliced by model.
• Token and cost tracking. Input tokens, output tokens, cost in USD computed against current pricing. Aggregated by user, by feature, by app.
• Caching. Helicone can transparently cache identical requests for a configurable TTL. The cache hit rate becomes a metric you watch; reducing redundant calls drops your monthly LLM bill noticeably.
• Rate limiting and key management. Per-user and per-tenant rate limits at the proxy level, without your application code knowing about it.
• User feedback capture. Attach a thumbs-up/thumbs-down via a separate API call referencing the request ID; thumbs-down requests are surfaced in the dashboard for review.

What you don't get

The proxy sees only what the proxy sees: LLM API calls. It doesn't see tool calls (unless those tools happen to themselves make LLM API calls through the same proxy). It doesn't see retrieval steps (unless retrieval involves an LLM call for embedding or reranking). It doesn't see the parent agent loop. It doesn't know that this call is step 4 of a 12-step run, or that step 4 followed from step 3's output. The trace tree we read in Lesson 1 doesn't exist at the proxy layer — only the leaf nodes are visible.

This is the central trade-off of the proxy pattern: you trade tool-call and multi-step detail for zero-code install. Whether that trade is right for you depends on what you're building.

When the Proxy Pattern Is the Right Call

Three scenarios where Helicone-as-proxy is the right first choice in 2026:

You're shipping fast and want observability today, not next sprint

The agent is in customer trial. Something's slower than expected, costs are higher than expected, and you suspect prompt issues — but you have no observability of any kind. The right answer is: ship Helicone in the next 30 minutes, look at the dashboard in the next 24 hours, and use whatever you learn to make the next call (often: "we need deeper tracing too; let's add LangSmith"). The five-minute install is a feature, not a compromise, when the alternative is shipping no observability for two more weeks.

The agent is shallow, not deep

Some agents are essentially "one LLM call with retrieval." Customer-support classification: receive an email, classify into a category, return. Even with retrieval, the retrieval is a single vector-DB lookup and the LLM call is the meaningful unit. For these agents, proxy-level observability captures most of what matters. The "tool call detail" we lose is one tool call, not 14.

The team isn't going to write decorators

You've shipped Helicone to three teammates and the other six on the team aren't going to wrap their LLM calls in @observe() decorators no matter how nicely you ask. They might forget. They might not bother. The proxy pattern bypasses that problem entirely — they don't need to know it's there. Set the base URL in the shared client wrapper, ship, done. For organizations with sprawling LLM usage across many product teams, this is often the only realistic way to get any observability coverage.

When the Proxy Pattern Isn't Enough

Three scenarios where you'll outgrow Helicone-only:

The agent is deeply multi-step

A research agent that issues 30 LLM calls and 50 tool calls per run, with branching decisions and retries, is opaque at the proxy layer. You see 30 individual LLM calls but no causal chain between them. You can't answer "which step produced this bad output?" because the proxy doesn't know there are steps. The right answer here is SDK-level tracing (LangSmith, Langfuse, Phoenix), with Helicone optionally alongside for the cost/latency-per-call view it does well.

You need tool-call evaluation

"How often does the agent call the wrong tool?" "How often does the agent retry the same tool after an error?" These are questions about behavior between LLM calls — they're invisible to the proxy. You need SDK-level instrumentation that sees the tool router, the tool dispatcher, and the agent's loop.

The compliance regime requires it

Some compliance regimes (healthcare, finance, EU government) require the LLM proxy to run inside your VPC. Helicone supports self-hosting (it's open-source under MIT-compatible license), and the self-hosted version is comparable to the SaaS in capability, but the lift is bigger than the cloud install. Plan for it accordingly.

The Five-Minute Install, Walked Through

Here's exactly what shipping Helicone looks like, end-to-end, for a Python agent already running in production.

Step 1: Sign up and get an API key

Visit helicone.ai, sign up (free tier covers up to 100,000 requests per month as of May 2026), generate an API key. Save it to your environment variable store as HELICONE_API_KEY.

Step 2: Change the base URL

Find every place you instantiate an OpenAI() client (or Anthropic(), or AzureOpenAI(), or Together()). Helicone supports all the major endpoints. Add a base_url argument pointing to the relevant Helicone endpoint, and a default_headers argument with the auth bearer:

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    api_key=OPENAI_API_KEY,
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    },
)

For Anthropic, the equivalent:

client = Anthropic(
    base_url="https://anthropic.helicone.ai",
    api_key=ANTHROPIC_API_KEY,
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    },
)

Step 3: Add metadata headers (optional but recommended)

This is the difference between a Helicone dashboard that's interesting and one that's actionable. Add headers identifying the user, the session, the feature, the tenant:

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[...],
    extra_headers={
        "Helicone-User-Id": user_id,
        "Helicone-Session-Id": session_id,
        "Helicone-Property-Feature": "customer-support-triage",
        "Helicone-Property-Tenant": tenant_id,
        "Helicone-Property-Agent-Version": "v4.2",
    },
)

Now your dashboard can slice by user, by feature, by tenant, by agent version. The filtering UX in Helicone is fast and clean once metadata is attached.

Step 4: Deploy, watch the dashboard, find the first surprise

Ship the change. Within minutes, requests show up in the dashboard. Look at:

• Total requests per hour and per day. Is it what you thought?
• p50/p95/p99 latency. Are tail latencies what you thought?
• Cost per day. Is the bill what you thought?
• Top users by request count. Any unexpected heavy users?
• Top errors. Is there a class of failure you didn't know about?

Almost every team we've watched ship Helicone-as-first-observability finds something they didn't know in the first 24 hours. A heavy user who's calling the API every 200ms. A retry loop that triples cost on a specific failure path. A model fallback path that's silently using the expensive frontier model when the cheap one is overloaded.

Cost Savings via Caching

Helicone's transparent caching is a feature worth understanding because it pays back the install cost on its own. The proxy can be configured to cache identical request bodies for a configurable TTL. When the cache hits, no LLM call is made — Helicone returns the cached response and you save the cost and the latency.

When caching helps

Caching matters most for agents with deterministic prompts and many duplicate inputs. Customer-support classification sees the same kinds of emails over and over. Document summarization sees the same documents through retries. Synthetic data generation pipelines hit the same prompts repeatedly. Any workload with high prompt redundancy can see cache hit rates of 20-40%, translating to comparable percentage cost savings.

When caching doesn't help (and can hurt)

Conversational agents with long, unique multi-turn histories rarely hit the cache. The full message history is part of the cache key, and every turn adds a new message. Cache hit rate on production conversational agents is often under 5%. Worse: caching responses that need to reflect real-time context (current inventory, current weather, current account state) returns stale data. The TTL matters. If your prompts include "today's date" or "current price," set the cache TTL short or skip the caching feature on those endpoints.

Real numbers

A 12-person ed-tech team running an essay-feedback agent enabled Helicone caching with a 24-hour TTL in February 2026. Their cache hit rate stabilized at 34% within two weeks (students re-submit essays for feedback iteratively; the same essay flows through the same prompt). Their monthly OpenAI bill dropped from $2,400 to $1,580 in March — 34% saved, almost exactly tracking cache hit rate. The Helicone subscription at that scale was $20/month. Net savings: $800/month.

Rate Limiting and Key Management

Beyond observability and caching, Helicone provides operational capabilities that often matter more in production than the dashboard itself.

Per-user rate limits at the proxy

You can configure rate limits per Helicone-User-Id at the proxy level. A user generating 1,000 requests per minute (probably an abuse case or a bug) is throttled at the proxy without your application code needing to track or enforce. This is particularly useful for teams shipping LLM features in user-facing products where abuse and bug-driven floods are real concerns.

Virtual keys

Helicone can mint "virtual keys" that proxy to your real provider key but with their own scope, rate limit, and budget. You can give different teams or different microservices different virtual keys with different limits, without distributing your real OpenAI key. When a virtual key is compromised or misused, you rotate that one without rotating your real provider credentials.

Provider failover

Helicone Vault supports configuring a fallback provider for a request. If OpenAI returns an error or is overloaded, Helicone can transparently route to Anthropic (with appropriate model mapping) or to a self-hosted endpoint. The application code doesn't see the failover; it just sees the request succeed. This pattern was significantly more common in 2024-2025 when provider outages were more frequent; by 2026 it's less urgent but still useful for high-availability requirements.

Proxy Plus SDK Tracing: The Best of Both

The most common 2026 pattern at teams with mature observability isn't proxy-only or SDK-only. It's both, used for different jobs:

• Helicone (proxy) for cost monitoring, per-user rate limiting, transparent caching, and the always-on view of "what's our LLM spend doing today." The proxy captures everything that goes through any LLM client, including parts of the system not yet instrumented with SDK tracing.
• LangSmith / Langfuse / Phoenix (SDK) for the trace-tree debugging we covered in Lesson 1. The SDK sees tool calls, retrieval steps, and the parent-child structure of multi-step agents.

The two don't conflict. A single LLM call ends up logged in both places — once in Helicone (with cost and latency) and once in LangSmith (as part of its parent trace). Cross-referencing is done via a shared request ID you pass through both systems' metadata.

The hand-off pattern

One concrete way teams wire this: every agent run generates a UUID at start. That UUID is set as Helicone-Session-Id on every LLM call from the run, and as the trace name (or root-span name) in the SDK-level tracer. When you find a bad run in LangSmith (via trace inspection), you can also pull up its cost profile in Helicone by searching for the session ID. When you find a cost outlier in Helicone, you can pull up its trace tree in LangSmith by the same ID.

A Real Shipping Story

A two-person consultancy was hired in January 2026 to ship an internal-policy Q&A agent for a 600-person manufacturing firm. The firm had zero LLM observability. The consultancy's first PR, on day three of the engagement, was a 14-line diff to the firm's existing FastAPI service that pointed the OpenAI client at oai.helicone.ai/v1 and added auth and metadata headers. They merged the PR on a Wednesday afternoon.

By Friday morning — 36 hours later — Helicone's dashboard showed three things the firm didn't know:

1. The agent was making 4x as many OpenAI calls as anyone expected, because a retry loop was firing on a class of errors that should have failed fast.
2. One employee in the legal department was running the agent in a batch script that hit it 8,000 times overnight, generating 60% of the day's cost.
3. The default model was set to gpt-5.5, not gpt-5.5-mini, because a developer had forgotten to roll back a hot fix from December.

The consultancy fixed all three issues in week two: capped retries, identified the batch-script user, switched the default model to mini. Monthly OpenAI cost dropped from a projected $14,000 to $3,200. The Helicone install paid for itself 600x in month one.

The deeper SDK tracing came in month three, after the firm had the appetite to instrument their Python code more invasively. By then the cost story was already won; the SDK tracing was for the next class of problems (which steps produce ambiguous outputs that humans then have to override).

When Helicone Is the Wrong Default

To be balanced about it. The proxy pattern isn't a fit when:

• Your provider doesn't have a Helicone-compatible proxy. The big ones (OpenAI, Anthropic, Azure OpenAI, Together, Groq, OpenRouter) are covered. Smaller or self-hosted endpoints might require workarounds.
• You're already invested in SDK tracing. If you have LangSmith fully wired up and your team is fluent in trace trees, adding Helicone is marginal. The cost/cache/rate-limit features still might be worth it, but as an optional secondary tool, not a primary observability surface.
• You can't tolerate the proxy latency. A few extra milliseconds is fine for almost every application. For very latency-sensitive real-time use cases (voice agents at sub-300ms targets), you might prefer the direct provider call and ship observability via SDK instead.
• You need synchronous PII redaction at the proxy. Helicone supports some redaction features in 2026, but they're not as mature as dedicated PII proxies (see the next lesson on Pre-Call Guardrails). For high-compliance workloads, pair Helicone with a dedicated guardrails layer.

What Helicone Tells You That SDKs Can't

Inversely, Helicone has views that SDK tracing platforms struggle with:

• The bill across all your apps. If your org has six products that call OpenAI from six different codebases, Helicone gives you the cross-cutting view automatically because it sits at the API boundary. SDK tracing requires every codebase to be instrumented.
• The actual user who burned the money. With Helicone-User-Id properly set, the dashboard can rank users by spend and let you investigate the top one. SDK tracing usually requires you to dig into individual traces to find user identity.
• Shadow LLM usage. If a developer spun up an experimental endpoint that calls OpenAI, and they pointed it at the company's Helicone proxy URL (in your shared dev environment), you see the traffic. SDK tracing requires the developer to have wired their code to your SDK; many won't.
• Cache effectiveness. SDK tracers don't operate at the cache layer. Helicone shows you exactly how much you saved.

Key Takeaways

• Helicone is observability you install in five minutes by changing a base URL. Your OpenAI/Anthropic client points at oai.helicone.ai/v1 (or the Anthropic equivalent), you add a Helicone-Auth header, and every request is logged with cost, latency, tokens, and metadata. No SDK to import, no decorator to wrap.
• The trade-off vs SDK-level tracing: you lose tool-call detail and the multi-step trace tree (the proxy only sees LLM API calls, not the agent loop or the tool calls between them); you gain zero-code install and zero-friction adoption across teams that won't write decorators.
• Use Helicone when: you need observability today, the agent is shallow (one LLM call with retrieval), or your team won't write decorators consistently. The first two scenarios are extremely common in 2026; the third is universal in larger orgs.
• Use SDK tracing instead when: the agent is deeply multi-step (research agents, complex reasoning loops), you need tool-call evaluation, or compliance requires VPC-resident proxies and the lift of self-hosting Helicone is too heavy.
• The five-minute install: sign up, get an API key, change the base URL, add the auth header, add optional metadata headers (user, session, feature, tenant, agent version). Ship. Watch the dashboard within minutes.
• What you almost always find in the first 24 hours: a heavy user nobody knew about, a retry loop tripling cost on a failure path, a default model regression, or a feature consuming far more tokens than expected.
• Transparent caching with configurable TTL. A real story: ed-tech team's cache hit rate stabilized at 34%, monthly OpenAI bill dropped from $2,400 to $1,580. The Helicone subscription was $20/month. Net savings $800/month. Cache hits when prompts are redundant; misses when prompts include unique history or real-time context.
• Operational features beyond observability: per-user rate limits at the proxy, virtual keys with their own scope and budget, provider failover (less urgent in 2026 than 2024-2025 but still useful).
• The 2026 pattern at mature teams: Helicone (proxy) for cost/cache/rate-limit and the always-on cross-app view, plus LangSmith/Langfuse/Phoenix (SDK) for trace-tree debugging. Cross-referenced by a shared session ID. The two don't conflict.
• A real shipping story: 14-line PR pointed an existing FastAPI service at Helicone on day three of an engagement. By Friday, the dashboard surfaced three issues (retry storm, batch-script abuser, default-model regression) that dropped monthly cost from a projected $14,000 to $3,200. The install paid for itself 600x in month one.