LiteLLM, Helicone, OpenRouter, Portkey, or Phantm?

If you run real LLM volume, the problem is not picking a model. It is that spend climbs in lockstep with usage, and every "gateway" you evaluate turns out to be a different product wearing the same name: a self-hosted proxy, an observability platform, a model marketplace, a governance layer, a cost optimizer. Pick the wrong one and you solve a problem you did not have while the bill keeps compounding.

Five names dominate this category: LiteLLM, Helicone, OpenRouter, Portkey, and Phantm. Every factual claim here comes from the vendors' own docs, pricing pages, and announcements as of July 2026, including two acquisitions this year that most comparisons have not caught up with.

The short answer

Every one of these tools calls itself an LLM gateway, but they are five different products. LiteLLM is the open-source proxy for teams that want to self-host and reach 100+ providers. Helicone is the observability layer, now in maintenance mode after its acquisition by Mintlify. OpenRouter is the fastest route to 400+ models behind one API key. Portkey, now part of Palo Alto Networks, is the governance and guardrails platform for regulated enterprises. Phantm is the cost-optimization gateway: the only one of the five that actively reduces token spend, 47.1% in a published 13,491-request evaluation, rather than routing, logging, or capping it. What follows is how each one earns its lane, and where each one stops.

What is an LLM gateway?

An LLM gateway (also called an AI gateway or LLM proxy) sits between your application and model providers like OpenAI and Anthropic, so every request passes through one place. That position is what makes everything else possible: one integration for many models, centralized keys and budgets, logging and failover, and, in one case, optimization of the requests themselves.

One distinction does most of the work in this comparison. Dashboards, per-key budgets, and spend alerts are cost tracking: they tell you what you spent. Caching, routing to cheaper models, trimming prompts, and shaping outputs are cost reduction: they change what you spend. Most gateway comparisons blur the two, and most gateways are far better at the first than the second. Beyond that, what actually separates these tools is observability, routing and failover, guardrails and governance, whether you can self-host, and how the pricing model behaves at volume.

LLM gateway comparison table

Read down the cost-reduction row and you can see the shape of the market: four tools that route, observe, or govern your traffic, and one built to shrink it. Each of the four is genuinely the right answer to a specific question, which is why this comparison gives every tool its due before making the case that, once the model bill becomes a line item, the cost question is the one that should win.

LiteLLM: the open-source, self-hosted LLM gateway

LiteLLM is where most teams start, and for good reason. It is the open-source workhorse of the category: a self-hosted proxy that puts one OpenAI-compatible interface in front of 100+ providers, with 52,000 GitHub stars and Netflix and Lemonade among the names on its homepage. Nothing leaves your infrastructure except the model call itself, and there is no percentage fee anywhere, so its economics never scale against you.

Its control plane is built around virtual keys: budgets and rate limits that cascade from org to team to user to key, with layered windows like $10 a day inside $100 a month. It also ships more cost-reduction machinery than it gets credit for, including exact-match and semantic response caching in the free tier, plus five routing strategies from latency-based to cost-based.

The tradeoff is that you operate it: a database, Redis for several routing features, and a config that grows with the project's very fast release cadence. Several features enterprises assume come standard, SSO at scale, audit logs, SCIM, sit behind a custom-priced license. And on the axis this post cares about, one detail matters: LiteLLM's cost-based routing picks the cheapest deployment of the same model. Whether a request needs a flagship at all, or a mini would do, is still a rule you write and maintain by hand. LiteLLM gives you every knob; it does not turn any of them for you.

Helicone: LLM observability first, now in maintenance mode

Helicone built the best answer to a different question: where is the money going? Integration is one line, and what you get back is the strongest observability in the group: every request logged with cost and latency, sessions that reconstruct multi-step agent traces, per-user and per-feature attribution, and a query language for slicing it all. At $79 a month flat with unlimited seats, it is the cheapest serious answer to "which feature is burning our budget," and through 2025 it was growing into a genuine gateway with a fast Rust proxy of its own.

Then the trajectory changed: Mintlify acquired Helicone in March 2026, and by Helicone's own announcement the product is now in maintenance mode. The service stays live and the code stays open source, but feature development has ended, which changes what "should we adopt this today" means.

On cost, Helicone was always a window rather than a lever, by design. Its caching is exact-match, so it saves money only when requests repeat verbatim, and its cost-optimization routing picks the cheapest provider serving the same model. Finding the waste is Helicone's job; removing it was always yours.

OpenRouter: one API for 400+ models

OpenRouter's pitch fits in a sentence: one API key, 400+ models from 70+ providers, one bill, no subscription, and provider pricing passed through without markup. Frontier models are usually available the day they ship, switching is a string change, and the platform routed a claimed 100 trillion tokens last month, with a $113M Series B behind it.

It also does more real cost work than the marketplace label suggests. Routing is price-weighted across providers by default, its NotDiamond-powered Auto Router is a learned model selector with a cost/quality dial, provider prompt caching passes through with sticky routing to keep hit rates up, and zero completion insurance refuses to bill you for empty or errored responses.

Two things put a ceiling on it. The first is the fee: 5.5% on credits and about 5% on bring-your-own-key traffic past a free allowance, a rounding error at $2k a month and roughly $60k a year at $100k. The second is structural: OpenRouter only decides where a request goes, never what it is. Your prompts arrive at the provider exactly as heavy as your app sent them, stale history, unbounded output and all, just at a well-chosen price per token.

Portkey: the enterprise AI gateway for governance and guardrails

Portkey assembled the fullest platform of the four: gateway, observability, prompt management, and its real differentiator, governance. Fifty-plus guardrails with deny, fallback, and retry actions, PII redaction, RBAC and SSO, hard budget caps in dollars or tokens, and SOC2, ISO 27001, HIPAA, and GDPR at the enterprise tier. If your blocker is a security review rather than a bill, this is the strongest resume on the page, and entry is cheap at $49 a month plus $9 per extra 100k logs. Worth knowing: the MIT open-source gateway is the routing plumbing, while caching, the logging UI, and PII redaction live in the hosted product.

As of June 2026, Portkey is part of Palo Alto Networks and is being folded into Prisma AIRS as the gateway layer of a security platform. Existing customers stay supported; for compliance buyers the backing is arguably a plus, and for teams that wanted an independent vendor it is a question mark.

On spend, Portkey is the best enforcer here: hard caps that actually stop traffic, clean attribution per workspace and key. Reduction is thinner: exact-match caching on paid plans, semantic caching enterprise-only and tightly constrained, and routing to cheaper models is a conditional rule you write yourself. Portkey will stop you from overspending; it will not spend less on your behalf.

Phantm: the LLM gateway built for cost optimization

Notice what all four have in common. Whether the lane is control, visibility, access, or governance, every one of them treats the request itself as untouchable. The prompt your app built, with its repeated system preamble, its twenty turns of stale history, its unbounded output, gets logged, routed, priced, and policed, but it reaches the model byte-for-byte as heavy as it left your app, answered by whatever model your code named. At high volume, that untouched payload is where most of the money is.

And cost is the one axis that scales with your success. Observability saves debugging hours once. Governance unblocks a deal once. But every new user makes the bill bigger, forever: a team spending $50k a month that cuts it 47% banks roughly $282k a year, growing with traffic. For plenty of AI products, that cut is the difference between a feature with margins and a feature you quietly throttle.

Phantm is built on the premise that the gateway, sitting in the request path and seeing every prompt, is the one place that waste can be removed in real time. It fronts OpenAI, Anthropic, and OpenAI-compatible endpoints, and it is narrower than the others by design: no content guardrails, no self-hosting today, nothing like 400 models. What it does instead is run every request through an optimization pipeline, and the pipeline starts with a decision none of the other four can make.

Before anything is routed, a trained classifier reads the request and judges how hard it actually is. Not a token-count threshold or a keyword rule: a small model trained on hundreds of thousands of real conversations, running locally inside the gateway in about 12 milliseconds, at zero cost per request. Its judgment drives everything downstream: which model tier answers, how aggressively the prompt can be trimmed, and what shape the output should take. It is also built to fail safe. The classifier is deliberately biased toward over-routing rather than under-routing, uncertain requests get bumped up a tier, and sampled shadow evaluations replay downgraded requests against the premium model to confirm quality is holding. In our published evaluation this moved 60.8% of general traffic and 72.8% of customer-support traffic off the baseline model, and quality held. OpenRouter's learned router is the closest comparison, and the difference is scope: it picks a model from a catalog; Phantm's classifier decides the treatment of the entire request.

The rest of the pipeline attacks the payload. Conversation pruning trims long threads by keeping the system prompt, the latest exchange, and the turns most relevant to the current message, dropping the rest verbatim; it fired on 45% of support-style requests and cut about 29% of input tokens where it ran. A compressor goes further on long prompts, removing low-information content while leaving code, JSON, and tool payloads untouched, and it checks its own work: if any instruction or constraint from the original would be weakened, the compression rolls back.

Then there is the mechanism nobody else runs: provider cache orchestration. OpenAI and Anthropic bill cached prompt prefixes at roughly a tenth of list price, but Anthropic caches nothing without explicit markers, and both ignore prompts under a threshold near 1,024 tokens, so a 900-token system prompt bills at full price on every call, forever. Phantm sets the markers automatically and pads recurring just-under-threshold prompts past the line, which is why input-token cache hit rates in our evaluation reached 89.8% on the Anthropic track. Output shaping handles the expensive side of the bill: output tokens cost five to six times input, so requests classified as short-answer work get format instructions and tight caps, trimming output about 12% overall while leaving tool calls alone. And to be straight about the optional semantic cache: it fired on half a percent of evaluation traffic, so none of the headline number depends on it.

These compound because each works on what the last left behind. A support assistant at a million requests a month on GPT-5.4, with a 1,200-token reused prefix, 800 fresh input tokens, and 500-token responses, costs about $12,500 at list price. Caching the prefix takes it to $9,800, shaping output to 350 tokens takes it to $7,550, and routing 60% of traffic to a mini tier lands near $4,400; the cost calculator runs the same math on your own traffic, and the levers themselves are covered in depth in our LLM cost optimization guide. That is the arithmetic version. The measured version: across 13,491 prompts spanning nine public benchmarks and production-style customer-support workloads, Phantm cut cost 47.1% overall and 60.1% on the support phase, with quality tested under formal TOST equivalence at ±0.2 on a 5-point scale, each provider's responses judged by the other provider's model. Every support source passed. To our knowledge no other vendor evaluation in this category has used formal equivalence testing at all; the full methodology is in the evaluation report.

The objections that should be forming are the right ones, so: latency, the whole pipeline adds under 200ms (P50 181.6ms, P95 206ms) because the classifier is local rather than another API call, and streaming passes straight through. Integration, a base URL swap. And risk, none: a free one-week pilot mirrors your traffic with shadow calls that never touch your live requests and produces a verified savings number measured on your own workload. Convert, and Phantm charges 18% of verified savings if you share data that helps train the routing models (which improves your savings), or 25% on the privacy tier. If it saves nothing, it costs nothing, which is the standard we think every claim in this category should be held to.

Which LLM gateway should you choose?

First, an honest floor: if you spend a few hundred dollars a month on inference, skip all five and call your provider directly. A gateway earns its place when volume does.

Past that, the decision is less "which is best" than "which question is yours." If the question is control, and you have platform engineers to own the infrastructure, LiteLLM in your own VPC is still the answer, at the unbeatable price of free. If it is visibility, Helicone remains the sharpest lens, provided you go in with eyes open about maintenance mode. If it is access, OpenRouter's catalog and zero-ops onboarding are unmatched while your spend is small enough that ~5% reads as convenience. If it is compliance, Portkey under Palo Alto Networks is the strongest governance resume on the page.

But if the question is the bill, and for teams running real volume it eventually is, the field narrows to one. Phantm is the only gateway of the five built to reduce spend rather than route, observe, or cap it, the only one whose routing, trimming, and shaping decisions come from a trained classifier instead of rules, and the only one that publishes equivalence-tested proof that quality survived the savings. The other lanes solve their problem once; a 47% cost reduction pays out on every request you serve for as long as you serve them. And since testing the claim costs a base URL swap and a week of shadow traffic measured against your own workload, the cheapest way to find out whether we are right is to measure it.