Omniousdocs
For providersBeta

Winning sessions

Opt in with a cached rate and a session TTL. Your prefix cache is your incumbency advantage.

Owner
Supply
Version
beta-2026.07
Verified
2026-07-13

Agentic loops are the highest-value traffic on the market, and they do not send one request. Every turn resends a growing shared prefix, and your prefix cache makes those re-reads nearly free, but only at your deployment. Sessions turn that physical fact into revenue: cache locality becomes a priced, bid-able term in the auction, and the provider that actually holds the cache wins the continuation turns. The customer-side view is on the sessions page; this is the provider read.

Opting in is two quote fields

Add price_in_cached (per 1M cached input tokens) and session_ttl_sto your signed quotes. Together they are a two-part tariff and a commitment: for sessions opened against this quote, you will hold the session's cache and honor these rates for that TTL. The reference quoter does it by default (CACHED_FACTOR=0.2, SESSION_TTL_S=900; set SESSION_TTL_S=0 to quote cache-blind, in which case the signing payload is byte-identical to a V1 quote). Only commit to a TTL your serving stack can actually keep the KV cache alive for.

Turn 1 locks your tariff

A session is awarded by an ordinary auction on its first turn, scored on forecast whole-session cost, so a strong cached rate can win you sessions even with a middling fresh rate. Winning locks your cleared second-score tariff for the session. On every later turn you are scored as the incumbent at those locked cached rates, while every challenger is scored at spot paying full re-prefill of the whole context, in both price and time-to-first-token. Sticky by default, contestable every turn.

Context Passport capabilities

not in the mainnet launchPassport cache scoring, delta handoffs, and speculative execution are private soak features and are gated off for the V1 mainnet launch. Advertising a capability does not activate it on a dark router.

Passports reuse the same cache advantage without pinning the customer to a session. Quote context_protocol=passport-v1, context_cache_ttl_s, and your cached input price to let a returning Passport compete as warm while that signed horizon is live. Set delta_handoff=true to accept the structured handoff + hot-tail form; delta-mode requests exclude providers that did not sign that capability. A challenger still receives portable text state and can win whenever its spot economics beat your cache advantage.

Speculative execution is explicit

Set SPECULATIVE=1 only when your serving stack supports prompt cancellation. The quote then signs speculative=trueand speculative_cancel_ms. During a customer-requested hedge, the router may start your attempt and cancel it before customer bytes if another provider produces output first. The window bounds time to first usable output; it never truncates the winning generation.

Your cache is a cost advantage, not market power

The lock cuts both ways, and it is the part worth reading twice. Nothing you re-bid mid-session changes what you are paid: your rates were fixed at award, while the market was still symmetric and rivals could still compete for the session on equal terms. You cannot ratchet prices against a captive loop, and rivals cannot free-ride on your cache. What you keep is the genuine gap between your cached-token cost and a challenger's re-prefill, which is exactly the asset your serving stack built. If a rival's spot price drops far enough that eating the re-prefill is worth it, the session migrates and its cleared rates become the new lock. Hold-up rent is unearnable by construction.

As a providerWinning a loop is sticky revenue at rates you set under competition. Your job is to make the cache advantage real: high hit rates, honest TTLs, and a cached rate that reflects your actual cached-token cost.
As a customerThe incumbent's discount reaches you at locked rates, and every turn stays contestable, so staying put is a priced choice, never a trap.

The stack that wins sessions

Because continuation turns are won on real cached cost, serving engines with high prefix-cache hit rates win more of them. SGLang's RadixAttention is best-in-class at shared-prefix reuse, which is why the first-party provider design (hyper, detailed in designs/hyper.md) builds on it: a better cache engine lowers your marginal cost and strengthens your position under the exact mechanism the market already rewards. One caution: the router measures TTFT on every turn, so serving cache-hit prices with re-prefill latency is self-evidencing, and breaking the session commitment is a quote-firmness fault like any other.

where this lives in code