May 2026 — 23 shipments, zero breaking changes, 30–90% cache savings

Between 2026-05-10 and 2026-05-12 we shipped 23 changes spanning cost optimisation, compliance, reliability, new modalities, new providers, enterprise features, and observability. None of them broke an existing integration. Three of them you can take advantage of today without writing any code.

Three things to try today

1. See your cache savings

Every /v1/chat/completions response now carries nex.cached_input_tokens (upstream cache hit) and nex.semantic_cache_hit (NexToken cache hit). Cache hits are billed at the documented discount:

• OpenAI prompt cache: 50% off the cached portion
• Anthropic prompt cache: 10% off the cached portion
• DeepSeek prompt cache: 10% off the cached portion
• Google Gemini cache: 25% off the cached portion
• Semantic cache (near-duplicate match): 95% off retail

If you use long system prompts or repeated context (chat apps with memory, agentic loops, RAG pipelines), expect 30–90% reduction on cached input tokens — visible per request, not bundled into a monthly bill.

2. Quote a request before you pay for it

POST /v1/tokenize and POST /v1/estimate-cost accept the same payload as /v1/chat/completions and return token counts plus a retail-USD estimate. Cheaper than learning your nightly bill blew up because someone fed a 50K-token document into a loop.

3. Switch to nex-auto

Set model: "nex-auto" in your existing code and the smart router picks the cheapest model that can handle each prompt — general, coder, reasoning, or long-context. The decision is surfaced in response.nex.smart_router.target_model so you can audit per request.

// Before
const r = await openai.chat.completions.create({
  model: "gpt-4o", messages
});

// After (one word changed)
const r = await openai.chat.completions.create({
  model: "nex-auto", messages
});
console.log(r.nex.smart_router); // {target_model, reason, score}

Cost optimisation

Compliance & safety

Reliability

• Redis-backed circuit breaker — provider trips replicate across all NexToken instances; restarts no longer reset state.
• In-provider retry + exponential backoff — 429/503/timeout retries the same provider (0.5s, 1.5s) before rotating, so transient blips don't bounce you to a different model.
• Region-aware routing — APAC users prefer APAC-hosted backends (Qwen / GLM / nex-pro); US users prefer OpenAI / Anthropic direct; EU users prefer Mistral / GLM.
• Multi-key load balancing — multiple keys per upstream rotate round-robin with per-key cooldown; one bad key no longer takes a provider offline.
• Real-time provider scoring — every call records P50/P95/P99 + success rate; routing uses live measurements instead of static heuristics.

New modalities

New providers — 7 new models behind 3 backends

Enterprise

Observability

Breaking changes

Pricing changes

• Cache savings now visible — apps with long system prompts or RAG context typically see 30–90% off cached input tokens.
• nex-auto published at $0.0003 / $0.0012 per 1K (general tier default). Actual underlying-model cost surfaced via nex.smart_router.
• Batch API discount: 30% off retail per item (sync). Async batch: 50%.
• Semantic cache hit discount: 95% off retail (5% residual covers the embed lookup).

All existing list prices unchanged.

Questions? Reply to your existing thread or write to support@nextoken.biz.