SambaNova's 20M token/day cap makes its RPD limits unreachable in 2026

Summary. SambaNova's rate limits page, read on 24 August 2026, gives its paid Developer tier 12,000 requests per day on DeepSeek-V3.1 and 48,000 on Meta-Llama-3.3-70B-Instruct, then adds a one-line note: "Developer Tier accounts are limited to 20M tokens per day across all models." Divide 20,000,000 by 12,000 and you get 1,667 tokens per request. Divide it by 48,000 and you get 417. Any workload averaging more than that burns the token budget long before it touches the request budget, so the request-per-day figures in the tables are decorative for everything except very short prompts. The same page defines TPD as a "Free tier only" measure, which is the opposite of what the note says. Prices are no easier to read: the pricing page returned no figures to a non-JavaScript client on 24 August 2026, while the API reference example shows $5.00 per million prompt tokens and $7.00 per million completion tokens, against $0.35 and $0.75 per million for gpt-oss-120b on Cerebras as of the same date.

What SambaNova published

The SambaNova model rate limits page opens by defining three units: RPM (requests per minute), RPD (requests per day), and "TPD: Tokens per day (Free tier only)". Four sections later, an inline note reads: "Developer Tier accounts are limited to 20M tokens per day across all models."

Both statements are on the same page. Only one can be true, and the note is the one that bills you.

Two other details in that page matter for anyone sizing a deployment. First, the 20M ceiling is an account-level pool shared across every model, not a per-model allowance. Second, the Developer tier production table lists four models and the Free tier table lists three: MiniMax-M2.7 is available only once a payment method is linked.

The arithmetic the tables hide

Every RPD figure implies a maximum average request size once a shared token pool exists. The table below divides SambaNova's published 20M daily token pool by each published Developer-tier RPD figure.

An 8,000-token prompt is unremarkable for retrieval-augmented answering or code review. At that size the account exhausts 20M tokens after 2,500 calls, which is 21 percent of the 12,000 RPD allowance on DeepSeek-V3.1 and 5 percent of the 48,000 on Llama-3.3-70B. The request ceiling is not reachable. It is also the number a buyer reads first, because it sits in a table and the token cap sits in a note.

There is a second inversion. SambaNova's Free tier gets 200,000 TPD against 20 RPD, which works out to 10,000 tokens per request. The Developer tier's 20M against 12,000 RPD works out to 1,667. Per request, the free tier's headroom is six times larger. The paid tier gives you a hundred times more tokens overall and six times less room per call, and neither ratio is stated anywhere on the page.

Every host publishes this differently

The four hosted-inference platforms we checked on 24 August 2026 do not agree on whether a daily token ceiling exists, where it is published, or whether cached tokens count against it.

Cerebras states the opposite policy outright: "Hourly and daily restrictions don't apply to Developer (Pay as You Go) tier users. You can use as many tokens as needed within your budget." Cerebras also runs a dual-bucket model, with a total token limit set at three times the uncached limit, so a 70 percent cache hit rate lets a 1M uncached TPM allowance carry up to 3M total tokens a minute. Groq's documentation says plainly that cached tokens do not count towards rate limits. SambaNova's page says nothing either way, which means a cache-heavy agent loop may be spending the same 20M pool twice over.

Together AI publishes no fixed numbers at all: "Dynamic rate limits adjust with usage, so there are no fixed per-model limits published. The most reliable signal is the response itself." Together's limits rise with sustained successful traffic and its 429 responses carry an x-ratelimit-reset header with a suggested retry interval. That is defensible engineering, and it also means a capacity plan built before launch is guesswork unless you buy a dedicated endpoint.

Cerebras publishes two different free-tier numbers

While checking the comparison, we found the same defect in a different shape. Cerebras' rate limits page gives the Free Trial tier 5 RPM and 30K TPM on gpt-oss-120b. Its GPT OSS model page gives the same model, same tier, 30 requests per minute and 60k input tokens per minute. That is a 6x fork on requests and a 2x fork on tokens between two pages of one documentation site. Both agree on 1M tokens per day and both agree the Developer tier is 1K RPM with 1M input TPM.

This is the second Cerebras documentation fork we have logged this week; the earlier one was a Cerebras docs price fork on gemma-4-31b. The pattern is not a Cerebras problem specifically. It is what happens when a component-rendered model page and a hand-maintained limits page are both treated as canonical.

The pricing page will not help you

SambaNova's Cloud pricing page returns a heading and no prices to a client that does not execute JavaScript, which includes most agents and most crawlers. The prices are reachable, but through the API: SambaNova's model list endpoint returns a pricing object with prompt and completion fields per model, alongside context_length and max_completion_tokens. A GET /v1/models call is the honest way to read the price list.

The documented example response is itself out of date. It shows DeepSeek-R1 at $0.000005 per prompt token and $0.000007 per completion token, which is $5.00 and $7.00 per million, and Qwen2.5-Coder-32B-Instruct at $2.00 and $3.00 per million. Neither model appears on the current rate limits page. For scale, Cerebras lists gpt-oss-120b at $0.35 per million input tokens and $0.75 per million output tokens with a 131k paid context window. Read the endpoint, not the example.

What breaks, and how to tell if it is you

The 20M ceiling binds on total tokens, so the exposure scales with prompt size rather than traffic. Three shapes are at risk.

Long-context retrieval. A 32,000-token prompt exhausts the pool in 625 calls a day. That is a small internal tool, not a product.

Agent loops. Each turn resends the accumulated transcript, so a ten-turn agent session at 8K average context spends roughly 80,000 tokens on one user task. The pool covers about 250 such sessions a day across the whole account.

Multi-model routing. Because the pool is shared, a batch job on Llama-3.3-70B can starve a latency-sensitive path on DeepSeek-V3.1 with no per-model isolation and no warning in the tables.

This is the same class of budgeting gap we found in Claude Code's US inference premium and budget cap, where the published ceiling and the billed unit were not the same thing. Check your own exposure by dividing your measured daily token volume, input plus output, by 20,000,000. If the answer is above about 0.6, you are one traffic week away from 429s. The real cost here is usually the migration, not the code.

What to do

Instrument tokens, not requests. SambaNova returns x-ratelimit-limit-requests, x-ratelimit-remaining-requests and the matching -day variants on every response, which tells you about requests. It publishes no equivalent header for the 20M token pool, so you have to count tokens yourself from the usage block on each completion and alert at a threshold you choose.

Separate the batch path from the interactive path. Groq exposes a service_tier parameter with performance, on_demand, flex and auto values, and its Batch API runs independently of per-model synchronous limits. Where a host offers that split, use it, so asynchronous work cannot consume the budget an interactive endpoint depends on.

Do not assume a cloud prototype ports to on-premises. SambaStack's supported model list tops out at Meta-Llama-3.3-70B-Instruct, DeepSeek-V3.1, gpt-oss-120b, Qwen3-32B and Llama-4-Maverick. MiniMax-M2.7, DeepSeek-V3.2 and gemma-4-31b, all available on SambaCloud, are absent. A team that prototypes on the cloud flagship and plans a regulated on-premises rollout has a model swap ahead of it, and SambaStack context lengths are set per bundle at deployment time, from 4K up to 128K depending on batch size.

Price caching before you assume it saves money. On Groq and Cerebras, cached tokens are exempt from the limits that matter. On SambaNova the documentation does not say, so the safe planning assumption is that they count.

India-specific considerations

For teams building in India, the shared 20M pool interacts badly with the usual staging pattern of running dev, QA and production against one vendor account to keep procurement simple. One account means one ceiling. Split the keys, or a load test in QA takes production down.

Where the workload touches personal data, the Digital Personal Data Protection Act 2023 makes the choice of inference host a processing decision, not just a cost one. A dynamic-limit provider that cannot commit to a published throughput number is harder to write into a data processing agreement than one that publishes fixed limits, and provisioned or dedicated capacity is usually the only form that carries a throughput commitment you can cite.

What is still unknown

SambaNova has not said whether the 20M figure is a hard reject or a soft throttle, what error code it returns, or whether it resets on a rolling window or at a fixed UTC hour. The rate limits page documents x-ratelimit-reset-requests-day for the request counter and nothing for tokens. Nor is it clear which of the two contradictory statements is stale: the "Free tier only" definition may predate the introduction of a Developer ceiling, or the note may be an error. We have asked and will update this page when SambaNova responds.

FAQ

How eCorpIT can help

Capacity planning for hosted inference is an instrumentation problem before it is a vendor-selection problem, and the numbers that bind are rarely the ones in the marketing table. Our senior engineering teams size token budgets against measured traffic, split batch and interactive paths across keys or service tiers, and build the token accounting that hosts like SambaNova do not expose in headers. We work under ISO 27001:2022 and CMMI Level 5 processes, which matters when the workload touches regulated data. Talk to our AI platform engineering team about your current token volumes and where your ceiling actually sits.

References

1. SambaNova model rate limits - SambaNova documentation, read 24 August 2026

1. SambaStack models and bundles - SambaNova documentation

1. SambaNova model list endpoint - SambaNova Cloud API reference

1. SambaNova Cloud pricing - SambaNova Cloud console

1. Cerebras rate limits - Cerebras Inference documentation

1. Cerebras OpenAI GPT OSS model page - Cerebras Inference documentation

1. Groq rate limits - GroqDocs

1. Groq prompt caching - GroqDocs

1. Groq service tiers - GroqDocs

1. Together AI rate limits - Together AI documentation

1. Cerebras docs price fork on gemma-4-31b - eCorpIT, August 2026

1. Kubernetes autoscaling for AI inference - eCorpIT, 2026

Last updated: 24 August 2026.