On this page · 11 sections
Summary. Moonshot AI's Kimi K3 is a 2.8-trillion-parameter open-weight model that activates about 104 billion parameters per token. Calling it through an API costs $3 per million input tokens and $15 per million output tokens (OpenRouter list price, July 2026). Self-hosting is a different order of problem. The mixture-of-experts design keeps every expert resident in memory, so the weights alone need roughly 1.4 TB of VRAM at the native 4-bit quantisation, which in practice means an 8x NVIDIA B200 node (1,536 GB) at the low end and 16x H200 (about 2.25 TB across two nodes) for the full 1-million-token context. At a mid-market rate near $5.50 per GPU-hour, an 8x B200 node runs about $32,000 a month, so the break-even against the $15/M API sits above 2 billion output tokens a month. Below that volume, the managed API wins on both price and operational effort. This guide sizes the hardware, prices the node against verified 2026 cloud rates, and works the break-even so you can choose between the API, a dedicated hosted endpoint, and full self-hosting.
Kimi K3 reached general API availability in the second half of July 2026 (OpenRouter dates the model to 16 July), and Moonshot published the downloadable open weights later that month on Hugging Face. It is a multimodal reasoning model aimed at coding, knowledge work, and long-horizon agentic workflows, and independent testing put it at the top of the open-weight field: 57 on the Artificial Analysis Intelligence Index against 51 for GLM-5.2, with a lead on the SWE Marathon and Program Bench coding evaluations. That is the reason teams want it in-house. What follows is the part the launch coverage skipped: what running it yourself actually costs.
Why 104 billion active parameters is a memory trap
The headline efficiency number for K3 is the 104 billion parameters it activates per token out of 2.8 trillion total. That sparsity is what makes each generated token cheap to compute. It does nothing for memory. A mixture-of-experts model has to hold all of its experts in VRAM at once, because any token can route to any expert; the router picks a small subset for each token, but the full 2.8 trillion parameters sit resident on the GPUs the whole time the endpoint is up. So the compute bill scales with 104 billion and the memory bill scales with 2.8 trillion. For self-hosting, memory is the binding constraint, and it is the reason the in-house economics look nothing like the per-token API price.
Moonshot ships K3 under a Modified MIT license, permissive enough for commercial use, with the weights on Hugging Face. The training data and training code are not included, so this is open-weight, not open-source, which is the caveat enterprises keep running into. Nathan Lambert, who writes the Interconnects newsletter, argued that this next scale of open model needs a much larger infrastructure lift before inference providers optimise it, the kind of work closed labs do privately before they announce. Any team that pulls the weights inherits that lift.
Sizing the VRAM by precision
Start with the weights, because they set the floor. Parameter count times bytes-per-parameter gives the resident weight memory, before any KV cache or activations. K3 ships natively in MXFP4, a 4-bit format, so the realistic planning number is the bottom row.
| Precision | Bytes/param | Weight VRAM | GPUs to hold weights |
|---|---|---|---|
| BF16 | 2.0 | ~5.6 TB | 40x H200 or 30x B200 |
| FP8 (Q8) | 1.0 | ~2.8 TB | 20x H200 or 16x B200 |
| MXFP4 (native) | ~0.5 | ~1.4 TB | 10x H200 or 8x B200 |
The arithmetic is deliberately simple: 2.8 trillion parameters at half a byte each is about 1.4 TB. On top of the weights you need the KV cache and activation buffers. K3 uses Kimi Delta Attention (KDA), a linear-style attention design that grows the KV cache far more slowly than full attention, so a 1-million-token context is less punishing than it would be on a vanilla transformer. It is still not free. Reserve 15 to 25 percent above the weight footprint for cache and runtime overhead, which pushes a real deployment target to roughly 1.6 to 1.75 TB of usable VRAM.
The hardware that actually fits
Two NVIDIA parts are in play. A B200 carries 192 GB of HBM3e, so eight of them on one baseboard give 1,536 GB. An H200 carries 141 GB, so eight give about 1,128 GB and sixteen give about 2.25 TB.
| Configuration | Total VRAM | Fits MXFP4 weights? | Notes |
|---|---|---|---|
| 8x B200 (1 node) | 1,536 GB | Yes, ~136 GB headroom | Tight for 1M context or high batch sizes |
| 16x H200 (2 nodes) | ~2.25 TB | Yes, comfortable | Tensor-parallel 16; needs fast node-to-node fabric |
| 8x H200 (1 node) | ~1,128 GB | No | Below the 1.4 TB weight floor |
| Rented GPU cluster | Varies | Depends on SKU | Fastest to stand up; no capex |
The single-node 8x B200 box is the practical minimum, and the 136 GB it leaves after weights is enough for moderate context and batch thanks to KDA. Push toward the full 1M-token window or high concurrency and you want 16x H200 across two nodes, with tensor-parallel size 16 and a low-latency interconnect between them. One more launch-week trap: KDA and K3's Stable LatentMoE layers are not in the stable vLLM or SGLang releases yet, so at the moment you run a nightly build and pass --trust-remote-code to load Moonshot's custom modeling file. Budget engineering time for that, not just GPU hours.
What the node costs per month
GPU rates move more than 5x depending on provider, region, and commitment, so the monthly bill is a range, not a point. The table prices a single 8x B200 node at 730 hours a month.
| Rate scenario | $/GPU-hour | 8x B200 $/hour | Monthly (730h) |
|---|---|---|---|
| Spot / reserved 36-month | ~$2.25 | ~$18 | ~$13,100 |
| Mid-market on-demand | ~$5.50 | ~$44 | ~$32,100 |
| Hyperscaler capacity block | ~$9.36 | ~$74.88 | ~$54,700 |
| Large-cloud list on-demand | ~$14.24 | ~$113.92 | ~$83,000 |
Those are verified 2026 figures: B200 capacity on specialist clouds such as Lambda sits near $5.50 per GPU-hour, AWS Capacity Blocks price Blackwell around $9.36, and the large-cloud on-demand list rate reaches roughly $14.24, while 36-month reserved contracts fall to about $2.25. Two cautions before you take the cheapest row. Spot capacity is interruptible, which is a poor match for a stateful serving node holding 1.4 TB of weights that take minutes to reload. And a production deployment is rarely one node: high availability usually means two, plus load balancing, storage, egress, and the time of an engineer who can keep a tensor-parallel-16 job healthy. The GPU line is the floor of the real cost, not the total.
The API baseline and the break-even
The comparison point is the managed API at $3 per million input tokens and $15 per million output tokens, with prompt caching cutting the effective input cost by 60 to 80 percent on repeated context. Break-even is straightforward: divide the monthly node cost by the API price per token to get the volume at which self-hosting matches the API on price alone.
| Node monthly cost | vs $15/M output only | vs $9/M blended (1:1 I/O) | vs $6/M blended (3:1 input-heavy) |
|---|---|---|---|
| $13,100 (reserved) | ~0.87B output tokens | ~1.46B tokens | ~2.18B tokens |
| $32,100 (on-demand) | ~2.14B output tokens | ~3.57B tokens | ~5.35B tokens |
| $54,700 (capacity block) | ~3.65B output tokens | ~6.08B tokens | ~9.12B tokens |
Read the middle row as the base case. An on-demand 8x B200 node needs somewhere between 2 and 5 billion tokens a month, depending on your input-to-output ratio, just to match the API price, and that is before you count the operations burden the API removes entirely. There is a second test the price math hides: throughput. Matching the API at 2.14 billion tokens a month means sustaining roughly 814 tokens per second, every second, 24 hours a day. A saturated node can produce that; a node serving business-hours or bursty traffic sits idle much of the day and never approaches its break-even volume. Utilisation, not sticker price, is what decides whether self-hosting pays. We reach the same conclusion in our DeepSeek V4 self-hosted versus API GPU cost breakdown and the Inkling 975B self-host cost analysis: the largest open models only pay for themselves at constant, high load.
The three paths, compared
Most teams do not face a binary. There is a managed API, a dedicated hosted endpoint on an inference provider, and full self-hosting in your own account.
| Dimension | Managed API | Dedicated hosted endpoint | Full self-host |
|---|---|---|---|
| Cost model | Per token ($3/$15) | Per GPU-hour on the provider | Reserved or capex plus ops |
| Best below | ~1-2B tokens/month, bursty | Steady mid volume, no data centre | High sustained volume |
| Data control | Data leaves to the provider | Provider VPC / isolated capacity | Fully inside your VPC |
| Ops burden | None | Low | High (nightly builds, TP16, HA) |
| Time to production | Minutes | Hours | Days to weeks |
The dedicated endpoint is the option teams overlook. Providers such as Together AI rent isolated GPU capacity by the hour (an H100 dedicated endpoint lists around $6.49 per GPU-hour), which gives you data isolation and predictable performance without buying, wiring, or babysitting hardware. It is often the right middle step before committing to owned infrastructure. Our guide to running local LLMs in production with vLLM, Ollama, and LM Studio covers the serving stack once you commit to hosting.
When self-hosting actually wins
Three cases justify the memory bill and the operations load. The first is data sovereignty: self-hosting keeps prompts and outputs inside your own environment and, for K3 specifically, off a provider API hosted in China, which is a real consideration for regulated workloads and is one of the main reasons enterprises pull the weights at all. The second is sustained high volume above the break-even band, where owned or reserved hardware genuinely undercuts per-token pricing. The third is deep customisation: full weights let you fine-tune and domain-adapt the model in ways an API cannot, which is a separate build decision we cover in our open-model fine-tuning and domain-adaptation service. If none of those apply, the API or a dedicated endpoint is the cheaper and calmer answer. If you are still deciding whether K3 earns a place in your stack at all, start with the Kimi K3 benchmarks and adopt-versus-wait analysis before you cost the hardware.
India-specific considerations
For Indian teams the arithmetic is the same but two factors shift the decision. First, cost in local terms: an on-demand 8x B200 node at about $32,000 a month is on the order of ₹3 crore a year at mid-2026 exchange rates, before power and staff, which puts owned inference out of reach for all but very high-volume products and pushes most teams toward reserved capacity or a hosted endpoint. Second, Blackwell-class capacity inside India is still constrained, so many teams rent from global clouds or a handful of Indian GPU providers, which reintroduces the data-residency question. Where prompts carry personal data, the Digital Personal Data Protection Act 2023 makes in-country or in-VPC hosting attractive, and that governance requirement, more than raw price, is often what tips an Indian buyer toward self-hosting. Our cloud FinOps playbook for Indian teams sets out how to model reserved-versus-on-demand commitments before you sign.
How eCorpIT can help
eCorpIT is a Gurugram-based, senior-led engineering organisation, founded in 2021 and certified for CMMI Level 5, MSME, and ISO 27001:2022. We size, deploy, and operate open-weight models such as Kimi K3 on your own cloud account or on dedicated hosted capacity, with the tensor-parallel serving, autoscaling, and cost modelling that keep a large mixture-of-experts endpoint healthy. We design these deployments aligned with DPDP Act 2023 requirements so data residency and consent handling are built in from the start. To scope a private deployment or a break-even analysis for your workload, see our private LLM deployment service or contact us.
FAQ
References
_Last updated: 29 July 2026._
