Inkling's 975B open weights: what self-hosting really costs vs the API in 2026

Summary. Thinking Machines Lab released Inkling on 15 July 2026 under an Apache 2.0 licence: a 975B-parameter Mixture-of-Experts model with 41B active, a 1M-token context window, and native text, image and audio input. The weights are free. The hardware is not. Thinking Machines' own model card requires at least 2 TB of aggregated VRAM for the BF16 checkpoint (8x NVIDIA B300 or 16x H200), or 600 GB for the NVFP4 checkpoint (4x B300, or 8x H200 at W4A16). Priced against Amazon's published EC2 Capacity Blocks rates of 15 July 2026, the cheapest of those configurations runs $29,053 a month. Against Inkling's Tinker API price of $4.68 per million output tokens, you would need to generate roughly 6.2 billion output tokens a month, about 2,362 tokens every second without pause, before the GPUs beat the API. Almost nobody clears that bar. The interesting question is not whether to self-host Inkling, but why you would.

The numbers that decide this, first

Inkling debuted at 41 on the Artificial Analysis Intelligence Index, three points above Nemotron 3 Ultra (38) and well clear of gpt-oss-120b (24). That makes it the strongest open-weights release from a US lab as of 15 July 2026. Thinking Machines does not dispute the ceiling above it. Its own launch post states plainly that "Inkling is not the strongest overall model available today, open or closed."

The pitch is economics, not raw intelligence. Artificial Analysis measured Inkling averaging 25K output tokens per Intelligence Index task, against 43K for GLM-5.2, 38K for Kimi K2.6 and 37K for DeepSeek v4 Pro. Fewer tokens for the same work is a direct cost saving on any metered API, and it compounds in agent loops that run the model thousands of times a day.

Holger Mueller, an analyst at Constellation Research, told SiliconANGLE that the business model matters more than the model: "Unlike its rivals, which charge for model access, Thinking Machines is charging for Tinker, the platform that companies will likely want to use to customize Inkling for their specific use cases. If it's successful, this will further accelerate the commoditization of large language models, and that's something that businesses are going to welcome, because it means they'll see an ROI on their AI investments much more easily."

That framing is the correct one, and it is why the self-host question deserves arithmetic rather than enthusiasm.

What the model card actually demands

Thinking Machines publishes hardware requirements in the Inkling model card rather than leaving them to guesswork. Two checkpoint formats ship, and they are not interchangeable in cost terms.

The architecture explains the footprint. Inkling is a 66-layer decoder-only transformer routing each token to 6 of 256 experts plus 2 shared experts. Only 41B parameters activate per token, which is what keeps inference fast, but all 975B parameters must sit in memory regardless. Sparse activation buys you speed, not VRAM.

Pricing the GPUs against AWS's published rates

Amazon publishes EC2 Capacity Blocks for ML pricing per instance and per accelerator. As of 15 July 2026 the relevant rates are $93.60 per hour for a p6-b300.48xlarge (8x B300, $11.70 per accelerator) in US West (Oregon) and US East (N. Virginia), and $39.799 per hour for a p5e.48xlarge (8x H200, $4.975 per accelerator) in Oregon, Ohio and Asia Pacific (Mumbai).

Take those rates from Amazon's pricing page rather than from any summary of it. Second-hand accounts of GPU pricing drift quickly, and Capacity Blocks rates in particular are revised on Amazon's own schedule rather than a press cycle's.

Running those rates out over a 730-hour month at full utilisation:

The fourth row is the one people miss. Inkling's NVFP4 checkpoint drops the requirement to 4x B300, and on paper that halves your GPU count. On AWS it halves nothing, because p6-b300.48xlarge ships as an eight-GPU instance and there is no four-GPU B300 SKU to rent. The quantisation win only converts into a bill reduction on the H200 path, where 16 GPUs collapse to 8 and the monthly cost drops from $58,107 to $29,053. Quantisation saves money only when your provider sells the smaller unit.

The break-even, and the assumption holding it up

Take the cheapest self-host at $29,053 a month and price it against Inkling's Tinker rate of $4.68 per million output tokens at 64K context.

$29,053 divided by $4.68 gives roughly 6,208 million output tokens. That is about 6.2 billion output tokens a month before the GPUs win. Spread across 730 hours, it means sustaining approximately 2,362 output tokens every second, continuously, for a month, with no idle time, no failed deploys and no traffic troughs.

On a blended one-to-one input-output mix at $1.87 in and $4.68 out, the blended rate is $3.275 per million and the break-even climbs to roughly 8.9 billion tokens a month.

Two things move that line, and both deserve flagging:

The Tinker price is discounted. Inkling currently carries what Thinking Machines' pricing documentation calls a limited-time 50% discount. At undiscounted rates of $9.36 per million output tokens, the break-even halves to about 3.1 billion output tokens a month. A promotional API price is a weak foundation for a hardware decision with a twelve-month lease behind it.

The throughput is unverified. Whether 8 H200s running an NVFP4 41B-active MoE can actually sustain 2,362 output tokens per second under real batching is not something we tested, and Thinking Machines does not publish it. That number is the load-bearing assumption in the whole calculation. Measure it on your own traffic before signing anything. If your rig delivers half that, your break-even doubles.

The honest read: the GPU bill is the easy part to model and the least likely part to decide the outcome. Utilisation is what kills self-hosting, not the hourly rate.

Today's price move makes the API case stronger

Thinking Machines raised Tinker prices effective 17 July 2026, increasing prefill and sample rates by about 50% and training rates by roughly 10%. Inkling is exempt. Its prefill stays at $1.87 and its sample stays at $4.68. Nemotron 3 Ultra, the model Inkling displaced at the top of the US open-weights table, moved from $1.66 to $2.49 on prefill and from $4.15 to $6.225 on sample.

As of today, Inkling is both smarter and cheaper than Nemotron 3 Ultra on the same platform. Combine that with the token-efficiency gap and the spread widens: at 25K output tokens per Artificial Analysis task, an Inkling task costs about $0.117 in output tokens. At GLM-5.2's 43K tokens and Nemotron's new $6.225 sample rate, comparable tasks land near $0.27. Those are our calculations from the two vendors' published figures, not benchmarked prices.

Thinking Machines' own blog reports Inkling matching Nemotron 3 Ultra on Terminal Bench 2.1 at roughly a third of the tokens. On agentic work it scores 63.8% on Terminal Bench 2.1 and 77.6% on SWEBench Verified, below GLM-5.2's 82.7% Terminal Bench score but ahead of Nemotron's 56.4%.

One wrinkle for anyone quoting these numbers: the launch blog and the model card disagree. The blog lists HLE text-only at 29.7%, GPQA Diamond at 87.2% and AA Omniscience at 2.1. The model card lists 30.0%, 87.9% and 1.0 for the same measures. The gaps are small and the model card is the formal document, so cite that one. It is a reminder to read the primary artefact rather than the launch post.

When self-hosting Inkling is the right call anyway

The break-even maths says rent. Three situations override it, and none of them are about cost per token.

Data residency and regulation. If inference cannot leave your infrastructure, the API is not an option at any price and the $29,053 is simply the cost of compliance. For Indian teams in regulated sectors this is the whole conversation, not a footnote to it.

Fine-tuning that becomes your product. Thinking Machines and Bridgewater Associates jointly reported taking an existing open model, training it further on Bridgewater's financial expertise, and scoring 84.7% on financial reasoning tests while running at a fraction of proprietary alternatives' cost. Both TechCrunch and SiliconANGLE noted those results come from the two companies' own evaluation, not an independent one, and the reported cost saving differs between accounts. Treat the direction as real and the multiple as unverified.

Switching costs you have not priced. Mitch Ashley of The Futurum Group put the architectural case to the Wall Street Journal: "Engineering teams should treat base-model selection as an architecture decision. The model an organization fine-tunes becomes part of its software substrate and switching costs compound with every downstream customization. That evaluation cannot be deferred."

That is the strongest argument in the release, and it has nothing to do with GPU hourly rates. If you fine-tune Inkling on proprietary data, the resulting weights are yours under Apache 2.0, and no vendor can reprice or deprecate them. Tinker itself retired eleven Qwen models, six Llama models and Kimi-K2-Thinking on 12 June 2026. Owning the weights is insurance against that, and insurance is not supposed to be cheaper than the risk.

India-specific considerations

AWS prices p5e.48xlarge in Asia Pacific (Mumbai) at $39.799 per hour, identical to Oregon and Ohio. The 8x H200 NVFP4 path therefore costs the same $29,053 a month in Mumbai as in the US, billed in USD. That is unusual and useful: for Indian teams, keeping Inkling inference inside the country carries no AWS Capacity Blocks premium over running it in Virginia. The p5.48xlarge H100 instances are actually cheaper in Mumbai at $31.464 per hour than in Oregon at $34.608.

For teams handling personal data under the Digital Personal Data Protection Act, that pricing parity removes the usual excuse for offshore inference. If residency is the reason you are self-hosting, Mumbai does not cost extra. Our write-up on DPDP compliance costs for Indian startups covers the deadlines that make this a board-level question rather than an engineering preference.

The catch is supply, not price. Capacity Blocks are reservations, and Amazon states that reservation prices are updated regularly based on supply and demand, with the next update scheduled for July 2026. A quoted rate is the rate at purchase, not a rate you keep.

The deployment stack, and what to check before you commit

The model card lists the frameworks that can serve Inkling directly on your own GPUs: SGLang, vLLM, TokenSpeed, Unsloth, or Hugging Face transformers. Thinking Machines worked with RadixArk on SGLang and Miles support, Inferact on vLLM, Lightseek on TokenSpeed, and Unsloth on llama.cpp.

A tensor-parallel launch across 8 H200s takes roughly this shape:

            # Illustrative shape only - confirm model-specific flags against
# the vLLM/SGLang docs and the Inkling model card before running.
vllm serve thinkingmachines/inkling \
  --tensor-parallel-size 8 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.92
          

Do not copy that into production. The NVFP4 checkpoint needs quantisation flags specific to your framework and driver stack, and W4A4 additionally requires SM100+ hardware. Confirm both against your framework's documentation. The point of the snippet is the shape of the problem: eight-way tensor parallelism, a context length you must choose deliberately, and memory headroom you must tune.

Storage deserves a line too. Tinker charges $0.10 per GB per month for checkpoints. A 975B-parameter model in BF16 is not a small artefact to move, version or roll back, and teams routinely forget that the weights have to land on the node before they can serve traffic. Our note on Kubernetes OCI image volumes for model weights covers the distribution problem this creates.

How this compares with the last open-weights decision

This is the second time in two months the same question has landed. We ran the same arithmetic for GLM-5.2 in our open-weight self-host versus API analysis, and the shape of the answer has not changed: the API wins until utilisation is high and sustained, and then it loses quickly.

What has changed is the licence and the customisation path. GLM-5.2 still scores higher on Terminal Bench 2.1 at 82.7% against Inkling's 63.8%. Inkling answers with Apache 2.0, native audio and image input, a 1M-token context in the open weights, and a fine-tuning platform the vendor actively supports. If your workload is pure coding throughput, GLM-5.2 remains the stronger model on that benchmark. If you intend to fine-tune and own the result, the calculus differs.

Teams already tracking GPU spend as their top FinOps concern should note that this decision is where that spend originates. The reservation is made once and paid for months. Our analysis of the AWS Capacity Blocks price rise and the hybrid routing framework for API spend both apply directly here, because the realistic answer for most teams is neither pure self-host nor pure API.

FAQ

How eCorpIT can help

eCorpIT is a CMMI Level 5 certified technology consultancy in Gurugram, and our senior engineering teams work through exactly this decision with clients running AI workloads on AWS, Azure and Google Cloud. We model the utilisation and throughput assumptions against your real traffic before any GPU reservation is signed, because the break-even calculation above is only as good as the tokens-per-second number underneath it. Where residency under the DPDP Act forces self-hosting, we design the deployment around that constraint rather than pretending the economics drove it. Talk to us about pricing your open-weights deployment against the API before you commit to a reservation.

References

1. Inkling: Our open-weights model - Thinking Machines Lab, 15 July 2026

1. Inkling Model Card - Thinking Machines Lab, 15 July 2026

1. Models and Pricing - Tinker Documentation, Thinking Machines Lab

1. Amazon EC2 Capacity Blocks for ML pricing - Amazon Web Services

1. Thinking Machines has released Inkling, the new leading U.S. open weights model - Artificial Analysis, 15 July 2026

1. Thinking Machines amps up its bet against one-size-fits-all AI with its first open model, Inkling - Connie Loizos, TechCrunch, 15 July 2026

1. Mira Murati's Thinking Machines drops Inkling, an open-weights model anyone can access - Mike Wheatley, SiliconANGLE, 15 July 2026

1. Thinking Machines' first model bets big on customization - Madison Mills, Axios, 15 July 2026

1. Inkling model weights - Hugging Face, Thinking Machines Lab

1. Model deprecations - Tinker Documentation, Thinking Machines Lab

1. Tinker Cookbook - Thinking Machines Lab, GitHub

1. Artificial Analysis: Inkling model page - Artificial Analysis

Last updated: 17 July 2026. Prices and benchmark figures verified against vendor pages on 17 July 2026; AWS Capacity Blocks reservation rates change with supply and demand.