On this page · 9 sections
Summary. On its Q1 2026 earnings call on 29 April 2026, Amazon CEO Andy Jassy said the AWS custom-silicon business passed a $20 billion annual revenue run rate, holds more than $225 billion in Trainium revenue commitments, and would run at roughly $50 billion a year if it sold chips like a standalone vendor. Trainium3, AWS's first 3nm accelerator, started shipping at the start of 2026 and is "30-40% more price-performant than Trainium2," which itself carries about 30% better price-performance than "comparable GPUs." A Trn3 UltraServer packs up to 144 chips for up to 362 FP8 PFLOPs. Those are AWS's own numbers. The awkward truth for a buyer in July 2026 is that no independent per-token inference benchmark for Trainium exists yet, while NVIDIA H200 and B200 have been measured in public: H200 as low as $0.226 per million input tokens and $2.955 per million output tokens, B200 landing near $0.10-$0.50 per million tokens for typical FP8 chat. This article separates the vendor claims from the measured data, attributes every benchmark to its baseline, and gives a decision rule for when Trainium is worth the Neuron migration and when it is not.
What Trainium3 actually is
Trainium3 is the third generation of Annapurna Labs' training and inference accelerator, and the first AWS chip built on a 3nm process. AWS announced general availability of Amazon EC2 Trn3 UltraServers at re:Invent 2025. A single Trn3 UltraServer scales to 144 Trainium3 chips and delivers up to 362 FP8 PFLOPs, which AWS positions as up to 4.4x more compute performance, 4x greater energy efficiency, and almost 4x more memory bandwidth than a Trainium2 UltraServer.
Read that "4.4x" carefully: it is an UltraServer-to-UltraServer number, not a per-chip figure. AWS's per-chip claim is narrower. In its own testing on OpenAI's open-weight GPT-OSS model, AWS reported 3x higher throughput per chip and 4x faster response times for Trn3 versus Trn2 UltraServers. Jassy's "30-40% more price-performant than Trainium2" is the per-chip price-performance framing from the earnings call. Three different baselines, three different numbers. Any comparison that blends them into one headline is doing marketing, not engineering.
The networking is the part that matters for large-model inference. A new NeuronSwitch-v1 gives 2x more bandwidth inside each UltraServer, and Neuron Fabric cuts inter-chip communication to under 10 microseconds. At the top end, EC2 UltraClusters 3.0 can connect up to 1 million Trainium chips, ten times the previous generation. This is the same lineage as Project Rainier, the cluster AWS built with Anthropic that wired together more than 500,000 Trainium2 chips, which AWS describes as its largest AI compute cluster and about five times the infrastructure used for Anthropic's prior model generation.
Trainium2 vs Trainium3 vs NVIDIA H200 and B200
The four accelerators most India-serving and global teams weigh for 2026 inference are Trainium2, Trainium3, NVIDIA H200, and NVIDIA B200. They are not drop-in substitutes. The table below sets them against the vectors that actually drive a buying decision, and flags where a number is a vendor claim rather than an independent measurement.
| Vector | AWS Trainium2 | AWS Trainium3 | NVIDIA H200 | NVIDIA B200 |
|---|---|---|---|---|
| Process node | 5nm-class | 3nm (first AWS 3nm chip) | Hopper (4nm-class) | Blackwell (4nm-class) |
| Availability (Jul 2026) | GA, "largely sold out" per AWS | GA, "nearly fully subscribed" per AWS | GA across clouds | GA; AWS P6-B200 on Capacity Blocks |
| Price-performance claim | ~30% better than "comparable GPUs" (AWS) | 30-40% better than Trainium2 (AWS) | Baseline for many comparisons | Successor to H100/H200, higher throughput |
| Independent per-token benchmark | Not yet published | Not yet published | Measured (SemiAnalysis InferenceMAX) | Measured (SemiAnalysis InferenceMAX) |
| Software stack | AWS Neuron SDK only | AWS Neuron SDK only | CUDA, broad ecosystem | CUDA, broad ecosystem |
| Portability | AWS only | AWS only | Every major cloud and on-prem | Every major cloud and on-prem |
The pattern is clear. NVIDIA wins on portability and on the existence of public, reproducible benchmarks. Trainium wins on AWS's price-performance claims and on supply commitments large enough that Anthropic and OpenAI have signed multi-year, multi-gigawatt deals for it. For a deeper hardware-economics view of the NVIDIA side, our analysis of AMD Helios versus NVIDIA Vera Rubin rack pricing and the B200 versus H100 inference cost per token go one layer deeper on rack and token economics.
The cost-per-token question, answered honestly
Here is the sentence most vendor blogs will not write: as of July 2026, there is no public, apples-to-apples inference benchmark that measures Trainium cost per token against NVIDIA. SemiAnalysis, whose open InferenceMAX suite is the closest thing the industry has to a neutral referee, lists Trainium2 and Trainium3 as backends still being added. Its published numbers are NVIDIA and AMD.
What the InferenceMAX data does show, for NVIDIA, is a wide spread that depends entirely on model, precision, framework, and how much latency you accept per user. On H200 with state-of-the-art techniques (multi-token prediction, disaggregation, wide expert parallelism), costs fall as low as $0.226 per million input tokens and $2.955 per million output tokens. On B200, measured cost per million tokens ranges from about $0.02 for a small model in FP4 to $0.31 for a large model in FP8, with realistic production FP8 chat at 50-100 tokens per second per user typically in the $0.10-$0.50 per million range.
Against that, AWS offers claims, not measurements. Its Trainium3 launch cites customers including Anthropic, Karakuri, Metagenomi, NetoAI, Ricoh, and Splash Music cutting training and inference costs by up to 50% versus alternatives, and Decart hitting 4x faster inference for real-time generative video at half the cost of GPUs. Amazon Bedrock, used by more than 125,000 customers, now runs most of its inference on Trainium. Those are real deployments, but "up to 50%" and "comparable GPUs" never name which NVIDIA part is the baseline, so you cannot reconstruct the math.
Jassy was specific about the price-performance ladder:
Our Trainium2 chip has about 30% better price-performance than comparable GPUs, and has largely sold out. Trainium3, which just started shipping at the start of 2026 and is 30-40% more price-performant than Trainium2, is nearly fully subscribed.
Treat that as the ceiling of what AWS will commit to on the record, and treat the absence of an independent benchmark as the risk. The table below keeps each figure with its source and baseline so nobody blends them by accident.
| Metric | Value | Source | Baseline it is measured against |
|---|---|---|---|
| H200 input tokens | $0.226 per M (best case) | SemiAnalysis InferenceMAX | H200, advanced serving techniques |
| H200 output tokens | $2.955 per M (best case) | SemiAnalysis InferenceMAX | H200, advanced serving techniques |
| B200 cost per M tokens | $0.02-$0.31 (typ. $0.10-$0.50 FP8 chat) | SemiAnalysis InferenceMAX | B200, varies by model/precision |
| Trainium2 price-performance | ~30% better | AWS (Jassy, Q1 2026) | Unnamed "comparable GPUs" |
| Trainium3 price-performance | 30-40% better | AWS (Jassy, Q1 2026) | Trainium2 |
| Trainium "up to 50% lower cost" | up to 50% | AWS launch, named customers | Unnamed "alternatives" |
For a pricing anchor on the NVIDIA side inside AWS itself: after the 1 July 2026 increase, EC2 Capacity Block rates run $12.355 per accelerator-hour for P6-B200 and $14.04 for P6-B300. We cover that move in AWS EC2 Capacity Block GPU price rise. AWS does not publish a clean per-chip-hour Trainium number that maps onto those, which is one more reason a direct token-cost comparison stays out of reach for now.
The Neuron migration tax
Trainium runs on the AWS Neuron SDK, not CUDA. That single fact is the whole cost-benefit calculation. NVIDIA's real moat is not the silicon, it is the years of CUDA kernels, libraries, and framework integrations that everything from PyTorch to vLLM assumes by default.
The good news for teams evaluating a move: Neuron has matured. It supports serving with vLLM, training and inference with PyTorch and JAX, and custom kernels through NKI. Recent releases track current framework versions (PyTorch 2.8 and 2.9, JAX 0.6.2 and 0.7, vLLM V1), and Neuron SDK 2.26.0, announced in September 2025, added Llama 4 Scout and Maverick (beta) and the FLUX.1-dev image model on Trn2. In practice, existing PyTorch code often runs with modest changes, largely swapping the CUDA device for the Neuron device on your tensors.
The tax shows up at the edges, and it is real:
| Migration factor | Ports easily | Costs engineering time |
|---|---|---|
| Framework | Mainstream PyTorch, JAX, vLLM paths | Custom CUDA kernels, exotic ops |
| Model coverage | Supported architectures (Llama, common transformers) | New or niche models before Neuron support lands |
| Toolchain | Neuron Graph Compiler, DLC images | Profiling and debugging habits built around CUDA |
| Version discipline | Pinned, supported Neuron releases | Breaking changes: Neuron 2.30 dropped PyTorch/XLA training |
| Exit strategy | Stay multi-framework | Lock-in: Trainium runs only on AWS |
The last row is the one finance and architecture teams underweight. Choosing Trainium is choosing AWS for that workload, full stop. If your board wants a credible multi-cloud exit, that constraint has to be priced in alongside the token savings. Anthropic can justify it because it co-develops low-level Neuron kernels with Annapurna's team; a mid-size product company usually cannot.
Who should switch, and who should not
The decision is rarely "Trainium or NVIDIA" across the whole company. It is workload by workload.
| Profile | Recommendation | Why |
|---|---|---|
| Steady, high-volume inference on Amazon Bedrock | Strong fit for Trainium | Bedrock already runs most inference on Trainium; you inherit the savings with no porting |
| Large, stable training runs already on AWS | Evaluate Trainium seriously | Committed-capacity economics and Project Rainier-class scale favour it |
| Fast-moving model experimentation, newest architectures first | Stay on NVIDIA | CUDA gets day-one model support; Neuron support can lag |
| Multi-cloud or on-prem requirement | Stay on NVIDIA | Trainium is AWS-only; portability has real option value |
| Small team, no spare platform engineers | Default to NVIDIA, or use Trainium only via Bedrock | The Neuron migration tax needs engineers you may not have |
If you want the broader capacity picture behind these calls, see our AI compute capacity planning pillar and the GPT-5.6 inference cost breakdown for the model-API side of the same budget.
India-specific considerations
For Indian enterprises and the global capability centres operating from Bengaluru, Hyderabad, and Gurugram, the Trainium question is mostly a FinOps and data-residency question. Committed GPU capacity is expensive and scarce; our India GPU cloud rental pricing note tracks the ₹ rates. Trainium's appeal is the promise of cutting a large, steady inference bill, but the AWS-only constraint interacts with two Indian realities. First, teams that must keep personal data inside Indian regions under the Digital Personal Data Protection Act 2023 (DPDP) should confirm the specific instance families are available in the AWS region they are pinned to before committing. Second, a single-vendor accelerator weakens negotiating use, which matters when procurement cycles are annual and rupee budgets are fixed. The pragmatic path for most Indian mid-market teams is to consume Trainium indirectly through Bedrock, where the savings arrive without a migration project, and to keep net-new training portable on NVIDIA until a workload is stable enough to justify the lock-in.
FAQ
How eCorpIT can help
eCorpIT is a Gurugram-based, CMMI Level 5 and ISO 27001:2022 certified engineering organisation that helps teams model accelerator economics before they sign a multi-year commitment. Our senior engineering teams benchmark your real inference workload, quantify the Neuron migration effort against your CUDA footprint, and design a Bedrock-first or portable-NVIDIA path that fits your data-residency and FinOps constraints. If you are weighing Trainium against NVIDIA for a 2026 budget, talk to us about a workload-level assessment.
References
_Last updated: 27 July 2026._
