On this page · 11 sections
Summary. Google's Ironwood (TPU7x) and NVIDIA's B200 land in almost the same place on paper: 192 GB of high-bandwidth memory per accelerator, roughly 7.37 TB/s of HBM bandwidth on Ironwood against about 8 TB/s on the B200, and near-identical FP8 dense throughput of about 4.6 versus 4.5 PFLOPS. Ironwood went generally available from November 2025 as Google's seventh-generation TPU; NVIDIA's Blackwell B200 shipped across every major cloud through 2025. The real decision is not the chip. It is where you can rent it, what software you already run, and how you pay. Ironwood scales to a 9,216-chip superpod with about 1.77 PB of shared HBM and 9.6 Tb/s inter-chip links, but only on Google Cloud. The B200 rents from roughly $3.70 to $14.24 per GPU-hour across providers as of July 2026, and also runs on-premises. This article compares both on memory, throughput, scale, software lock-in and cost, and gives a decision matrix for AI teams, including Indian teams weighing region availability and data residency.
If you already track B200 versus H100 inference cost per token, this is the sibling question: what happens when the alternative is not another GPU but Google's custom silicon.
The short answer
Pick Ironwood TPU7x when your serving stack is already on Google Cloud, your models run on JAX or PyTorch through XLA, and you need a very large single memory pool for decode-heavy or Mixture-of-Experts inference. Pick the NVIDIA B200 when you need multi-cloud or on-premises placement, when your team is committed to CUDA and TensorRT-LLM, or when portability across providers matters more than squeezing the last unit of price-performance from one vendor.
Both are strong inference accelerators. The mistake is treating this as a benchmark contest. At this tier the hardware is close enough that switching cost, availability and utilisation decide the bill.
Spec-by-spec: Ironwood TPU7x vs NVIDIA B200
Ironwood is one chip built from two chiplets. Each chiplet carries one TensorCore, two SparseCores and 96 GB of HBM, so a full chip exposes two TensorCores, four SparseCores and 192 GB of HBM, per Google's TPU7x documentation (last updated 16 July 2026). The B200 is a dual-die Blackwell GPU with 208 billion transistors and fifth-generation Tensor Cores that add native FP4.
| Specification | Google Ironwood (TPU7x) | NVIDIA B200 (Blackwell) |
|---|---|---|
| HBM per accelerator | 192 GB | Up to 192 GB HBM3e (some HGX configs list 180 GB) |
| HBM bandwidth | ~7.37 TB/s | ~8 TB/s |
| Peak FP8 (dense) | ~4.6 PFLOPS (4,614 TFLOPs) | ~4.5 PFLOPS |
| Lower-precision peak | 2,307 BF16 TFLOPs | ~9 PFLOPS FP4 (dense) |
| Max scale-up domain | 9,216 chips per superpod, ~1.77 PB shared HBM, 9.6 Tb/s ICI | 72 GPUs per rack (GB200 NVL72) |
| Software | JAX and PyTorch (no TensorFlow), XLA, vLLM | CUDA, PyTorch, TensorRT-LLM, vLLM |
| Placement | Google Cloud only | AWS, Azure, Google Cloud, Oracle, neoclouds, on-prem |
Two numbers matter for inference. Memory capacity decides how large a model, and how big a KV cache, you can hold without spilling. Both sit at 192 GB. Memory bandwidth decides decode throughput on memory-bound token generation, where the B200's roughly 8 TB/s edges Ironwood's 7.37 TB/s. On paper this is a narrow gap, not a generation apart.
Where Ironwood pulls ahead
The TPU story is about the pod, not the chip. Google connects up to 9,216 Ironwood chips into a single superpod linked by Inter-Chip Interconnect at 9.6 Tb/s, giving workloads access to about 1.77 PB of shared HBM in one domain. Optical Circuit Switching reconfigures the fabric around a failed link with no human intervention, which matters when a single job spans thousands of chips for days. NVIDIA's rack-scale answer, the GB200 NVL72, links 72 GPUs over NVLink; beyond the rack you scale over InfiniBand or Ethernet rather than one optical fabric.
Google reports Ironwood delivers a 10x peak performance improvement over TPU v5p and more than 4x better performance per chip than the previous v6e (Trillium) generation for both training and inference. On the software side, Google's GKE Inference Gateway load-balances across TPU servers and, per Google, cuts time-to-first-token latency by up to 96% and serving cost by up to 30%. Because vLLM now supports TPUs, teams can move an inference service between GPUs and TPUs with only minor configuration changes, which lowers the cost of testing the switch.
There is external validation of demand. Anthropic said it plans to access up to 1 million TPUs. "Ironwood's improvements in both inference performance and training scalability will help us scale efficiently while maintaining the speed and reliability our customers expect," said James Bradbury, Head of Compute at Anthropic.
Where the B200 pulls ahead
The B200 wins on reach. It rents on AWS, Azure, Google Cloud, Oracle and a long list of specialist "neocloud" providers, and it runs in your own data centre. Ironwood runs on Google Cloud alone. For a regulated enterprise that needs an on-premises deployment, or a team that refuses single-cloud lock-in, that difference ends the debate before specs enter it.
The second advantage is software gravity. A decade of CUDA, plus TensorRT-LLM and a deep tooling ecosystem, means most existing inference code, kernels and observability already target NVIDIA. Ironwood supports JAX and PyTorch through XLA but does not support TensorFlow, and moving custom CUDA kernels to a TPU is real engineering work. If your team ships on CUDA today, the B200 is the path of least resistance. The same portability logic runs through our analysis of the NVIDIA Rubin buildout and enterprise cloud budgets.
The cost model: how you actually pay
Sticker throughput is not the bill. Utilisation, idle time, networking and migration effort are. The two platforms even price in different units.
| Dimension | Ironwood TPU7x (Google Cloud) | NVIDIA B200 (multi-cloud) |
|---|---|---|
| Billing unit | Per chip-hour, sold in 4-chip VMs (tpu7x-standard-4t) | Per GPU-hour |
| On-demand indicative | Published per chip-hour on the Google Cloud TPU pricing page; varies by region and commitment | ~$3.70 to $14.24 per GPU-hour across providers (July 2026) |
| Spot / preemptible | Spot TPU VMs available | ~$3.59 per GPU-hour reported on some providers |
| Discount levers | Committed-use (1 and 3 year), calendar-mode and long-term reservations, All Capacity mode | Reserved and committed GPU-hours |
| Smallest viable unit | 4-chip VM (single host) | 1 GPU |
| Lock-in risk | Google Cloud plus XLA/JAX | CUDA ecosystem |
Google does not publish a single headline chip-hour figure the way GPU aggregators list a range, because TPU rates depend on region, commitment and capacity mode; the Google Cloud TPU pricing page is the source of record. B200 rental, by contrast, spreads widely: aggregators put on-demand pricing from about $3.70 per GPU-hour on lower-cost providers to $14.24 per GPU-hour on hyperscaler instances as of July 2026, with spot pricing near $3.59. That spread is the real lesson. The same B200 can cost four times as much depending on where you rent it, so provider selection often saves more than chip selection. Indian teams costing this in rupees should read our breakdown of GPU cloud rental pricing in India alongside the global ranges here.
The honest cost comparison is per useful token served at your target latency and utilisation, not per advertised PFLOP. A cheaper accelerator running at 40% utilisation loses to a costlier one at 85%. Model that before you commit.
What changes with the eighth-generation TPU
Ironwood is no longer Google's newest silicon. On 22 April 2026, at Google Cloud Next, Google announced its eighth generation as two specialised chips: TPU 8t for training and TPU 8i for inference, both scheduled for general availability later in 2026. TPU 8i is the one to watch for serving. Google pairs 288 GB of HBM with 384 MB of on-chip SRAM, roughly 3x the previous generation, to keep a model's active working set on-chip, doubles inter-chip bandwidth to 19.2 Tb/s for Mixture-of-Experts models, and claims 80% better performance-per-dollar than Ironwood, enough to serve nearly twice the volume at the same cost. Both eighth-generation chips claim up to 2x better performance-per-watt than Ironwood.
The practical read: if you are buying committed capacity now for a 2026 serving workload, Ironwood is the available, proven option today. If your timeline is flexible into late 2026, price the TPU 8i story into your plan rather than locking a long commitment against Ironwood at full rate. This cadence is exactly why we treat accelerators as a rolling decision in our guide to AI compute capacity planning.
A decision matrix
| Your situation | Lean Ironwood TPU7x | Lean NVIDIA B200 |
|---|---|---|
| Serving stack already on Google Cloud | Yes | Only if multi-cloud is required |
| Codebase is CUDA / TensorRT-LLM | Migration cost is high | Yes |
| Runs on JAX or PyTorch via XLA | Yes | Yes |
| Needs on-premises or multi-cloud placement | No | Yes |
| Very large single shared-memory pool | Yes (superpod) | Rack-scale via NVL72 |
| Portability across vendors is the priority | No | Yes |
| Lowest achievable rental with provider shopping | Limited to Google Cloud | Yes (wide price spread) |
India-specific considerations
Indian AI teams face two constraints beyond raw price: region availability and data residency. B200 capacity is available through multiple clouds and Indian GPU providers, which gives buyers negotiating room and the option to keep inference in-country. TPU availability is tied to Google Cloud regions, so confirm the specific region and capacity mode before committing.
Data residency is the second factor. Where inference touches personal data of Indian users, the Digital Personal Data Protection Act 2023 (DPDP) governs how that data is handled, so the deployment region and the provider's data-processing terms belong in the decision, not just the hourly rate. Public cloud demand in India is rising fast: Gartner forecast end-user public cloud spending in India would reach $17.5 billion in 2026, up 28.1% year on year, with infrastructure-as-a-service the fastest-growing segment as AI workloads scale. That demand is precisely what makes disciplined accelerator selection worth the effort.
FAQ
How eCorpIT can help
eCorpIT is a Gurugram-based, CMMI Level 5 and ISO 27001:2022 certified engineering organisation, and a partner of AWS, Microsoft and Google. Our senior-led teams help AI-heavy companies benchmark inference on TPUs and GPUs against their own models, model cost per useful token at real utilisation, and choose a placement and commitment plan that fits both budget and data-residency requirements. If you are weighing Ironwood, the B200 or the coming eighth-generation TPUs, talk to our team for a workload-specific assessment.
References
Last updated: 27 July 2026.
