Production AI inference platforms 2026: Modal vs Baseten vs Runpod vs Replicate

Summary. On list prices checked on 22 October 2026, one NVIDIA H100 costs $3.95 an hour on Modal ($0.001097 a second, GPU only), $4.79 on Runpod Serverless flex workers, $5.49 on Replicate ($0.001525 a second) and $6.50 on Baseten ($0.10833 a minute). Modal, Runpod and Replicate bill per second; Baseten bills per minute and bundles a base allocation of CPU and memory, which Modal charges separately. The 2026 news is money and ownership. Modal raised a $750M Series D at a $15.75B valuation on 14 October 2026. Baseten raised $1.5B in June at $13B and bought agent-sandbox maker Blaxel in September. Replicate now sits inside Cloudflare, which is moving Replicate's hosted models onto its own infrastructure. Our worked example, Llama 3.3 70B on two H100s serving 2 million requests a month, costs $3,927 to $9,490 a month on dedicated GPUs against $2,288 on Together AI's per-token API. Pick Modal for Python-native serverless with region pinning, Baseten for managed enterprise serving with self-host and hybrid options, Runpod for the lowest raw GPU rates and no Pod egress fees, and Replicate for public-model APIs and media models. For India-resident data, Baseten self-host or an Indian GPU cloud is the cleaner path.

About this article. GPU prices, billing rules, funding announcements and compliance claims were checked against vendor pages and documentation on 22 October 2026. Vendors change pricing often, so re-check before you commit. No vendor paid for inclusion or placement. Researched with AI assistance and reviewed by our editorial team.

What changed in 2026, and why the platform choice matters now

The money concentrated. Modal says it crossed $1 billion in annualised run rate and raised a $750M Series D led by Accel at a $15.75B post-money valuation on 14 October 2026. Baseten raised a $300M Series E at a $5B valuation in January, a $1.5B Series F at $13B on 22 June, and on 10 September acquired Blaxel, which gives each agent its own microVM sandbox.

Replicate changed owner. Cloudflare announced on 17 November 2025 that Replicate was joining it, promised existing APIs would keep working, and said Replicate's 50,000+ models would come to Workers AI. On 16 April 2026 Cloudflare wrote that the Replicate team had joined its AI Platform team and that it was "replatforming the hosted models onto Cloudflare infrastructure". Bring-your-own-model on Workers AI, packaged with Replicate's Cog, was then being tested with internal teams and design partners. Replicate's changelog shows no entry after 21 April 2026.

Every platform now sells tokens too. Baseten's Model APIs list GPT OSS 120B at $0.10 per million input tokens and $0.50 per million output. Modal launched per-token Shared Endpoints, which plan credits do not cover, and Runpod sells Public Endpoints. Choosing between a platform and an API is now often a choice inside one vendor.

Runpod formalised compliance. Runpod, which retired its old capital-P RunPod styling in June 2025, lists an ISO/IEC 27001:2022 certificate valid from 27 August 2026, plus SOC 2 Type II and SOC 3 reports.

The GPU-hour rate is one line on the bill. Idle charges, CPU and memory surcharges, region multipliers and the choice between dedicated GPUs and per-token APIs decide whether the same traffic costs $460 or $9,490 a month.

The four platforms, one at a time

Modal: Python-first serverless with per-second billing

Modal runs Python functions, web servers and model endpoints in containers on a multi-cloud GPU pool; you declare the GPU in a decorator. The client SDK is Apache-2.0; the platform is proprietary and cloud-only.

Pricing. Modal lists per-second GPU rates: B300 $0.001972, B200 $0.001736, H200 $0.001261, H100 $0.001097, A100 80 GB $0.000694, L40S $0.000542, A10 $0.000306, L4 $0.000222 and T4 $0.000164. CPU costs $0.0000131 per physical core per second and memory $0.00000222 per GiB per second on top. Egress is $0.04 per GiB beyond the plan allowance, which is 1 TiB on Starter. Starter is $0 plus compute with $30 of monthly credits and 10 concurrent GPUs; Team is $250 a month with $100 of credits and 50 GPUs; Enterprise adds volume discounts, Okta SSO, audit logs and HIPAA. SOC 2 is listed on every plan.

Regions. Pinning containers to a region applies a multiplier of 1.15x for a broad region such as ap or 1.75x for a narrow one such as ap-south, and requests can be routed through Mumbai.

Strengths: no idle charge after scale-down, GPU memory snapshots (still an alpha feature) that Modal says speed cold starts, and your choice of vLLM, SGLang or a custom server. Limits: functions are preemptible by default, and GPU functions have no non-preemptible option; HIPAA is Enterprise-only. Best for: Python teams with spiky or batch workloads.

Baseten: managed dedicated serving, Model APIs and self-host

Baseten runs dedicated deployments on its own inference stack, sells per-token Model APIs and offers training. Models are packaged with Truss, which is MIT-licensed.

Pricing. Baseten lists dedicated instances per minute: T4 $0.01052 ($0.63 an hour), L4 $0.01414 ($0.85), A10G $0.02012 ($1.21), A100 80 GB $0.06667 ($4.00), H100 $0.10833 ($6.50) and B200 $0.16633 ($9.98). You pay while a model deploys, scales and serves, not while it is scaled to zero. Model APIs include GPT OSS 120B at $0.10/$0.50 and Kimi K3 at $3.00/$15.00 per million input/output tokens. Each listed price is a base instance with a set amount of CPU and memory; larger CPU and memory configurations cost more.

Plans. Basic is pay-as-you-go; Pro adds priority access to scarce GPUs and volume discounts; Enterprise adds self-hosting in your VPC, hybrid deployment, control over data residency and custom regions.

Strengths: the only published self-host and hybrid path of the four, with SOC 2 Type II and HIPAA stated. Limits: the highest H100 list rate here, per-minute billing, and non-default regions via sales. Best for: custom or fine-tuned models in production that need enterprise controls.

Runpod: cheapest raw GPUs, serverless workers and Pods

Runpod sells Serverless endpoints, Pods (GPU instances you manage) and Clusters. Its Python SDK is MIT-licensed.

Pricing. The pricing page, updated 8 October 2026, lists Serverless flex workers per hour: B200 $8.64, H200 $5.93, H100 $4.79, A100 $2.72, the 48 GB L40S class $1.75 and the 24 GB L4 class $0.69. Pods on Secure Cloud list H100 SXM at $3.99 an hour, A100 SXM at $1.79 and B200 at $7.99; the cheaper Community Cloud lists H100 SXM at $2.69. Flex workers scale to zero and are billed from start to full stop, rounded up to the second, including model loading; the default idle timeout is 5 seconds, and discounts for always-on active workers go through sales. Pods carry no ingress or egress fees and offer 3- or 6-month savings plans.

Strengths: the lowest list rates here, 31 regions and a sub-200 ms FlashBoot cold-start claim (vendor figure). Limits: an $80-an-hour default spend limit, and Pods leave scaling and failover to you. Best for: cost-sensitive teams that can own their serving container.

Replicate: public-model marketplace, now part of Cloudflare

Replicate offers API access to thousands of public models, many for image, video and audio, plus private deployments packaged with Cog, which is Apache-2.0.

Pricing. Replicate lists T4 at $0.000225 a second ($0.81 an hour), L40S $0.000975 ($3.51), A100 80 GB $0.0014 ($5.04), H100 $0.001525 ($5.49) and 2x H100 $0.00305 ($10.98); H200 capacity needs a committed-spend contract. Most public models bill only for processing time. Private models bill for setup, idle and active time, except fast-booting fine-tunes.

Strengths: the broadest public catalogue and simple per-prediction economics for media models. Limits: private deployments pay for idle time, no region selection is published, and the enterprise page mentions data processing agreements and indemnity but names no audit report. The roadmap now follows Cloudflare. Best for: calling public image, video and audio models.

Head-to-head comparison

Where the alternatives fit

Sources for the alternatives: Together AI, Fireworks, fal, Lambda, AWS Bedrock, Google Cloud's locations page, where Vertex AI documentation now sits under Gemini Enterprise Agent Platform, and E2E Networks.

Decision factors that move the bill

Cold starts. No published figure covers loading 70 GB of weights. Runpod's FlashBoot and Modal's snapshots help small and medium models most, and Runpod's own billing docs say cold start time is mostly your handler loading models into GPU memory. For a 70B model, plan to keep one replica warm, which turns serverless into an always-on bill.

Scale to zero. It pays only when traffic has gaps. At 2 million requests a month the average is 0.76 requests per second, so there is little idle time to recover. It pays off for internal tools, batch jobs, evals and bursty media models. For queue-driven scaling on your own cluster, see our KEDA and HPA inference autoscaling guide.

Surcharges. Per-second versus per-minute billing barely matters for always-on replicas. Modal's separate CPU and memory charges (about $0.89 an hour for 8 cores and 64 GiB), region multipliers and Replicate's idle billing on private models matter more.

Serving engine. Throughput per GPU depends on engine, precision and batching more than on the platform. Modal and Runpod let you run vLLM, SGLang or TensorRT-LLM yourself; Baseten runs its own stack; Replicate runs whatever is in your Cog container. Confirm engine and quantisation before comparing hourly rates.

Egress and commitments. Text egress is trivial (about 2.4 GB a month in our example, assuming 4 bytes per token) but matters for media outputs. Modal Enterprise accepts committed spend through the AWS and GCP marketplaces; Baseten, Runpod and Replicate negotiate volume or reserved discounts.

Decision table: pick X if…

Worked cost example: Llama 3.3 70B at 2 million requests a month

Scenario. Llama 3.3 70B Instruct in FP8 on two H100s (tensor parallel 2) serves 2,000,000 requests a month at 800 input and 300 output tokens: 1.6 billion input plus 0.6 billion output, 2.2 billion tokens in total. A month is 730 hours (2,628,000 seconds).

Throughput assumption. NVIDIA's TensorRT-LLM performance overview lists 2,209 output tokens a second per GPU for this model in FP8 on H100 with tensor parallel 2, at 1,000 input and 1,000 output tokens. That is a vendor-run, offline maximum-throughput test. Latency-bound serving runs well below it, so we assume 35%: 2 × 2,209 × 0.35 ≈ 1,546 output tokens a second per replica. This is a planning assumption, not a measurement.

Load check. Average demand is 600 million ÷ 2,628,000 ≈ 228 output tokens a second; at an assumed 3x peak, about 685. One replica covers it and stays on all month, because a 70B model is slow to load: 2 GPUs × 730 hours = 1,460 GPU-hours. Modal's CPU and memory are assumed at 8 cores and 64 GiB.

What the numbers say. The per-token API is 1.7 to 4 times cheaper, before a second replica for redundancy doubles every dedicated line. Modal's $6,415 buys about 6,168 million Together tokens, 2.8 times today's volume. At that volume the assumed peak reaches about 1,920 tokens a second, beyond one replica's 1,546, so you need a second replica before break-even. For a stock 70B model on H100s, dedicated serving does not win on price under these assumptions.

What changes the answer. The same NVIDIA table lists 6,920 output tokens a second for Llama 3.3 70B in FP4 on one B200. One B200 on Modal with the same CPU and memory costs ($6.2496 + $0.8888) × 730 ≈ $5,211 a month. At 35% of benchmark (about 2,422 tokens a second) it breaks even with the $1.04 API at roughly 5,011 million tokens, about 4.6 million requests a month, with peak demand still inside one GPU, provided FP4 quality is acceptable. Our B200 vs H100 cost-per-token analysis covers that trade-off, and our AWS G7 and G7e Blackwell benchmark and Ironwood TPU7x vs B200 comparison price the hyperscaler alternatives.

Model choice matters even more: GPT OSS 120B on Baseten Model APIs costs 1,600 × $0.10 + 600 × $0.50 = $460 a month for this traffic. For small models, Together's Llama 3 8B Instruct Lite at $0.14 per million tokens comes to $308, while an always-on Modal L4 costs $583 for the GPU alone. Dedicated serving still makes sense for fine-tuned weights no API hosts, strict latency targets, guaranteed capacity or region-locked data; our hybrid routing framework covers splitting traffic.

India, DPDP and data residency notes

If your legal team requires India-resident processing of personal data under the DPDP Act or sector rules, the four differ sharply.

Modal offers an ap-south container region and Mumbai routing at the 1.75x multiplier ($11,226 a month in our example). Its data residency guide says application logs, Volumes, Images, Secrets, asynchronous payloads and payloads over 2 MiB are stored in the US even when compute is pinned, and Shared Endpoints route through us-west. Staying in-region means Indian bucket mounts, synchronous payloads of 2 MiB or less and no personal data in logs.

Baseten publishes self-hosted and hybrid deployment with data residency control on Enterprise; confirm which Indian cloud regions it supports. Runpod lists data centres in North America, Europe, Japan and Australia and names no Indian region. Replicate publishes no region selection.

Indian GPU clouds. E2E Networks lists H100 at $3.99 an hour before tax. Yotta's Shakti Cloud lists an on-demand 1x H100 VM at ₹356 an hour. The IndiaAI compute portal lists empanelled H100 SXM rates of ₹153 an hour for one GPU and ₹306 for two, before GST, with a subsidy of up to 40% subject to IndiaAI approval, for eligible researchers, students, DPIIT-recognised startups and MSMEs, and government entities. Our 2x H100 setup at that rate is ₹306 × 730 = ₹2,23,380 a month before subsidy and GST. These are GPUs, not a managed serving layer, so budget for vLLM or SGLang, autoscaling and monitoring.

Hyperscalers, EU and US. Google lists H100, H200, L4 and T4 in Mumbai and H200 in Delhi. AWS supports Bedrock Custom Model Import in N. Virginia, Ohio, Oregon and Frankfurt, with Frankfurt at $0.07144 per unit-minute. Modal's broad eu pin costs 1.15x and Fireworks charges 1.5x for region-restricted deployments. For US healthcare data, Baseten and Runpod list HIPAA, Modal only on Enterprise; ask Replicate. We design applications aligned with DPDP, GDPR and HIPAA requirements, but vendor artefacts and your contracts decide compliance.

Implementation checklist

1. Record requests per month, input and output tokens, peak-to-average ratio and latency target before comparing anything.

1. Price the per-token option first; if a provider hosts your model, that rate is your baseline.

1. Benchmark your model on your engine and precision, and replace our 35% assumption with measured tokens per second.

1. Set minimum warm replicas explicitly for models above about 30 billion parameters and treat them as fixed cost.

1. Add every surcharge: CPU and memory, region multipliers, idle time, storage and egress.

1. Check concurrency and spend limits before launch (Modal Starter 10 GPUs, Runpod $80 an hour).

1. Map where requests enter, where containers run and where logs, payloads and weights are stored.

1. Collect SOC 2, ISO 27001, HIPAA BAA and DPA documents before signing.

1. Put an AI gateway in front so traffic can move between APIs and dedicated deployments; our enterprise AI gateway comparison covers the options.

1. Keep one portable vLLM or SGLang image that runs on any platform or your own Kubernetes cluster.

How eCorpIT can help

eCorpIT is a Gurugram-based technology organisation, founded in 2021, assessed at CMMI Level 5 and MSME certified, with senior-led engineering teams working across AWS, Microsoft and Google platforms. Our AI and machine learning services team models your traffic, benchmarks your model on the engines and GPUs you are considering, compares per-token APIs with dedicated deployments and builds portable serving on Modal, Runpod, Baseten, Indian GPU clouds or your own Kubernetes cluster. Talk to us at /contact-us/.

Last updated: 22 October 2026.

References

1. Pricing — Modal

1. GPU cloud pricing — Runpod

1. Pricing — Replicate

1. Pricing — Baseten

1. $1B annualized run rate and $750M Series D — Modal

1. Announcing Baseten's $300M Series E — Baseten

1. Announcing our Series F — Baseten

1. Blaxel is joining Baseten — Baseten

1. Replicate is joining Cloudflare — Cloudflare blog

1. Cloudflare's AI Platform: an inference layer designed for agents — Cloudflare blog

1. Changelog — Replicate

1. Shared Endpoints — Modal documentation

1. Compliance — Runpod

1. Modal client LICENSE — GitHub

1. Region selection — Modal documentation

1. Memory snapshots — Modal documentation

1. Preemption — Modal documentation

1. Truss LICENSE — GitHub

1. Instance types and resources — Baseten documentation

1. runpod-python LICENSE — GitHub

1. Serverless pricing — Runpod documentation

1. Pods pricing — Runpod documentation

1. Serverless — Runpod

1. Cog LICENSE — GitHub

1. Enterprise — Replicate

1. Pricing — Together AI

1. Pricing — Fireworks AI

1. Pricing — fal

1. Pricing — Lambda

1. Amazon Bedrock pricing — Amazon Web Services

1. Locations for machine learning services — Google Cloud documentation

1. GPU cloud pricing — E2E Networks

1. TensorRT-LLM performance overview — NVIDIA

1. Data residency on Modal — Modal documentation

1. Shakti Cloud pricing — Yotta

1. IndiaAI compute portal price calculator — IndiaAI

1. IndiaAI compute portal — IndiaAI

1. Import a customised model into Amazon Bedrock — AWS documentation