On this page · 9 sections
Summary. In 2026 you can run useful AI inside a phone with no network and no per-token bill. On 8 June 2026 Apple shipped its third-generation Foundation Models: AFM 3 Core, a 3-billion-parameter on-device model, and AFM 3 Core Advanced, a 20-billion-parameter sparse model that activates just 1 to 4 billion parameters per request. Any iOS app reaches them through the Foundation Models framework with no API key, no network call, and no per-token cost, inside a 4,096-token on-device context. Google's Gemini Nano does the same on Android through AICore and the ML Kit GenAI APIs. The trade-off is capability: a cloud model such as GPT-5.6 or Claude Sonnet 5 is far larger and smarter, but it bills $1 to $50 per million tokens, adds a network round-trip, and sends user data off the device. For most apps the right answer is neither extreme; it is a hybrid that runs common tasks on-device and escalates hard ones to the cloud. This guide shows how to decide, with the numbers.
The question teams ask is "on-device or cloud AI?" The better question is "which task runs where?" A summarise-this-note feature and a complex multi-step research agent have almost nothing in common in cost, privacy exposure, or latency, and forcing both down the same path wastes money or ships a worse product. Below is how the two options actually compare in 2026, what each costs, and a build pattern that uses both.
What on-device AI can do in 2026
Two years ago, on-device generative AI meant autocomplete. In 2026 it means a real, if small, language model built into the operating system.
On Apple platforms, the Foundation Models framework gives any third-party app a Swift API to the on-device model. Apple describes two on-device models in its third generation: AFM 3 Core, a 3-billion-parameter dense model, and AFM 3 Core Advanced, a 20-billion-parameter model that uses a sparse architecture to activate only 1 to 4 billion parameters at a time, storing the full model in flash and swapping experts into memory per prompt. Apple reports the updated AFM 3 Core was preferred on 45.6 percent of text prompts against 23.3 percent for its 2025 model, and its text-to-speech scored 4.15 on a 5-point scale versus 3.87 for the prior system. The on-device model handles language understanding, structured output, and tool calling, now with image inputs, inside a 4,096-token window shared between input and output. Bigger jobs route to Private Cloud Compute with a 32K budget.
On Android, Gemini Nano runs inside AICore, a system service that executes the model on-device using the phone's own silicon. Google exposes it through the ML Kit GenAI APIs, which ship ready-made features: prompt, summarisation, proofreading, rewriting, image description, and speech recognition. Gemini Nano 4 is in developer preview through AICore and reaching flagship devices during 2026, with a Structured Output API and prefix caching for production use. The capability depends on hardware: AICore targets recent MediaTek Dimensity, Qualcomm Snapdragon, and Google Tensor platforms, so older phones fall back to cloud.
The headline for both platforms is the same. The model is on the device, the data never leaves, there is no API key, and there is no per-token invoice. What you trade away is raw capability and a large context window.
On-device vs cloud: the trade-offs that decide it
Here is the comparison that should drive the architecture, not the marketing.
| Dimension | On-device (AFM 3, Gemini Nano) | Cloud (GPT-5.6, Claude, Gemini) |
|---|---|---|
| Per-request cost | $0, no API fee | $1 to $50 per 1M tokens |
| User data | Stays on the device | Sent to the provider |
| Latency | Local, no network round-trip | Network-dependent |
| Works offline | Yes | No |
| Model capability | Small (3B to 20B sparse) | Frontier-scale |
| Context window | 4,096 tokens on Apple on-device | Up to 1M tokens |
| Device support | Recent flagship silicon only | Any device with a connection |
| Battery and thermal | Uses the device's own compute | Offloaded to the server |
Read the table as a routing guide. Anything private, high-frequency, latency-sensitive, or needed offline argues for on-device. Anything that needs a large context, frontier reasoning, or a capability the small model lacks argues for cloud. Most real apps contain both kinds of work, which is why the interesting design is the split, not the choice.
Cost: the number that surprises teams
The per-token price looks small until you multiply by usage. Take a consumer app doing one million AI actions a month, each roughly 500 input and 200 output tokens, so about 700 million tokens a month. A mid-tier cloud model at a blended $2 per million tokens costs about $1,400 a month; a premium model at $10 or more per million runs past $7,000. The same workload run on-device costs nothing per request, because the compute is the user's phone and the battery it draws.
That is the case for on-device at scale, but it is not free of cost. The bill moves from a per-token line to engineering: you build for two code paths, test across a fragmented fleet of chips, handle devices that cannot run the model at all, and accept a smaller model that will sometimes produce a weaker answer than a frontier cloud model would. For a low-volume app, a cloud API is cheaper and simpler than the engineering to make on-device reliable. For a high-volume app where 80 percent of requests are simple, running that 80 percent on-device and sending the hard 20 percent to the cloud can cut the AI bill by most of its value while keeping the smart fallback. Our LLM cost and routing framework works through that split in detail.
Privacy, DPDP and why on-device matters in India
On-device inference has a compliance benefit that is easy to undervalue. When the model runs locally, the user's prompt and data never leave the phone, so there is no cross-border transfer and no third-party processor to account for. For an app handling health notes, financial details, or private messages, that is the difference between a short data-protection review and a long one.
This matters directly under India's Digital Personal Data Protection Act, 2023 (DPDP), which governs how personal data is processed and shared. Running summarisation or classification on-device keeps that content inside the user's device and out of a foreign API log, which is the exact traffic a DPDP review scrutinises. Apple reinforces this at the platform level: its models run on-device or on Private Cloud Compute, which Apple says never stores or shares user data, and it states it does not use users' private personal data to train the models. We design applications aligned with DPDP requirements rather than claiming certification, and for privacy-sensitive apps the on-device path often decides the architecture before cost does. Our write-up on privacy-first AI architecture covers the enterprise version of this pattern.
A hybrid architecture that ships
The pattern that works in production is a router: try the on-device model first, escalate to the cloud only when the task needs it. Apple made this easier in 2026 by opening the Foundation Models framework to any provider, local or server, with Anthropic and Google adding Swift packages so Claude and Gemini models sit behind the same API as the on-device model.
The routing logic is simple to state and worth getting right:
- Run on-device when the task is summarisation, extraction, classification, rewriting, or short structured output, when the input fits the 4,096-token window, when the data is sensitive, or when the app must work offline.
- Escalate to the cloud when the task needs a large context, multi-step reasoning, current knowledge, or a capability the small model fails at in your evaluation.
- Measure the on-device failure rate on your own tasks, not on a benchmark, and tune the escalation threshold to it.
| App type | Sensible default | Why |
|---|---|---|
| Notes, journaling, messaging | On-device first | Private content, frequent short tasks, offline value |
| Customer-support or research assistant | Cloud, on-device for drafts | Needs large context and current knowledge |
| Healthcare or fintech app | On-device where possible | DPDP and data-residency pressure |
| Media or productivity with heavy usage | Hybrid router | Volume makes the on-device cost saving real |
The one rule that saves projects: decide the split with a real evaluation on your own data before you build, because the on-device model's quality on your specific tasks, not its benchmark average, determines how much you can keep off the cloud.
How to build it without shipping twice
The cost of a hybrid design is that you now maintain two inference paths and a router between them. Done casually, that doubles the surface area for bugs. Done well, the router is a thin layer with one interface, a capability check for the device, an evaluation harness that scores on-device output against a threshold, and a single cloud fallback. Apple's decision to put third-party models behind the Foundation Models framework helps, because the on-device and cloud calls share a shape. On Android the same discipline applies across AICore and a cloud Gemini call. The teams that struggle are the ones that treat on-device as a bolt-on late in the build rather than a routing decision made at the architecture stage, when the data model and the privacy posture are still open. For teams shipping on both platforms, our Flutter app development and native iOS and Swift practices build this router once and reuse it.
FAQ
How eCorpIT can help
eCorpIT designs and builds mobile apps with the AI running in the right place: on-device for privacy and cost, cloud for capability, and a hybrid router when an app needs both. Founded in 2021 and based in Gurugram, we are a CMMI Level 5 and MSME-certified, senior-led engineering organisation, and a partner of AWS, Microsoft, and Google. We build with the Foundation Models framework, Core ML, Gemini Nano and ML Kit, and cloud APIs, and we design aligned with DPDP requirements so sensitive data stays where it should. If you are planning AI features for an iOS, Android, or Flutter app, talk to our team and we will scope the architecture and the cost model with you. See also our Android and Kotlin app development service.
References
_Last updated: 18 July 2026._
