Technology

On-device AI vs cloud AI: what changes for your phone

Some AI features now run entirely on the chip inside your phone; others still send your request to a data centre. Here is how to tell the two apart, why memory decides which phones qualify, and what each approach does to privacy, battery and data use in India.

Chander Prakash

By Founder & Editor

9 min read

Illustrative cover: On-device AI vs cloud AI: what changes for your phone
Illustration: Pointales

“On-device AI” means the AI model does its work on the chip inside your phone. “Cloud AI” means your request travels to a server in a data centre, is processed there, and the answer comes back. Since 2024, small models of one to four billion parameters, plus dedicated chips called NPUs, have made useful local AI possible. Most phones now use both, and the mix decides your privacy, your battery, your data bill and whether a feature works on a train with no signal.

What “on-device” actually means

A language model is a large grid of numbers, called parameters, that text is multiplied through to produce an answer (our guide to how large language models work explains the mechanics). The big chatbot models have hundreds of billions of parameters and run on racks of data-centre GPUs. An on-device model is the same idea, shrunk to fit a phone. Apple’s 2025 foundation models report describes “a ~3B-parameter on-device model optimized for Apple silicon”, alongside a much larger server model. Google’s Gemini Nano “runs in Android’s AICore system service, which leverages device hardware to enable low inference latency”, according to Android’s developer documentation.

The CPU and GPU can run models, but modern chips add a neural processing unit (NPU), a block built for the repetitive multiply-and-add arithmetic of AI. Microsoft’s Copilot+ PC guide describes it as using “energy on AI tasks more efficiently than a CPU or GPU” and sets a floor of “more than 40 trillion operations per second (TOPS)” for that class of laptop. Phone-chip makers rarely publish TOPS: Qualcomm’s Snapdragon 8 Elite Gen 5 page claims an NPU 37% faster and 16% better per watt than its predecessor; MediaTek’s Dimensity 9500 page claims 1.58-bit “BitNet” model support and 128K-token text processing on the phone. Both are vendor claims, not independent benchmarks.

Why phones can do this now

Models got small. A carefully trained model of 2–4 billion parameters can summarise, rewrite, translate and answer short questions well enough for daily use. Microsoft’s Phi-3 technical report (April 2024) shows it: its 3.8-billion-parameter phi-3-mini “can be quantized to 4-bits so that it only occupies ≈1.8GB of memory”, and the authors ran it on an iPhone 14 “natively on-device and fully offline” at “more than 12 tokens per second”.

Weights got compressed. Quantisation stores each parameter in fewer bits. At 16 bits a parameter takes two bytes; at 4 bits, half a byte, so a 3-billion-parameter model shrinks from about 6GB to about 1.5GB. Apple’s June 2024 model post describes “a mixed 2-bit and 4-bit configuration strategy — averaging 3.7 bits-per-weight”. Google’s Gemma 3n (June 2025) has 5- and 8-billion-parameter models built to run “with as little as 2GB (E2B) and 3GB (E4B) of memory”.

Chips got NPUs, built on the most advanced manufacturing processes, which is why launches lead with “3nm” (see what 3nm and 2nm really mean).

Why 8GB vs 12GB RAM matters more than the chip

The practical limit is memory, not speed. The model’s weights must sit in RAM while it runs, alongside the operating system, your apps and the working memory for the text being processed. A 1.5–2GB model on a 6GB phone leaves little room; on 8GB it is workable; on 12GB there is space for a bigger or less-compressed model. Vendors do not say this directly, but their requirements do. Apple’s support page for the current Apple Intelligence lists two storage tiers: “Up to 14 GB of storage on device” for the iPhone 17 Pro, iPhone 17 Pro Max, iPhone Air, and iPads and Macs with “at least 12GB of unified memory”, and “Up to 8 GB” for other eligible devices, which start at the iPhone 15 Pro. More memory, bigger model.

On Android, Google publishes a device list for its on-device Gemini Nano features. As of October 2026 it is dominated by flagships, grouped into “nano-v2”, “nano-v3” and “nano-v4” tiers, and the cheapest entries are upper-mid-range models such as the POCO X7 Pro and vivo T4 Ultra. Google notes that “different versions of Gemini Nano may return different output from the same prompt”: the same feature name can mean a weaker model on a cheaper phone.

What each approach is good at

On-deviceCloudHybrid
LatencyNo network round trip, but speed depends on the phone’s chipFast on good networks; stalls on bad onesLocal where possible, cloud for the rest
PrivacyData stays on the phone; not used for training, per Samsung and GoogleYour text or image leaves the phone; terms govern retention and trainingDepends on which path a request takes
OfflineWorks without a connectionDoes not workLocal features work; others wait for signal
CapabilityShort tasks: summarise, rewrite, translate, describe an imageLong reasoning, large documents, coding, toolsHard requests go to the bigger model
CostNo per-request cost; paid for in RAM and storageA server cost per request, funded by subscriptions or adsLower cloud bills; more engineering
FreshnessModel updates arrive with OS or system updatesProvider can update the model any timeMixed

Privacy is the strongest on-device argument. Android’s AICore “doesn’t store any record of the input data or the resulting outputs after processing them”, according to Google. Samsung’s Knox documentation states that on-device features “don’t use your data for training machine learning models”, while cloud features “may be used for model training”. Under the DPDP Act, whether a company processes your personal data at all is the first question, so expect “on-device” to become a compliance claim as much as a performance one.

Capability is the strongest cloud argument. Google’s own guidance says Nano “has on-device constraints such as token limits, device hardware limitations, and model capability boundaries”. A small model is also more likely to invent answers on hard questions, so our guide to why AI chatbots hallucinate applies with extra force on-device.

How hybrid designs work

Apple’s Private Cloud Compute (PCC), announced in June 2024, is its answer to the privacy cost of the cloud. Apple’s privacy disclosure says many features run “entirely on device”; when a task needs more power, “Apple Intelligence may send your request to Private Cloud Compute”. Its stated design rules include “stateless computation on personal user data”, “no privileged runtime access” and “verifiable transparency”, with the promise that “no user data is retained in any form after the response is returned” and that it “isn’t accessible to anyone other than the user — not even to Apple”. Apple publishes the server software so researchers can check; these remain Apple’s claims.

Google’s approach is a routing rule. Its hybrid inference guide lets an app run “on-device whenever possible” and, “if the on-device model isn’t available”, send the request to a cloud Gemini model; developers can also route by “device system health (for example battery levels and processor load)” and “user query complexity”, and Google notes that doing routine tasks locally “helps reduce cloud inference costs”.

Samsung ships the choice as a switch. Its Galaxy AI support page describes a “master switch” at Settings > Galaxy AI > “Process data only on device”, which “disables all cloud-based processing”. Translation and Google Messages writing features are listed as on-device; “auto formatting, auto summarizing, auto corrections, and image editing” are cloud-based, and “most other features require a network connection”. The split varies by model.

What it means for battery, storage and your data

Storage. On-device models are downloaded to your phone: up to 8GB or 14GB on Apple’s current requirements. On Android, Google says AICore lets apps share one copy of Nano, which “conserves storage space”.

Battery. Running a model locally draws from your battery rather than a data centre’s. The NPU exists to make that affordable, yet Google’s routing guidance still treats “battery levels” as a reason to send work to the cloud.

Data. A cloud request costs mobile data both ways; text is tiny, images and audio are not. India’s data is cheap: TRAI’s yearly performance indicators for FY2025-26, released in October 2026 and reported by TelecomTalk, put operators’ average revenue at ₹8.02 per GB, with the average user consuming 25.51GB a month. Here, coverage, not cost, is the reason to prefer on-device: offline translation on a train is the practical win.

What it means in India specifically

Most phones sold here are not on the list. India’s market is weighted towards budget and mid-range devices with 6–8GB of RAM, which Google’s Nano list and Apple’s iPhone 15 Pro floor exclude. “AI phone” marketing on a ₹15,000 device usually means cloud features, camera processing, or a thin layer over a cloud chatbot.

Indian languages are mostly a cloud story so far. Google’s Gemini app supports nine Indian languages, but that is the cloud app; Google says Nano’s language support “may vary depending on the particular device’s configuration”. Sarvam’s Sarvam-1 (October 2024) is a 2-billion-parameter model for ten Indian languages plus English, with a tokeniser the company says needs 1.4–2.1 tokens per word, against the “4 to 8 tokens per word” it says other multilingual models need for Indic text; that matters on a phone, where a small model’s limits are counted in tokens. Sarvam’s claim that it is “well-suited for” edge devices is a vendor claim, not a shipping phone feature, as of October 2026.

How to tell what a feature does from a product page

Product pages rarely say “this sends your data to our servers”. They do leave signals.

  1. Look for “on device” or “locally”, and their absence. Apple’s privacy page names features that run “entirely on device”; Samsung’s support page sorts features into on-device and cloud. If a description says nothing about where it runs, assume cloud.
  2. Look for “requires an internet connection”. Samsung’s page says exactly that for its photo-editing features; a feature that needs a connection is not on-device.
  3. Check the device list, not the brand. Google’s and Apple’s lists say which models qualify, and which Nano tier.
  4. Find the switch. If a setting exists to block cloud processing, turn it on and see which features go grey; that is the honest map.
  5. Read RAM before TOPS. A TOPS number without a stated test is marketing; RAM decides whether the better model runs at all.

The point: On-device AI keeps your data on the phone, works offline and costs the app maker nothing per request, but it is limited to small models, and only phones with enough RAM get the good ones. Cloud AI is more capable and works on any phone, at the price of sending your data out and needing signal. Most “AI phones” use both, so the useful questions for a buyer are which features stay local, what the switch to force that is, and how much memory the phone has.

Sources

Share