Local or cloud: where your AI should run
Running AI on your own machine went from hobby to practical.
The question is no longer whether you can, but which work belongs where.
The trade
| Cloud assistants | Local models | |
|---|---|---|
| Capability | The strongest models available | Smaller models, closing the gap on everyday tasks |
| Privacy | Data goes to the provider under their terms | Never leaves your machine |
| Cost | Subscription or per use | Free to run once you own the hardware |
| Setup | Sign in and go | An app install and a model download |
| Offline | No | Yes |
The tools
Hardware math in one rule
Memory decides what you can run.
| Memory available | Comfortable model size at 4-bit |
|---|---|
| 8 GB | Around 3–4B parameters |
| 16 GB | Around 7–14B |
| 32 GB | Up to around 30B |
| 64 GB and up | 70B-class models and large mixture-of-experts |
A model quantized to 4 bits needs about half a gigabyte per billion parameters, plus headroom for the context and your other apps. On Apple Silicon, unified memory counts. On a PC, the model should fit in your graphics card's VRAM to run fast.
Total versus active parameters
Many new open models use a mixture-of-experts design:
a large total parameter count, with only a small slice active for each token. Memory has to hold the total. Speed follows the active count. That is why some big models run surprisingly fast on a well-equipped laptop.
What we recommend
Go hybrid. Keep a cloud assistant for hard reasoning, research, and anything that needs current information.
Install LM Studio, download a small instruct model that fits your memory, and ask it to summarize a document you would never upload. That one test tells you whether local belongs in your workflow.
Use a local model for private documents, bulk drafts, and work you want to do offline.
Resources
Every link this breakdown leans on, in one place.