How to Run an AI Model Locally—And Why You Might Want To

What it claims

The guide's practical content goes past its headline figures. It lists exact Ollama commands, LM Studio's minimum specs, memory figures for OpenAI's gpt-oss models and Ollama's local API port. It also flags a privacy trap: some Ollama model names run remotely. On capability, it cites Epoch AI's May 29, 2026 analysis that open-weight models have trailed the best closed models by an average of four months since January 2026. It adds that closed models won 24 of 31 tests in Alibaba's own comparison.

How to use it

Ollama install, as listed on Ollama's download page:

curl -fsSL https://ollama.com/install.sh | sh

On Windows, in PowerShell:

irm https://ollama.com/install.ps1 | iex

To download and chat with a model (type /bye to exit):

ollama pull gemma4:e2b
ollama run gemma4:e2b

Ollama exposes a local API at http://localhost:11434, so other apps on the same machine can call the model.

Names ending in :cloud, such as gemma4:cloud, run on Ollama's servers and need no download. They do not keep prompts local. The article states that these API calls are not stored or used for training, but gives no source for that claim.

LM Studio installs from lmstudio.ai and searches Hugging Face for models inside the app. Its documented requirements are:

• Apple Silicon: macOS 14 or newer with 16GB+ RAM. 8GB Macs can run smaller models.
• Windows: at least 16GB RAM, 4GB dedicated VRAM and a CPU with AVX2.
• Intel Macs: not supported.

LM Studio added MCP (Model Context Protocol) support in version 0.3.17, which gives local models web search and file access. It can run fully offline once model files are downloaded, including for documents attached to a chat.

Memory figures by model:

DeepSeek R1 is under MIT, and its model card permits commercial use and derivative works. Google's Multi-Token Prediction drafters for Gemma 4 are on Ollama under Apache 2.0. They speed up generation by up to 3x through speculative decoding: a small model drafts several tokens, and the target model accepts the whole sequence in one forward pass if it agrees, adding one token of its own in that pass.

Limits

Gemma 4's small models have a 128K-token context window and the larger ones 256K. Longer contexts need more memory. When a model spills from VRAM into system RAM, responses slow down a lot.

Qwen3.8-Max has 2.4 trillion total parameters with 95 billion active, far beyond consumer hardware. The four-month gap figure applies to the most capable open models, which are the ones that do not fit on local machines.

The 3x drafter speedup is Google's "up to" figure. The article gives no benchmark setting for it.