Back to blog

Running local LLMs on a normal laptop in 2026: what's actually usable

Published on: June 4, 2026

min read

Laptop on a desk showing code in a dark editor
local LLM AI workflows Ollama privacy

I wanted to know how much you can genuinely do with a local model on an ordinary laptop in 2026 — no $4,000 GPU, no cloud bill. So I tested it. Here’s what actually fits on 8, 16, and 32GB of RAM, what quantization really buys you, and the honest line where local stops being good enough and the cloud takes over.

One warning before anything else, because it nearly tripped me up: the web is full of AI-generated articles confidently recommending models that don’t exist. I saw “Gemma 4,” “Qwen 3.5,” “Phi-5,” and “DeepSeek V4” cited as if they were real. They aren’t — none of them appear on any official model page. Every model named below I confirmed against its actual Ollama or Hugging Face page. If a guide names a version you can’t find on the vendor’s own page, assume it’s hallucinated.

The tools: Ollama, LM Studio, llama.cpp

You don’t run a model directly; you run it through a runtime. Three matter:

  • Ollama — the popular default. One install on macOS, Windows, or Linux, then ollama run <model> downloads and runs it. On Apple Silicon it uses the GPU automatically. It exposes an HTTP API too, which is what makes it nice for scripting and automation.
  • LM Studio — a desktop GUI for the same job. Better if you want to browse, download, and compare models without touching a terminal.
  • llama.cpp — the open-source C/C++ inference engine underneath both. You rarely use it directly, but it’s the thing actually doing the work.

My short version: Ollama for headless and scripted use, LM Studio for clicking around and trying things.

Quantization, in plain English

You can’t talk about local models without quantization, so here’s the no-jargon version.

A model’s weights are normally stored at high precision (16-bit). Quantization stores them at lower precision — commonly 4-bit — which shrinks the file and the RAM it needs, in exchange for a small quality loss. The format you’ll see is GGUF, and the most common setting is Q4_K_M (“4-bit, K-quant, medium”): most weights at 4-bit, the most sensitive ones kept higher. The quality drop versus full precision is usually small — low single digits on standard benchmarks — and the memory saving is large. It’s why a laptop can run a 7B model at all.

The rule of thumb that follows: at 4-bit, a 7B model needs roughly 4–5GB for weights, plus 1–2GB for runtime and context. That single estimate lets you predict what’ll fit before you download 5GB to find out.

What actually fits, by RAM tier

This is the part I actually tested. Models named here are all real and downloadable as of mid-2026.

8GB RAM — small but real

You’re in 1B–3B territory, with a 7B possible but tight (a ~4.5GB model leaves little headroom for the OS). Good fits:

  • Llama 3.2 (1B, 3B) — Meta’s small models, genuinely capable for their size.
  • Gemma 3 (1B, 4B) — Google’s; the 270M and 1B are tiny, the 4B is a solid all-rounder.
  • Qwen 3 (0.6B–4B) — Alibaba’s; a strong family at the small end.
  • DeepSeek-R1 distilled 1.5B — a small reasoning-tuned model.

Honest expectation: these are good for summarising, classifying, drafting, and simple Q&A. They are not going to debug your code or reason through anything hard.

16GB RAM — the sweet spot

This is where local gets genuinely useful. 7B–8B models run comfortably, 12–14B with care:

  • Llama 3.1 (8B) — the workhorse.
  • Mistral 7B (v0.3) — fast, Apache-licensed, ~4.4GB at 4-bit, 32K context.
  • Gemma 3 (12B) — steps up noticeably if you have the headroom.
  • gpt-oss-20b — OpenAI’s open-weight model (Apache 2.0, released August 2025). It’s a mixture-of-experts design that fits in about 16GB of memory, which makes it arguably the strongest open model you can run on a normal laptop. This was the one that genuinely surprised me.
  • DeepSeek-R1 distilled 7B/8B — for reasoning-style tasks.

32GB RAM — comfortable headroom

14B models run comfortably, and 30B-class models become possible with quantization:

  • Phi-4 (14B) — Microsoft’s open model.
  • Qwen 3 (14B / 30B) — the 30B with quantization.
  • Gemma 3 (27B) — at the upper edge.
  • DeepSeek-R1 distilled 14B/32B.

A note on what doesn’t fit: Llama 4 is real, but its smallest variant (Scout) is a 109B-parameter mixture-of-experts that needs roughly 60GB+ even at 4-bit, because all the expert weights have to be loaded. It is not a normal-laptop model, whatever an SEO listicle tells you. Same with Llama 3.3, which only comes in 70B.

Apple Silicon vs an NVIDIA GPU

One architectural point explains a lot of confusion. A discrete NVIDIA GPU can only use its own VRAM — say 24GB on a high-end consumer card. Apple Silicon uses unified memory shared between CPU and GPU, so the whole RAM pool can hold model weights and still be GPU-accelerated. The consequence: a Mac with lots of unified memory can load models that exceed any consumer NVIDIA card’s VRAM, while below that ceiling a discrete GPU is usually faster. Pure-CPU inference works but is slow — usable speed comes from either path.

The honest limitations

I’d be doing you a disservice if I pretended local matches the cloud. It doesn’t:

  • Capability gap. The models that fit a laptop (1B–14B) are smaller and weaker than frontier cloud models like Claude or GPT. Even gpt-oss’s large sibling only claims to approach a mini hosted model — and that one needs an 80GB GPU, not a laptop.
  • Speed without a GPU. CPU-only inference is slow enough to be annoying for interactive use.
  • Context costs RAM. Long context windows inflate memory use, so big contexts are constrained on small machines.
  • No tooling out of the box. You get a model and an API. The agentic tooling and integrations that come built into hosted assistants are not turnkey here.

Where local actually wins

Despite all that, there are jobs where I genuinely reach for local:

  • Privacy. Data never leaves the machine — the right call for anything sensitive (health, legal, client confidential).
  • Offline. A model that works on a plane or with no connection.
  • Cheap bulk work. Classifying, tagging, or extracting across thousands of items with no per-token bill. This is the underrated one — at volume, free-per-run local processing beats a metered API.
  • First drafts and summaries. Good enough quality, instant, private.

And where the cloud still wins, clearly: hard multi-step reasoning, very long context, and serious coding. That’s not pessimism — it’s just matching the tool to the job.

So, is it worth it?

For me, yes — as a complement, not a replacement. A 7B model on 16GB handles a real slice of my everyday tasks privately and for free, and gpt-oss-20b raised the ceiling of what “local” means more than I expected. But I haven’t cancelled anything in the cloud, and I won’t, because the hard stuff still goes there.

If you’ve got 16GB and a free evening: install Ollama, pull mistral or llama3.1, and try it on your actual work. The gap between “I read about local models” and “I know what mine can do” is one download wide.

Sources

  • Ollama — github.com/ollama/ollama and the model library at ollama.com/library
  • LM Studio — lmstudio.ai; llama.cpp — github.com/ggml-org/llama.cpp
  • OpenAI gpt-oss (20b/120b, August 2025) — openai.com/index/introducing-gpt-oss and huggingface.co/openai/gpt-oss-20b
  • Model pages verified individually: Llama 3.1/3.2/3.3, Qwen 3, Mistral 7B, Gemma 3, Phi-4, DeepSeek-R1 (ollama.com/library)
  • Quantization / GGUF references and community RAM-requirement discussions (llama.cpp)