Run AI Models Locally on Windows: Ollama, LM Studio and Hardware Requirements

You can run AI models locally on Windows today with two free tools (Ollama and LM Studio) and any PC with 16 GB of RAM or an 8 GB graphics card is enough for genuinely useful models like Llama 3.1 8B or Qwen3 8B. No subscription, no internet connection, no data leaving your machine. The only real constraint is memory: at the standard 4-bit quantization, a model needs roughly 0.6 GB per billion parameters.

What do you need to run AI models locally on Windows?

Three things: the right OS version, enough memory, and a tool that loads models. That’s the whole checklist.

A local LLM is a language model whose weights live on your own drive and run on your own CPU and GPU: nothing is sent to a server. Ollama’s official Windows requirements are Windows 10 22H2 or newer (Home or Pro), an NVIDIA driver of version 551.61 or newer if you have that brand of card, and at least 4 GB of disk for the installer itself: models add “tens to hundreds of GB” on top, per the docs. No administrator rights are needed; the installer drops into your user directory.

Memory is where people misjudge. VRAM is your graphics card’s memory: the fast lane for inference; RAM is slower but bigger, and both tools will split a model across the two. The rule of thumb, documented in local-LLM engineering guides: about 0.6 GB per billion parameters at 4-bit precision, ~1 GB at 8-bit, 2 GB at full precision, plus headroom for the KV cache, the scratch memory that grows as context gets longer.

Rule of thumb: at 4-bit quantization a model needs ~0.6 GB of memory per billion parameters. A 7B model fits in ~5 GB, a 32B model in ~20 GB, before context overhead.

Ollama vs LM Studio: which should you install?

Both are free, both run the same underlying engine (llama.cpp), and both can serve OpenAI-compatible APIs. They differ in surface and license.

Ollama is open source under the MIT license, so you can use it commercially without asking anyone. Since version 0.10 it also ships an official desktop app for Windows and macOS that handles downloading and chatting without the terminal. Its sweet spot is being a quiet background service: an API on localhost:11434 that editors, scripts, and agent frameworks can call. As of November 2026 the current release is v0.40.15, published November 3 per the project’s GitHub releases. Nuance: Ollama sells a separate paid cloud service, but local use stays free.

LM Studio is a polished GUI app, current stable version 0.4.28 (October 31, 2026, per the official changelog). It’s proprietary, but since July 8, 2025 it’s been free to use at work: the company removed the old commercial-license requirement. Version 0.4.0 in January 2026 added parallel requests with continuous batching and a headless deployment mode (“llmster”) for servers. The docs list chat with local models, Hugging Face search and download, an OpenAI-compatible server, MCP tool-use support, offline document chat (RAG), and speculative decoding.

A quick way to decide:

• Want to chat and experiment: compare models side by side, tune context and GPU sliders? → LM Studio.
• Want to build something: scripts, coding assistants, automations calling a local API? → Ollama.
• Want the open-source stack or a lighter GUI? → Ollama, or Jan (Apache 2.0, actively developed) as a GUI alternative.

What about GPT4All? Its last release was February 25, 2025 and its GitHub has seen no code pushes since May 2025: community threads openly ask whether it’s dead. Nomic has posted no sunset notice, so treat it as dormant rather than dead, and do not start there in 2026.

How much RAM and VRAM do you need for local LLMs?

Match the model to your memory. Here’s the tier-by-tier picture for 4-bit quantized models: speeds are community benchmarks, not vendor claims, since the docs don’t publish official tokens-per-second tables.

Hardware tier	Typical setup	Models that fit (Q4)	Rough generation speed
Entry laptop, 8-16 GB RAM, no GPU	CPU or integrated graphics	Gemma 4 E2B/E4B, Qwen3 4B, Llama 3.2 3B	~5-15 tok/s on CPU
Mainstream desktop	16 GB RAM + 8-12 GB VRAM GPU (RTX 3060-class)	Llama 3.1 8B, Qwen3 8B, Gemma 3 12B, Qwen2.5-Coder 14B	~15-45 tok/s
High-end PC	32 GB RAM + 24 GB VRAM GPU (RTX 3090/4090-class)	Gemma 3 27B, Qwen3 32B, Mistral Small 3.2 24B, DeepSeek-R1-Distill-32B	~35-50 tok/s for 27-32B
Enthusiast / workstation	64 GB+ combined memory or multi-GPU	gpt-oss-120b (quantized), Llama 4 Scout (~61 GB at Q4), Qwen3.5 mid-size MoE	varies widely by config

Three things bend these numbers. First, speed is mostly memory-bandwidth-bound, which is why a 4090-class card posts ~100-150 tok/s on an 8B Q4 model while a CPU manages single digits. Second, context length costs memory: long chats can add several GB. Third, tiers blur if you split layers between RAM and VRAM: slower, but it’s how people run 27B models on 16 GB machines at tolerable speeds.

Buying hardware for this? The 12 GB VRAM class (RTX 3060 12 GB) is the value sweet spot for 8B-14B models; 24 GB opens the whole 27-32B range.

The best local models to download right now (November 2026)

The Ollama library is the quickest pulse check on what people actually run. llama3.1 leads with about 120.9 million pulls, deepseek-r1 has about 94.8 million, and the newer generations are climbing fast: gemma4 (28.8M), qwen3.5 (24.8M), and OpenAI’s gpt-oss (14.8M) are all in the top tier.

Shortlist by job:

• General chat: Gemma 4 (Google, Apache 2.0, April 2, 2026) spans E2B/E4B edge sizes to 26B MoE and 31B dense, with 128K context on small models. Qwen3 (0.6B-32B dense plus 30B-A3B MoE) is the strongest per-parameter generalist family; successor Qwen3.5 launched February 2026 with a 397B-A17B MoE flagship, with library sizes down to 0.8B.
• Laptops and old PCs: Gemma 4 E2B/E4B, Qwen3 4B, or Llama 3.2 1B/3B. These run on CPU and iGPUs.
• Coding: Qwen2.5-Coder (0.5B-32B) is the established local coding workhorse, and Qwen3-Coder 30B (an MoE) is the newer step up.
• Reasoning: DeepSeek-R1 distills (1.5B up to 671B in Ollama’s library) and gpt-oss-20b. OpenAI’s open-weight reasoning model.
• One-model-for-everything on a 24 GB GPU: Mistral Small 3.2 24B (Apache 2.0) or Gemma 3 27B. The 32B DeepSeek-R1 distill also fits at Q4.
• Money-no-object quality: Llama 4 Scout is a 109B MoE needing ~61 GB of VRAM at 4-bit (community measurements); Maverick needs ~224 GB. Datacenter-hobbyist territory, not desktops.

Two practical notes. Llama 3.1 8B is old (2024) but still Ollama’s most-pulled model, and a fine default at the 8B tier. And for MoE models like Qwen3-Coder 30B, the “30B” is total parameters while only a few billion are active per token, so they punch above their memory class.

How to set up Ollama on Windows (step by step)

The whole thing takes about ten minutes, most of it download time.

1. Check your specs. Windows 10 22H2 or newer, and if you have an NVIDIA card, driver 551.61 or newer (per the official docs). Update your GPU driver first if it’s old.
2. Download the installer from ollama.com/download and run OllamaSetup.exe. No admin rights needed; it installs to your home directory.
3. Confirm it’s running. Ollama starts in the background automatically. Open a terminal and run ollama --version: you should see v0.40.x as of November 2026.
4. Pull your first model. Run ollama pull llama3.1:8b (or ollama run llama3.1:8b, which pulls and starts a chat in one step). On 16 GB RAM this is the comfortable default.
5. Chat in the terminal or the app. Type a message and get a reply locally. The official desktop app gives you the same models in a window if you’d rather not use the CLI.
6. Try the API. The server listens on http://localhost:11434: point any OpenAI-compatible client at it. This is what makes Ollama worth keeping installed even if you mainly use LM Studio.
7. Set your model storage location before you accumulate 100 GB. Models land on your C: drive by default; set the OLLAMA_MODELS environment variable and restart Ollama if you want them on another drive.

How to set up LM Studio on Windows (step by step)

1. Download the Windows x64 (or ARM64) build from lmstudio.ai and install it.
2. Let it detect your hardware. It identifies your GPU and offers matching runtimes. CUDA for NVIDIA, Vulkan for AMD and Intel. Grab the runtime it suggests.
3. Search for a model. Use the built-in Hugging Face search, filter by size that fits your memory, and prefer Q4_K_M quantized GGUF files as the default choice.
4. Download and load it. Click download, then load the model into memory when it finishes.
5. Chat, and tune only if needed. The default settings are sane. If the model feels slow, lower the GPU offload slider; if you need longer conversations, raise the context slider and watch the memory estimate update.
6. (Optional) Start the local server. The Developer/Server tab exposes OpenAI-compatible endpoints on your network: same idea as Ollama’s API, with a GUI to manage it.
7. (Optional) Add MCP servers if you want the model to use tools, or try headless “llmster” mode if you’re putting it on a box with no desktop.

Do you need an NVIDIA GPU? AMD and Intel reality check

NVIDIA is the path of least resistance: CUDA support is mature in both tools, and Ollama’s docs simply ask for driver 551.61 or newer.

AMD works on Windows, with one asterisk per tool. Ollama supports AMD GPUs through a ROCm v7/HIP7-capable driver stack or through Vulkan, now enabled by default in the docs as the fallback: older RX 6000-series cards are the ones most likely to need workarounds. LM Studio accelerates Radeon and Intel Arc cards through its Vulkan engine, and a 0.3.14 release added per-GPU controls for mixed systems; one open tracker bug notes Vulkan device selection misbehaving in mixed NVIDIA + AMD/Intel setups. Intel Arc is on Vulkan in both tools: serviceable, but clearly the third-supported vendor.

If you’re choosing a GPU for local AI, NVIDIA first, a recent AMD Radeon second. If you already own an AMD or Intel card, try Vulkan before buying anything: recent RX 7000/9000-series owners report good results.

What local models still can’t do in 2026

Local is genuinely good now: everyday writing, summarizing, coding help, document Q&A. Where it still loses to frontier cloud models:

• Hard reasoning. Frontier cloud models keep a real edge on complex multi-step problems; hybrid local-first/cloud-fallback setups exist precisely because of this gap.
• Long contexts. Small local models degrade over very long documents; 7B-14B models are fine for short tasks and lose precision on extended sessions.
• Current information. No built-in web search or fresh training data unless you add retrieval tooling yourself.
• Agentic reliability. Multi-step agent work is where small models stumble most.
• Throughput for heavy use. Your hardware caps speed; a cloud datacenter doesn’t.

The privacy, cost, and offline wins are real. For anything sensitive (client documents, personal records, proprietary code) local is the only choice that guarantees nothing leaves the building.

Which setup should you pick?

• 16 GB RAM laptop, no GPU: LM Studio + Gemma 4 E4B or Qwen3 4B. Usable, not fast.
• Desktop with a 12 GB GPU: Either tool + Llama 3.1 8B or Qwen3 8B at Q4_K_M. This is the sweet spot for most people.
• 24 GB GPU: Add Mistral Small 3.2 24B or a 32B reasoning distill and you’re covered for serious single-user work.
• Developer or builder: Ollama + the API, with LM Studio as your experiment bench.
• Business use: Both tools are cleared for work use; check model licenses. Apache 2.0 families (Gemma 4, Qwen, Mistral Small) are friction-free. Keep a cloud fallback for hard reasoning.

FAQ

Can I run AI models locally on a Windows PC?

Yes. Windows 10 22H2 or newer is the official baseline for Ollama, and LM Studio supports x64 and ARM64 Windows. You don’t need a GPU: CPU-only inference works, just slowly. Practically, 16 GB of RAM runs 8B-12B models at usable speed, and an 8 GB+ VRAM graphics card makes everything noticeably faster. Both tools are free downloads.

Which is better for Windows, Ollama or LM Studio?

Pick LM Studio if you want a graphical app: browse, download, and chat with models, tweak settings with sliders, and expose an OpenAI-compatible server when needed. Pick Ollama if you want a background service and a simple API on localhost:11434 for scripts, editors, and agents. Many people install both; the model files are separate but the experience overlaps.

What are the best local models to run in 2026?

For general use: Gemma 4 (E2B to 31B), Qwen3 (0.6B-32B dense), and Llama 3.1 8B. For coding: Qwen2.5-Coder or Qwen3-Coder 30B. For reasoning: DeepSeek-R1 distills and gpt-oss-20b. For a 24 GB GPU: Mistral Small 3.2 24B or Gemma 3 27B. Match the model size to your memory using the 0.6 GB-per-billion-parameter rule at 4-bit quantization.

How much RAM do I need to run a local LLM?

A 4-bit (Q4_K_M) model needs roughly 0.6 GB per billion parameters plus 2-4 GB for the operating system and context. That means 16 GB of RAM handles an 8B model, 32 GB handles a 32B model on CPU, and 70B-class models realistically need a 24 GB GPU plus 64 GB of system RAM. Context length adds memory on top, sometimes several GB.

How much does it cost to run AI models locally?

The software is free: Ollama is MIT-licensed open source, LM Studio is free for personal and work use, and Jan is open source (Apache 2.0). Models are free downloads too. The real cost is hardware you may already own, or a GPU upgrade (roughly a 12 GB card for comfortable 8B-14B use, a 24 GB card for 27B-32B models). Ollama sells a separate paid cloud service, but local use stays free.

Do local models work with AMD or Intel GPUs on Windows?

Yes, with caveats. Ollama supports AMD GPUs through ROCm v7 drivers or Vulkan, and LM Studio runs AMD Radeon and Intel Arc cards through its Vulkan engine. NVIDIA’s CUDA remains the smoothest path. Older AMD cards can need driver tweaks, and one known LM Studio limitation is flaky Vulkan device selection in mixed-GPU systems.

What can local models not do compared to ChatGPT?

Local models lag frontier cloud models on hard reasoning, degrade over very long documents, have no built-in live web access, and are less reliable at multi-step autonomous agent work. They also cap out at whatever your hardware fits. They win on privacy, offline use, no per-token limits, and zero subscription cost.

Can I use Ollama and LM Studio for commercial work?

Yes. Ollama’s MIT license permits commercial use. LM Studio dropped its work-license requirement on July 8, 2025 and is now free for internal business use, though it remains proprietary rather than open source. Check each model’s license separately: Gemma 4 and Mistral Small are Apache 2.0, while Meta’s Llama models carry their own community license.

What is quantization and why does Q4_K_M matter?

Quantization compresses a model’s numbers from 16-bit to 4-bit so it fits in less memory. Q4_K_M is the community’s default 4-bit format in GGUF files: roughly 73% smaller than full precision with minor quality loss for most tasks. Downloads named Q4_K_M are the safe default; Q6_K or Q8_0 trade size for a little more accuracy.

What should I install and download first?

Install LM Studio if you want the gentlest start, and download one 8B-class model such as Llama 3.1 8B or Qwen3 8B at Q4_K_M. Chat with it a little, then try a smaller model (Gemma 4 E4B) if it feels slow. If you want to wire AI into scripts or an editor, install Ollama and pull the same model with one command. Add a coder or reasoning model once the basics work.

Sources

• Ollama Windows documentation: official system requirements, installer behavior, OLLAMA_MODELS
• Ollama GitHub releases: v0.40.15, November 3, 2026
• Ollama model library: model sizes and pull counts (checked November 8, 2026)
• Ollama’s new desktop app announcement
• LM Studio docs: platforms, features, MCP and headless support
• LM Studio free-for-work announcement. July 8, 2025 licensing change
• LM Studio changelog: v0.4.28, October 31, 2026
• LM Studio Desktop App Terms
• Qwen3 announcement: model sizes and context lengths
• Qwen3.5 announcement. February 2026, first 397B-A17B model
• Gemma 4 announcement (Google blog). April 2, 2026
• How to run Gemma 3 locally (getdeploying), per-size memory figures
• RAM requirements for local LLMs (apxml)
• LLM VRAM requirements (Spheron)
• DeepSeek-R1-Distill-Qwen-32B model card (Hugging Face)
• Mistral Small 3.2 24B memory estimates (willitrunai)
• GPT4All “Is GPT4All dead?” issue (GitHub)
• Jan (jan.ai): open-source local LLM app
• Inference speed benchmark resources (r/LocalLLaMA)
• What a local LLM can and can’t replace (XDA, February 2026)