How to run AI locally on your own computer or phone
Download free AI models and run them offline, so your information never leaves your device. For people who value privacy or control and don't mind something a bit technical.
Who it’s for: Advanced users, developers and privacy-conscious professionals.

Most AI chatbots run on a company's computers, so what you type is sent over the internet. Open models offer another way. You download the model itself and run it on your own laptop or phone, even with no internet connection. Your notes and documents never leave your device.
This course starts with the basics. You'll learn the difference between closed models and open-weight models, which you can download and run yourself. You'll also see what makes a model fully open source. Then you'll weigh the trade-offs, because local models are private and free to run but often slower or less capable. You'll meet the main families, such as Gemma, Qwen, DeepSeek, Mistral and Llama. Then you'll run a small model on your phone with the internet switched off, and chat with one on your computer in LM Studio.
Later lessons cover matching a model's size to your computer's memory and running models with Ollama. You'll also read a model licence before you build on it. You'll ask a local model about a private document and check each answer against the passage it quotes. The course is for privacy-minded people, developers and professionals who don't mind something a bit technical.
By the end you’ll be able to…
- Explain open weights versus open source and check a model licence
- Run a Gemma or Qwen model offline on your laptop
- Pick a model size and quantisation that fits your memory
- Answer questions over private documents without uploading them
Short on time? Start with the essentials
These 4 lessons cover the most useful ideas in this course. Each one takes about a minute.
Open weights, open source, closed: what you can do
Explain the difference between closed models, open-weight models and fully open-source models, and what each lets you do.
Why run AI on your own device
Weigh the benefits of local AI (privacy, offline use, no subscription) against the trade-offs in quality and speed.
Download and chat with a model in LM Studio
Install LM Studio, pick a model that fits your machine and chat with it fully offline.
Match model size to your memory
Estimate the memory a model needs from its parameter count and quantisation, and pick one your device can run.
Every lesson in this course
This course has 11 video lessons of about a minute each, from the basics to expert habits. Each one ends with a question, three takeaways and its sources.
Foundations 2 lessons
Open weights, open source, closed: what you can doEssential
Explain the difference between closed models, open-weight models and fully open-source models, and what each lets you do.
“You can download some of today's strongest AI models free. Here's the catch.”
Why run AI on your own deviceEssential
Weigh the benefits of local AI (privacy, offline use, no subscription) against the trade-offs in quality and speed.
“On a plane with no Wi-Fi, a local model still answers. Your data never leaves.”
Building skills 3 lessons
Meet the families: Gemma, Qwen, DeepSeek, Mistral, Llama
Name the major open-weight families (Gemma, Qwen, DeepSeek, Mistral, Meta's Llama and Muse), who makes them and what each is known for.
“Five big families, dozens of sizes. Here's who makes what, and what each is good at.”
Run a Gemma model offline on your phone
Install an on-device AI app, download a small Gemma model and use it in airplane mode.
“Airplane mode on. Ask your phone to summarise a note. It still works.”
Download and chat with a model in LM StudioEssential
Install LM Studio, pick a model that fits your machine and chat with it fully offline.
“No terminal needed: pick a model, click download, start chatting offline.”
Advanced 3 lessons
Match model size to your memoryEssential
Estimate the memory a model needs from its parameter count and quantisation, and pick one your device can run.
“A 31B model needs about 70 GB at full precision, but about 18 GB at 4-bit.”
Run and script models with Ollama
Run a model with Ollama, switch models and call it from a short script through its local API.
“One command, ollama run, and a model is answering in your terminal.”
Read a model licence before you build on it
Compare Apache 2.0, MIT and custom community licences and spot restrictions on commercial or large-scale use.
“Apache 2.0, MIT, 'community' licence: one of these can limit big companies.”
Expert 3 lessons
Ask a local model about a private document
Attach a private document to a local model in an app such as LM Studio, ask questions offline, and check each answer against the passage it quotes.
“Your contracts, your laptop, zero uploads: build private document Q&A.”
Local, hosted open or frontier API: decide
Choose between a local open model, a hosted open model and a frontier API based on quality, cost, privacy and control.
“The best open models trail the frontier by months, not years. Is that enough?”
Test a local model on your own tasks
Build a small test set from your real tasks and compare two local models before relying on either.
“Leaderboards can't tell you how a model handles your invoices. Test 10 of them.”
Apps we recommend
These are the tools this course uses, and what each one is best for. We have no partnership with any of them.
Ollama
Best for: Running local models from the command lineOne command downloads and runs open models; local API for your own scripts and apps.
FreeLM Studio
Best for: Running local models with no terminalDesktop app to browse, download and chat with open models, with a local server.
FreeHugging Face
Best for: Finding and comparing open modelsThe main hub for open model weights, model cards, licences and datasets.
FreemiumRun a private AI assistant on your own machine
Choose a real task you would rather keep off the cloud, such as summarising notes or drafting from private documents, using only data you are allowed to use. Pick two open models that fit your hardware, check their licences, and test both on ten of your own tasks offline.
What you’ll make: A one-page setup note: your hardware, two models with size, quantisation and licence, test scores and your pick.
Check your own work
- Each model's estimated memory need is recorded next to what it actually used
- Each licence is checked for your intended use, with any restrictions noted
- Both models are scored on the same 10 tasks against a pass bar set in advance
- Shown to work with networking off
- The note says which tasks you would still send to a hosted model, and why
Further reading and viewing
We’ve chosen these articles, research papers, videos and guides from other trusted sources.
Ollama documentation
Official docs for installing Ollama, running models and using its local API.
LM Studio docs
Official guide to downloading models, chatting offline and running a local server.
Gemma 4: Byte for byte, the most capable open models
Google's open-weight family with phone, laptop and workstation sizes under Apache 2.0.
LLM Course
Free hands-on course on using, fine-tuning and sharing open models.
Open weights model leaderboard
Independent benchmarks comparing open-weight models on quality, speed and price.
Questions about this course
Can I run AI on my own computer?
Yes. Free apps such as LM Studio and Ollama let you download an open model. You can then chat with it on your own computer, even offline. You'll need enough memory, though smaller models run on ordinary laptops. They're usually not as capable as the biggest online assistants, but they're private and cost nothing to run once downloaded.
What is an open-weight model?
A model's weights are the numbers it learned during training. When a company releases those weights, anyone can download the model and run it on their own hardware. That's an open-weight model. It isn't always fully open source, because the training data and code may stay private. The licence matters too, since some restrict commercial or very large-scale use.
How much memory do I need to run an LLM locally?
It depends on the model's size and how compressed it is. An LLM, or large language model, is measured in parameters, and bigger models need more memory. A technique called quantisation shrinks a model so it fits on smaller machines, with some loss of quality. The course shows you how to estimate the memory a model needs before you download it.
Is local AI completely private?
Running a model on your own device means your prompts and documents aren't sent to an AI company. That's a big privacy gain. But privacy also depends on the app you use, where your files are stored and whether anything on your computer syncs to the cloud. Download models and apps only from their official sources, and read the app's privacy settings.
Do I need to be a programmer for this course?
No, but you should be comfortable installing apps and trying something a bit technical. The early lessons use point-and-click apps on your phone and computer. Later lessons introduce a simple command-line tool and a short script for those who want them. Each lesson takes about a minute, and your first five lessons in Tau are free.
Get early access to this course
Tau opens soon. Join the waitlist and we’ll email you as soon as you can start this course.
We’ll only email you about Tau, and you can unsubscribe any time. Privacy policy.
We’ll email you as soon as Tau opens.


