Most people who use a chatbot every day are renting one by the message. Every prompt travels to a data centre, sits in a log somewhere, and stops working the moment the broadband drops. Run the model on your own laptop and that arrangement reverses: nothing leaves the machine, there is no monthly bill, and you can pull the network cable out mid-sentence. You do inherit the hardware limits, though, and a few minutes of checking before you download anything will save you an evening of frustration.

Check Your Hardware First

The single number that decides what you can run is memory. A model's weights have to sit in RAM or video memory while it answers, and if they do not fit, the machine starts shuffling data to disk. That is when a supposedly instant reply takes ninety seconds.

Rough guide for a chat-sized model of 7 to 8 billion parameters at 4-bit quality:

• 16GB of system RAM: a comfortable baseline for a CPU-only machine. Windows and a few browser tabs will eat 6GB before you start.
• 8GB of video memory: enough to keep a 7–8B model entirely on the graphics card, which is the difference between five words a second and thirty.
• Apple silicon with 16GB unified memory: works well, because processor and graphics chip share the same pool.
• 8GB total RAM: possible, but stick to 3–4B models and expect modest results.
• Disk space: each quantised model file runs roughly 2–8GB. Keep 20–30GB free, on the internal SSD rather than a slow external drive.

A mid-range laptop with no dedicated graphics card can still run a chatbot. It feels more like reading than talking, which is fine for drafting and summarising.

Pick the Software You Will Actually Use

The graphical route

LM Studio and Jan both offer a searchable model browser, a chat window and sliders for context length and memory use. Install, pick a model, click download, click chat. Nothing touches a terminal, and both warn you if a model is too large before you commit to it.

The command-line route

1. Install Ollama from its website; builds exist for macOS, Windows and Linux.
2. Open a terminal and pull a model — a 7–8B instruct model in a Q4 build is a sensible first choice.
3. Run it. The first start loads weights into memory and can take ten to thirty seconds.
4. Chat in the terminal, then leave it running so later questions skip the loading step.

Command-line tools use less memory overhead and are easier to script, but the graphical apps are the better starting point if you have never opened a terminal.

Understand Model Names Before You Download

Filenames look like noise, yet every part matters. The number before the B is the parameter count — 7B, 8B, 12B. Bigger models generally reason better and need more memory. The code after that, such as Q4_K_M, describes quantisation: how hard the original weights were compressed.

Q4 sits in the sweet spot for most laptops, since quality loss is small and memory use roughly halves compared with an 8-bit build. Q5 and Q6 are slightly better and heavier. Q3 and below squeeze into tight machines but start making visible mistakes with arithmetic and grammar, and a 7B model at Q4 often beats a 13B model at Q3 on both speed and answers.

Context length matters too. The context window is the working memory for your conversation, and a long one costs extra memory on top of the weights. Start at 4,000 to 8,000 tokens unless you genuinely need to paste in long documents.

The Privacy Argument, Honestly Stated

Everything you type stays on the disk. Contract drafts, medical correspondence, half-finished code from a work project, notes about a client — none of it goes anywhere. For anyone handling confidential material, that alone justifies the download.

There are other practical wins. It works on a train, on a plane, and during the broadband outage that kills your video calls. No per-message charges, no usage caps, no quiet change to a service's terms that alters how your prompts are stored.

Two caveats. First, local does not mean automatically trustworthy: download models from a well-known hub, check that the file size matches what was advertised, and avoid random executables bundled alongside a model. Second, nobody is filtering the output. A local model will cheerfully produce confident nonsense, so verify anything that carries weight. If you are using it to interpret a contract, a diagnosis or a tax rule, treat the reply as a first draft and confirm it with a qualified professional.

Six Traps That Catch First-Timers

1. Expecting a cloud-class assistant. A 7B model on a laptop is a capable writing and summarising tool, not a substitute for the largest hosted systems. Judge it on the jobs it can do.
2. Downloading too large a model. If the weights exceed available memory the machine swaps to disk, and speed collapses. Size down or quantise harder.
3. Leaving the context window maxed out. Set to 32,000 tokens, a small model can consume several extra gigabytes for the conversation alone.
4. Ignoring heat. Thin laptops throttle after a few minutes of sustained load. A stand, a hard surface and a sensible power profile help more than you would think.
5. Running on battery. Many machines cut processor power when unplugged, roughly halving output speed.
6. Forgetting the knowledge cut-off. There is no live web access by default, so ask about yesterday's news and you will get a polite invention.

Quick Wins for Speed

Before spending money, try these. Close the browser — a dozen tabs can hold more memory than your model. Reduce context length. Drop from Q5 to Q4. Keep the model loaded between sessions instead of quitting the app. If your software supports layer offloading, push layers onto the graphics chip until it runs out of video memory, then step back one. And keep the model file on internal storage: reading a 5GB file from a USB drive every time you ask a question is painful.

A Sensible First Afternoon

Check how much memory you have. Install one graphical app. Download a single 7–8B instruct model at Q4. Ask it to summarise a page of text you already know well, then rewrite an email you have drafted. Time a few responses with a stopwatch so you know what normal looks like on your machine. If answers are too slow, try a smaller quantisation before a smaller model. If they are too weak, move up a parameter size once you know your memory headroom.

Keep two or three model files rather than twenty. A quick small one for short questions and a slower, larger one for real work beats a folder of downloads you never open.

Photo: Daniil Komov / Pexels