Can Your Mac Run a Local AI Model? An Honest Hardware Guide

Article 2 in the Bring Your Own AI Model series.

If you read the first article in this series, you already know the appeal of Bring Your Own AI Model (BYOLLM): your assistant answers using a model running on your own Mac, so day-to-day replies cost nothing per message and never leave your machine.

The obvious next question is the practical one: Will it run on the Mac I actually have?

This guide gives you a straight answer — based on what we've genuinely tested, not on a spec-sheet fantasy. BYOLLM is an advanced-user feature, and hardware is the first place people get tripped up. Let's get it right.

The one hard requirement: Apple silicon

For the model side, you need a Mac with Apple silicon — an M-series chip (M1, M2, M3, M4 and their Pro/Max/Ultra variants). This isn't a preference; it's how the local model software works. It's built for Apple silicon, full stop.

A quick but important distinction:

• Running the assistant itself on a Mac (our "Spare Mac" device option) works on Apple silicon or Intel Macs, on macOS 14 (Sonoma) or newer.
• Running a local model for that assistant to use requires Apple silicon.

So an older Intel Mac can happily host your assistant — it just can't be the machine running your local model. If you want the full BYOLLM setup, Apple silicon is non-negotiable.

One caveat we won't paper over: the local model software has its own minimum macOS version, and it's higher than the macOS 14 floor for hosting the assistant. We haven't re-verified that exact version for this article, so keep your Mac updated and check the requirements before you commit.

Why "unified memory" is the number that matters

On a Mac with Apple silicon, the processor and graphics share one pool of fast memory — Apple calls this unified memory. For running an AI model, that pooled memory is the thing that decides what's possible.

A model has to fit in memory to run well. The bigger and more capable the model, the more memory it needs. Because Apple silicon shares memory between the chip and graphics, that whole pool is available to the model — which is exactly why these Macs are good at this in the first place.

So when you look at a Mac's spec, the number to care about here isn't the chip name alone. It's the RAM (unified memory) figure.

What we actually tested (a real example)

Here's the honest, first-hand part — the reason to trust the rest of this page.

In our own testing we ran a 4-bit 26B-class model. In plain terms: a fairly capable model, compressed so it takes less memory. While running, it used roughly ~15 GB of memory.

But the model isn't the only thing using RAM. On top of it you also have:

• The small workspace the assistant runs in (~4 GB), and
• macOS itself, doing its normal job.

Add those up and you can see why headroom matters. Squeeze a 15 GB model onto a Mac with barely more than that and nothing is left for the operating system.

Our published guidance

• 32 GB Mac — comfortable for the tested case above. This is the setup we can genuinely stand behind for a capable local model.
• 16 GB Mac — possible, but only with a noticeably smaller model. Expect to trade capability for fit.
• 8 GB — that's the floor for running the assistant workspace, not for running a local model on top of it. Don't plan a local-model setup around 8 GB.

You may see hardware guides elsewhere throwing around neat "tier tables" — 8 GB experimental, 64 GB pro, 96 GB workstation, and so on. We're not going to invent those, because we haven't tested them. What we can tell you is what we measured: comfortable at 32 GB, tight-and-smaller-only at 16 GB. If you have a larger Mac, you have more headroom for a bigger model — but we won't quote performance numbers we didn't record.

How fast is it?

In our testing on an M-series Mac, replies came back in a median of about 10 seconds across a 22-turn conversation, with a range of roughly 5 to 46 seconds.

The usual honest caveat applies: speed depends heavily on your specific Mac and the model you choose. A more powerful chip and a smaller model are faster; a heavier model on a modest Mac is slower. Treat those numbers as a real-world reference point, not a promise.

The trade-offs you should plan for

BYOLLM is genuinely powerful, but running your own model is a different experience from using a top-tier cloud model. A few things to go in with eyes open:

Less-capable models struggle with complex, multi-step tasks. In our own testing, a smaller model once reported finishing a step it hadn't actually completed. The model you pick is your choice, and we don't support the model itself or your Mac's hardware — so choose with your workload in mind.

Your Mac has to stay awake and online. This runs on your machine. If it sleeps, restarts, or loses its network connection, the model stops answering. We tell every Mac user to disable sleep and, on laptops, keep the lid open.

After a reboot, the model does not restart itself. The assistant comes back, but the local model software doesn't — nothing relaunches it automatically. You'll need to start it again.

When the local model is down, your assistant does not secretly switch to the cloud. By design, there's no silent fallback to Claude. If the model stops, your assistant stops answering until you bring the model back.

You'll notice a problem before we do. When the model goes down, you see it first — a message times out after about three minutes. Our system detects the outage and sends you a single email in roughly twelve minutes. We can't promise to warn you before anything breaks.

Two things that are still required

A couple of points people often misread:

1. You still need an Anthropic key. BYOLLM makes your conversations cost nothing per message, but Dex — the AI sysadmin that keeps your assistant healthy — keeps using Claude via your Anthropic key (far more cheaply than full conversations). "No key at all" is something we're thinking about for later; it does not exist today. So this isn't "fully free," and we won't pretend otherwise.

1. Your subscription doesn't change. BYOLLM is a setting, not a plan. Your Launch My AI subscription stays exactly the same. (You do still pay for your own AI usage and, if you run in the cloud instead, your own server — but BYOLLM specifically doesn't add a plan cost.)

Two supported setups — and one that isn't

For BYOLLM, exactly two arrangements are supported:

• Same Mac: your assistant and your model both run on one Apple-silicon Mac.
• Same home network: your assistant runs on a spare computer or Raspberry Pi, and your model runs on an Apple-silicon Mac on the same home network.

Reaching your home model from somewhere else over the internet is not something we ship. If a guide tells you to point your existing cloud assistant at a Mac at home, that's not a setup we support.

So — can your Mac run it?

Here's the short version:

• Apple silicon? Required for the model. Intel can host the assistant only.
• 32 GB unified memory? Comfortable for a capable model, based on our real testing.
• 16 GB? Workable with a smaller model.
• Willing to keep the Mac awake, restart the model after reboots, and choose a model that suits your work? Then you're a good candidate for BYOLLM.

If that sounds like your setup, the Bring Your Own AI Model page walks through turning it on. And because this is an advanced feature, take your time — the payoff is a private assistant whose everyday answers run entirely on hardware you own.

Next in the series: choosing a model that fits your Mac — and your work.