How to Run Your AI Assistant on a Local Model on Your Mac: A Step-by-Step Setup Guide

This is the third post in our series on running your assistant on your own Mac's AI.

• The overview explained what "bring your own model" means and who it's for.
• The hardware guide helped you decide whether your Mac has enough memory.
• This post covers setup: what to prepare, which settings to choose in the dashboard, and what to watch for once it's running.

One thing first. In the dashboard this feature is called "Use your own computer's AI," and it's labelled Advanced users only. That label is accurate. Most people are better served by the standard setup, where your assistant uses Claude through your own Anthropic account. If you have a capable Mac, want conversations that cost nothing per message, and don't mind some upkeep, read on.

Before You Start: A Quick Checklist

You'll need all of the following:

• An Apple-silicon Mac with enough memory. A large model needs about 15GB of memory to itself.
• It runs comfortably on a 32GB Mac.
• A 16GB Mac works only with a noticeably smaller model.
• Intel Macs, Windows PCs, Linux machines and Raspberry Pis can't run the model side of this feature.
• oMLX, a free app that runs AI models on a Mac. It's the only model app this feature is designed and documented for.
• An assistant running on hardware you own, not on a cloud server. Step 1 explains this.
• Your Anthropic key, still connected. This may be surprising, so here's why. Dex, the built-in AI sysadmin that monitors and repairs your setup, keeps using Claude. Dex costs far less than your conversations would, but it isn't free. A local model doesn't remove the need for an Anthropic account.

Step 1: Get Your Assistant Running on Your Own Hardware

A local model only works with an assistant that runs on a computer you own. We call this BYOD (bring your own device), and we covered it in the BYOD launch announcement.

If you already have a cloud assistant, it can't be moved. You choose where an assistant runs when you create it, and that choice is permanent. To use a local model, you'd set up a new, self-hosted assistant.

If you keep your cloud assistant as well, the new one counts as an additional assistant on your plan. That's a Family Add-On at $6.99/month, with a lower launch-offer price for the first 6 months while the offer applies.

Next, choose between the two supported setups. If you're not sure which suits your home, our guide to choosing between one Mac or two machines compares them in more depth.

Option A: One Mac (recommended)

Your assistant and the model both run on the same Mac. This is the simplest option and the one we recommend.

• On a Mac, BYOD needs macOS 14 or later.
• Setup starts with a one-time step of about 10 minutes. Then you paste one command the portal gives you.
• Your assistant runs in its own separate workspace on the Mac, and your normal files aren't touched.

The catch is memory. The Mac has to hold your assistant and the model at once, which is one reason 32GB is the comfortable size for a large model.

Option B: Two Machines on One Home Network

Your assistant runs on another computer in your home, such as a Raspberry Pi, a spare Linux machine or a Windows PC. The model runs on your Mac. Both machines have to be on the same home network.

Reaching your Mac's model over the internet isn't available. A cloud-hosted assistant can't use a model on your home Mac, and neither can a self-hosted one somewhere else. We've switched that off on purpose.

Step 2: Install oMLX and Get a Model Running

On your Mac, install oMLX, download a model, and start it. Choose a model that suits your memory. The hardware guide explains how to match model size to your Mac.

You don't need to write down the model's name. The dashboard reads the list of available models straight from oMLX, and you pick from that list.

Before you move on, make sure oMLX is open and your model is loaded. The dashboard can only find a model that's running.

Step 3: Connect It in the Dashboard

Open your assistant in the dashboard and find "Use your own computer's AI." Then:

1. Pick your setup type: one Mac, or two machines on your home network.
2. Test the connection. This checks that your assistant can reach oMLX.
3. Choose a model from the list the dashboard pulls from oMLX.
4. Tick the box confirming you've read the cautions. Please read them.
5. Save.

Saving rebuilds your assistant's software so it can use the new model. This can take a few minutes. In our testing, loading the model for the first time took over a minute. Let it finish before you decide something is wrong.

Step 4: Test It Before You Rely on It

Send your assistant several messages before you depend on it. Ask everyday questions, and also give it the kind of multi-step job you'd normally hand it. Watch for three things:

• Quality. How good the answers are depends on the model you choose.
• Less-capable models struggle with multi-step work like reading files, running things and handling pictures.
• In one of our tests, a less-capable model reported a result for a step it hadn't finished.
• Check the work, especially early on. You're responsible for the answers you act on, and we can't support your Mac or the model you choose.
• Speed. In a 22-turn test, the median reply took about 10 seconds, with replies ranging from 5 to 46 seconds. Your speed will depend on your Mac and your model. Don't expect instant replies.
• Visible "thinking." Some models include their private reasoning in their replies. If yours does and it bothers you, try a different model.

Living With It: What Usually Goes Wrong

With the standard setup, Dex handles most of the upkeep, as described in how to keep your private AI assistant running without touching a terminal. A local model adds a few jobs that fall to you, because your Mac is now a working part of your assistant.

Your assistant stops answering if your Mac sleeps, loses its network connection or restarts, or if oMLX stops. It does not switch to Claude on its own.

• A message waits about 3 minutes, then gives up.
• We send one email alert, typically within about 12 minutes.
• On a one-Mac setup, though, a sleeping Mac can't send that email either. In that case the unanswered message may be your only sign that something is wrong.

oMLX doesn't reopen itself after a restart. This is the most likely thing to go wrong. After a macOS update, a power cut or any reboot, you have to open oMLX yourself and make sure your model is loaded. Make it a habit: whenever your Mac restarts, open oMLX.

A few habits that help:

• Change your Mac's energy settings so it doesn't sleep while you expect your assistant to answer.
• Keep the Mac plugged in and connected to your network.
• If your assistant goes quiet, check that oMLX is running before anything else. Our guide to what happens when your Mac's local model stops answering walks through getting it back.

For more day-to-day routines, see our guide to living with a local AI model on your Mac.

What It Costs

Your LaunchMy.ai pricing doesn't change. Local models aren't a separate plan or an add-on. Starter is $5.99/month for your first 6 months during the launch offer, then $9.99/month. Our monthly cost breakdown covers the full picture.

With a local model:

• Conversations cost nothing per message. You pay for the electricity your Mac uses.
• Dex still uses your Anthropic account for monitoring, maintenance and repairs. It costs far less than conversations with Claude, but it isn't zero.
• No cloud server bill, because your assistant runs on your own hardware. That's a difference in where it runs, not a cheaper LaunchMy.ai plan.

This isn't "completely free," but a heavy user could spend noticeably less on conversations.

A Note on Privacy

With a local model, your conversations are processed on your own Mac instead of being sent to Anthropic. That's a real privacy benefit. There are two caveats:

• Your messages still travel through the chat app you use to reach your assistant, such as Telegram.
• Dex's maintenance work still goes through Claude.

It's a meaningful improvement, not total isolation. Our plain-English privacy map walks through what stays on your Mac and what doesn't.

Switching Back Is One Click

Going back to Claude is one click in the dashboard, and returning to your local model is one click too. One approach is to use Claude for important work and the local model for everyday questions. Our guide to telling whether your Mac's AI is earning its keep can help you decide when each makes sense. If your Mac is going to be off for a while, such as when you're travelling, switch to Claude first.

Is This Setup Right for You?

This setup is a good fit if:

• you have an Apple-silicon Mac with plenty of memory,
• you're comfortable choosing and testing a model, and
• you'll reopen oMLX after a restart.

If that sounds like more upkeep than you want, the standard setup is designed for you. Your assistant uses Claude, Dex handles the maintenance, and you don't have to manage a model at all. You can find more detail on the Bring Your Own AI page in the Power Users section of our site.