Can I run AI on my own computer instead of paying for ChatGPT?
By Kevin Evers, Founder of Vision Driven Solutions ·
A friend of mine, Mike, runs AI on his own computer instead of paying for ChatGPT. I run a local model too, for transcription and a few other small tasks.
When business owners ask me about this, it is usually about privacy. They do not want client information leaving the office. Here is what I have seen so far, including what I have not figured out yet.
What running AI locally means
The AI model is downloaded to your own computer and runs there. A free tool called Ollama is a common way to do it. It downloads a model and runs it on your desktop, and once the model is on your machine it works without an internet connection.
The difference from ChatGPT is where your words go. With a paid online service, what you type is sent to that company to be processed. With a local model, it stays on the computer in front of you.
What you get
- Privacy. Notes with client details, internal documents and meeting transcripts stay on your machine.
- No monthly bill that can change on you later.
- It works when the internet does not.
What you give up
- Quality. The models small enough to run on a normal desktop are not as capable as the ones OpenAI, Anthropic and Google run in their data centers.
- Speed. On an older computer, a local model can be slow.
- Setup and upkeep. Someone has to install it and keep it updated, and there is no support line to call.
- Connections. The big online services can search the web or work with your email and calendar. A local setup usually does none of that unless someone builds it.
Where it makes sense in a small business
I use a local model for transcription, because the recordings often have client details in them and good enough is good enough for meeting notes. First drafts and summaries of internal documents are another reasonable fit.
I would not use one for anything that needs the best reasoning available, current information from the web, or answers that go straight to a customer.
The test I would run first
Pick one real task you already use ChatGPT for. Run it on a local model and on the paid service, side by side, with the same input. Use a sample with nothing private in it, since the paid side will see it.
If the local answer is good enough for that task, you know where it fits. If it is not, you found out in an afternoon instead of after canceling a subscription.
What I have not figured out yet
I have not tried a local model on bigger work. Mike is pushing his further, and I want to see what he is still using it for in a month. I will add what we learn here.
Frequently asked questions
- Is a local AI model really private?
- If it runs fully on your computer and you have not connected it to outside services, what you type stays on that machine. Check the settings of whatever tool you install, and remember the computer itself still needs normal security.
- Do I need a special computer?
- A newer computer with plenty of memory helps. Smaller models still run on older machines, just slowly. Try it on the computer you already have before buying anything.
- Is Ollama free?
- Ollama is free to download and use. The models it runs are free to download as well, though each model has its own license terms.
- Should I cancel ChatGPT?
- Not before you run the side-by-side test on your own tasks. I expect a lot of owners to end up using both: a local model for private material and a paid service for the hardest work.
Sources
Have a question like this about your own business? A 20-minute call is the fastest way to find out if it is the real one.
Book a 20-minute call