Who Runs the Models? You Do.

Part 3 of a 3-part series to help you learn and use open-source models

If you are reading this series in its entirety, it is because you decided to go on a journey with me to install open-source AI models on your local computer.

In this final installment, I provide you with some ideas and tips for using your new open-source AI model on your computer, including:

• Create AI agents on your computer, for free

• Understanding how open-source licensing works

• Compare open-source models to determine what fits your needs

• Remove models you do not like to free up space

If you missed the previous articles, you can find Part 1 here and Part 2 here.

---

Your AI, your rules, your AI agent

If you thought using a local LLM on your computer was cool, there is a new world of possibilities. For example, you can use an Agentic AI workflow tool, like n8n, to automate repetitive manual tasks.

The following image shows a simple AI agent I created that summarizes any new articles I write for this Bill Talks AI newsletter. Your AI agent can work with or without an internet connection.

Want to learn how to use n8n to create your own personal AI agents? Leave a comment! Comments are open to paid members.

The cost of playing boss with open models

As I shared in Part 1 of this series, you can download many open-source AI models. Also, even though they may label themselves as open-source, they do not all offer fully permissive use.

For example, many budding AI entrepreneurs will download a hot new AI model and move it to production. Then, they learn there are strings attached, such as special licensing requirements.

While I highly recommend you read every license before you try it, there is an easy path I can share with you. Locate the model you are considering using, then use ChatGPT (or similar) and prompt for a review of how permissive the license is.

Here, for example, is a prompt for Meta’s Llama model (not to be confused with the Ollama app):

Prompt: Evaluate how permissive the license is for Meta’s Llama 3.3. Use this license page as your reference: https://www.llama.com/llama3_3/license/

AI Response (edited for brevity): The Llama 3.3 license is permissive in allowing usage, modification, and redistribution, making it suitable for a wide range of applications. However, it imposes specific conditions to ensure responsible use and to protect Meta's interests, especially concerning attribution, acceptable use, and commercial distribution at scale. While not as open as licenses like Apache 2.0, it provides substantial flexibility for developers and researchers.​

• High-Scale Commercial Use: Entities with products or services exceeding 700 million monthly active users must obtain a separate license from Meta, which is granted at Meta's discretion.

• EU Restrictions on Multimodal Models: The license restricts the use of multimodal components of Llama 3.3 by individuals or companies based in the European Union, although end-users in the EU can use products incorporating these models.

And here is what the Mistral 7b response model returns.

Prompt: Evaluate how permissive the license is for Mistral's 7B model. Use this license page as your reference: https://ollama.com/library/mistral:7b/blobs/43070e2d4e53

AI Response (edited for brevity): The permissiveness of the Apache 2.0 license means that developers and organizations can:​

• Integrate Mistral 7B into proprietary software without the obligation to disclose source code.​

• Deploy the model across various platforms, including cloud services and on-premises environments.​ Mistral AI | Frontier AI in your hands

• Fine-tune the model for specific applications and distribute the customized versions.​ Mistral AI - Help Center

This level of flexibility makes the Mistral 7B model an attractive option for both open-source projects and commercial enterprises seeking to leverage advanced language modeling capabilities without restrictive licensing constraints.​

As you can see, Meta’s Llama has more restrictions than Mistral’s license. If you want to use an LLM on your computer, most open-source models allow you to own the content you generate, and most will enable you to use them however you like. If you want to use the model to scale a business later, that is where legal implications can occur, so remember to read those licenses.

Switch without missing a beat

Let’s say you use the Mistral 7b model you installed in Part 1 of this series. However, you do not like the way it responds to your prompts. Perhaps it is too verbose, not verbose enough, or does not match the style you are looking for.

You might want to try comparing it to other models. That means downloading new ones and seeing how they respond.

As of this writing, Google offers a new open-source model they call gemma3. Are you wondering if that is a better option for your needs? You want to compare it to another model, like Mistral.

Here are the steps to download Google’s gemma3 using Ollama:

1. Go to the Ollama website and select the Models link.

2. Search for gemma3 (or whatever model you are interested in) and then select the model. You should see a model overview page similar to the following image. Locate the ollama run text and select the copy to clipboard icon.

3. Run terminal (Mac) or command prompt (Windows) on your computer and paste the command you just copied from the Ollama website. In my case, it is ollama run gemma3. Press the return key on your keyboard. If you do not already have Gemma on your computer, Ollama will download the file, which could take some time.

4. Once the download is complete, Ollama will automatically run the model, and you should see a prompt appear in the form of three greater-than symbols, like this: >>>. Type “Why is the sky blue?” and press the return key on your keyboard. You should see a response similar to the following image.

5. Once you know the model is running, type /exit to leave Ollama’s prompt mode and then quit the terminal window.

At this point, you should have at least two models on your computer:

• Mistral 7b

• gemma3

See it for yourself and trust what you know

When a new model releases (which seems to happen daily), they make a big marketing splash, touting “massive” evaluation numbers and making exaggerated claims to be “the best.”

But, “the best” for some may not be the best for you. Try it yourself to determine if you want to use that new model. You can use Open WebUI to compare one model to another.

Here is how you can subjectively compare two models using Open WebUI:

1. Open your browser and type your Open WebUI URL, which is likely: http://localhost:3000

2. Create a new chat and note that one model will already be listed. Likely, it will be mistral:latest (or gemma3:latest). No matter the default, select the + icon next to the default and then select the other model you want to use for comparison.

3. With both models selected, type a prompt that represents a typical prompt you plan to use, and then select the send icon. For this article, my prompt is:
   Write a short 1-paragraph comedic story about a chicken that crosses the road.

Notice how Open WebUI shows you two responses. One is from Mistral and the other from Gemma. Try some more prompts you might use, like “summarize this document” or “Create a response to this email.”

As it turns out, you might like one model for certain things (like summarizing a document) and another for other things (like writing). The great news is you can have it both ways. For example, writing? Select Mistral. Coding? Select Gemma.

You can create any number of chats with any number of different models!

Learn some hidden moves with Ollama (help commands)

To learn more about what you can do with Ollama and get specific instructions for Mac, Windows, or Linux, you can read the README.md file here.

When you run the terminal window, you can type ollama and press return on your keyboard to see all the available commands. You can even get more help for each command by following the instructions.

Let’s say you want to know more about the list command. You can use the help flag, like this:

ollama list -h

Over time, you will learn how to use these commands. You can even use Open WebUI to ask how to use Ollama!

Make space and keep what serves you (delete a model)

As you learned from Part 1 of this series, Ollama is the tool you use to download (pull) and run AI models. It is also the interface between chat apps and an open-source model. That is why you need Ollama installed to use Open WebUI.

Those model files can be large, taking up vast amounts of disk space. If you no longer use a model, I suggest you use Ollama for another task: removing (deleting) a model. Here is how you do it:

1. Run your computer's terminal (Mac) or command prompt (Windows) app. Type ollama list and press return on your keyboard. Ollama will list all models installed on your computer. In my example, I have three:

• gemma3:latest

• mistral-small3.1:latest

• Mistral:latest

2. In this example, you will delete the mistral:latest model. Type ollama rm mistral:latest and press return on your keyboard. In my case, that freed up 4.1 gigabytes of disk space on my computer.

   💡 The rm command is shorthand for the word remove.

3. Quit the terminal window.

Keep building. I’ll be here

If you want to learn more about using a local LLM on your computer and are a paying subscriber, let me know in the comments.