Ollama on Ubuntu: Install and Use Guide

This guide walks you through installing Ollama on Ubuntu and running your first local large language models. You’ll learn the quick install method, how to pull and run models, and how to interact with them via the command line and API. No prior experience with AI tools is required, just a terminal and an internet connection.

What You’ll Need

• Ubuntu 20.04 or newer (64-bit)
• At least 8 GB of RAM (16 GB or more recommended for larger models)
• An internet connection to download the installer and model files
• Basic familiarity with the terminal (opening a terminal, running commands)

Step-by-Step

1. Open a terminal. Press Ctrl+Alt+T or search for “Terminal” in the applications menu.
2. Install Ollama using the official script. Run the following command and press Enter. This downloads and runs the installer, which sets up Ollama as a system service.
   curl -fsSL https://ollama.com/install.sh | sh
3. Verify the installation. After the script finishes, check that Ollama is installed by running:
   ollama --version
   You should see output like ollama version 0.x.x. If not, close and reopen the terminal or log out and back in.
4. Pull a model. Ollama uses a library of open-source models. Start with a small, fast one like Llama 3.2 1B:
   ollama pull llama3.2:1b
   This downloads the model (about 1.3 GB). For a more capable model, try ollama pull llama3.2 (about 4.9 GB) or ollama pull mistral (about 4.1 GB).
5. Run the model interactively. Start a chat session with:
   ollama run llama3.2:1b
   You’ll see a >>> prompt. Type a question, like “What is the capital of France?” and press Enter. The model will respond. To exit, type /bye and press Enter.
6. Run a one-off prompt (non-interactive). You can also get a single response without entering the chat loop:
   ollama run llama3.2:1b "Explain the water cycle in one sentence."
7. Use the REST API (optional). Ollama runs a local API server on port 11434. To test it, open a new terminal and run:
   curl http://localhost:11434/api/generate -d '{"model": "llama3.2:1b", "prompt": "Hello!"}'
   You’ll get a JSON response with the model’s output.
8. List and manage models. See what you have downloaded with ollama list. Remove a model you no longer need with ollama rm llama3.2:1b.

Common Mistakes

• Running out of memory. Large models (7B+ parameters) need significant RAM. If you get an error like “failed to load model” or the system freezes, try a smaller model or close other applications.
• Not enough disk space. Models are large (1-5 GB each). Check your free space with df -h before pulling.
• Using the wrong model name. The exact tag matters. Use ollama list to see available models and ollama show llama3.2:1b to get details.
• Forgetting to restart the service after a manual install. If you install from source, you may need to start the service with sudo systemctl start ollama.

Tips

• Start with a small model. For testing, use llama3.2:1b or phi3:mini; they’re fast and require less RAM.
• Use the right model for the task. For coding, try codellama; for general chat, llama3.2 or mistral; for math, qwen2.5:7b.
• Set the context length for long conversations. Use ollama run llama3.2 --num-ctx 8192 to increase the context window (default is 2048).
• Keep Ollama updated. Run ollama update to get the latest version.
• Use the API for automation. You can write scripts that call the local API, making it easy to integrate LLM responses into your workflows.

Troubleshooting

• “curl: command not found” – Install curl with sudo apt install curl and try the install command again.
• “Permission denied” when running ollama – Add your user to the ollama group: sudo usermod -aG ollama $USER, then log out and back in.
• Model download is slow or fails – Check your internet connection. If you’re behind a proxy, set the HTTP_PROXY and HTTPS_PROXY environment variables before running the install or pull.
• “Error: model not found” – The model name may be incorrect. Run ollama list to see what you have, or check the official library at https://ollama.com/library for the exact tag.
• Ollama service is not running – Check with systemctl status ollama. If it’s inactive, start it with sudo systemctl start ollama.
• Out of memory errors – Reduce the number of concurrent requests, use a smaller model, or increase your swap space (not recommended for performance).

Frequently Asked Questions

Do I need a GPU to run Ollama?

No, Ollama works on CPU-only systems, but it will be slower. For better performance, an NVIDIA GPU with CUDA support is recommended. Ollama automatically uses the GPU if available.

How do I uninstall Ollama?

Run sudo systemctl stop ollama, then sudo rm /usr/local/bin/ollama and sudo rm -rf /usr/local/lib/ollama. Remove the service file with sudo rm /etc/systemd/system/ollama.service and reload systemd with sudo systemctl daemon-reload.

Can I use Ollama offline?

Yes, once a model is downloaded, you can use it without an internet connection. The API and CLI work entirely locally.

How do I update a model to a newer version?

Pull the same model again with ollama pull llama3.2:1b – Ollama will download the latest version if available.

Is it safe to run the install script?

The script is the official method recommended by Ollama. It installs the binary and sets up a systemd service. You can review the script first by opening it in a browser or using curl -fsSL https://ollama.com/install.sh | less before piping to sh.

Can I use Ollama with Python?

Yes, you can use the REST API with Python’s requests library. There’s also an official Python client package, but the API is straightforward for custom integrations.

Related

• Share Password Securely via AirDrop
• How to Seal Aluminum Windows: Step-by-Step Guide
• Fix Apps Pinning Themselves to Taskbar in Windows 10