Ollama with Google Collab Support

mainly screwing around with ai but since deepseek blocked US users, I decided to make a Collab document that allows you to use the base model with promt chainging. this AI is basically completely free but may not be as good as some of the 32b/70b or the giga 671b param models that they have. use whatever you can without maxing out the GPU, which should be the 14b with my testing (if using the free colab base model). feel free to use any model you want off of Ollama's site located here: https://ollama.com/search

Here's a mockup of all the libraries I used (asked Copt to generate it because I'm lazy)

in no way is this peak performance, just a little side project. :D

Libraries and Their Functions:

subprocess

Used to run shell commands and manage subprocesses.
Example: subprocess.Popen is used to execute shell commands to manage the Ollama server.

threading

Used to run tasks in separate threads to avoid blocking the main execution.
Example: threading.Thread is used to start the Ollama server in a separate thread, allowing the main script to continue running.

time

Provides various time-related functions.
Example: time.sleep can be used to pause execution for a given number of seconds, though it's not explicitly used in the provided code.

requests

Used to make HTTP requests.
Example: requests.get is used to verify if the Ollama server is running by checking the response from the server.

os

Provides a way of using operating system-dependent functionality.
Example: os.environ is used to set environment variables for CUDA and NVIDIA libraries, ensuring the system can find and use these libraries.

asyncio

Used to write concurrent code using the async/await syntax.
Example: asyncio.create_subprocess_exec is used to run subprocesses asynchronously, allowing the script to manage multiple tasks concurrently.

langchain

A library for building language model applications.
Example: langchain.prompts.ChatPromptTemplate is used to create prompt templates for the conversation chain.

langchain_ollama

A specific integration of LangChain for the Ollama language model.
Example: langchain_ollama.ChatOllama is used to initialize the Ollama language model.

logging

Provides a flexible framework for emitting log messages from Python programs.
Example: logging.basicConfig is used to configure the logging system, allowing the script to log information and errors.

langchain.prompts

Provides tools for creating and managing prompts.
Example: langchain.prompts.HumanMessagePromptTemplate is used to create human message prompts.

langchain.chains

Provides tools for creating and managing chains of prompts and responses.
Example: langchain.chains.ConversationChain is used to create a conversation chain.

langchain.memory

Provides tools for managing memory in conversations.
Example: langchain.memory.ConversationSummaryBufferMemory is used to manage conversation memory, allowing the AI to remember previous interactions.

langchain.callbacks.manager

Provides tools for managing callbacks.
Example: langchain.callbacks.manager.CallbackManager is used to manage callback handlers.

langchain.callbacks.streaming_stdout

Provides a callback handler for streaming output to stdout.
Example: langchain.callbacks.streaming_stdout.StreamingStdOutCallbackHandler is used to handle streaming output.

langchain.agents

Provides tools for creating and managing agents.
Example: langchain.agents.Tool is used to define tools for the AI agent.

pciutils (installed via apt-get)

A collection of programs for inspecting and manipulating configuration of PCI devices.
Example: Installed to ensure the system can interact with PCI devices.

curl (installed via apt-get)

A command-line tool for transferring data with URLs.
Example: Used to download and install Ollama.

langchain_community (installed via pip)

A community-driven extension of LangChain.
Example: Installed to provide additional functionality for LangChain.

---

How They Work Together

Setup and Installation

• The script starts by killing any existing Ollama processes and installing necessary packages (pciutils, curl).
• It then installs Ollama using a shell command executed via curl.

Environment Configuration

• Environment variables for CUDA and NVIDIA libraries are set to ensure the system can find and use these libraries.

Starting the Ollama Server

• An asynchronous function run_process is defined to run shell commands asynchronously.
• Another asynchronous function start_ollama_serve is defined to start the Ollama server.
• The server is started in a separate thread using threading.Thread to avoid blocking the main script.

Verifying the Server

• The script uses requests.get to verify if the Ollama server is running by checking the response from the server.

Pulling Models

• The script pulls the DeepSeek models using shell commands executed via ollama pull.

Installing Python Packages

• The script installs necessary Python packages (langchain, requests, langchain_community, langchain_ollama) using pip.

Initializing the Language Model

• The script initializes the Ollama language model using ChatOllama from langchain_ollama.

Defining Tools and Agents

• A mock search tool function is defined and added to an agent using langchain.agents.Tool and initialize_agent.

Creating Prompt Templates and Memory

• Prompt templates and memory for the conversation are created using langchain.prompts.ChatPromptTemplate and langchain.memory.ConversationSummaryBufferMemory.

Creating the Conversation Chain

• A conversation chain is created using langchain.chains.ConversationChain.

Chat Loop

• A chat loop function chat_with_ai is defined to handle user input and generate responses using the conversation chain.
• The chat loop runs until the user exits, handling user input, generating responses, and managing errors.

Running the Chat Loop

• The chat loop is run if the script is executed as the main module.

Errors while Running

This code is prone to errors as using a jupyter notebook trying to do all of this is not optimal at all. If you have any concerns or questions, feel free to put in an issue or whatnot.

The most common error while running is ollama not seeing the actual model when it's downloaded. I can't tell you why, but just restart the instance and it should work. I'm guessing it's because of Colab's timeout time and connectivity, but if there is something that can be changed or is found, shoot me a msg.