How to Run Large Language Models (LLMs) in Your Command Line?

LLM is a command-line utility and Python library for interacting with large language models. In the latest release (v0.5), it offers support for self-hosted language models through plugins.

Installation

LLM can be installed using pip, pipx, or Homebrew. The syntax for each is as follows:

♥️ Info: Are you AI curious but you still have to create real impactful projects? Join our official AI builder club on Skool (only $5): SHIP! - One Project Per Month

With pip: pip install llm
With pipx: pipx install llm
With Homebrew: brew install simonw/llm/llm

Key Features

Plugins: Users can install plugins that add support for additional models. This includes 17 models from the GPT4All project, Mosaic’s MPT-30B self-hosted model, and Google’s PaLM 2 (via their API).
Model Installation: With the new plugin system, users can install models directly on their machine. For instance, the llm-gpt4all plugin can be installed with llm install llm-gpt4all.
Running Prompts: Users can run prompts against a model. For instance, to run the prompt "The capital of Germany?" against the ggml-vicuna-7b-1 model, use llm -m ggml-vicuna-7b-1 "The capital of Germany?".
Logging: All prompts and responses are logged to a SQLite database. Users can view the most recent record with llm logs -n 1.

You can see the possible models by running the following command:

llm models list

Using LLM from Python

The new version also supports usage as a Python library. Here’s an example:

import llm

model = llm.get_model("gpt-3.5-turbo")
model.key = 'YOUR_API_KEY_HERE'

response = model.prompt("Ten names for a new programming language")
print(response.text())

You can also use conversations and send multiple prompts to the model within the same context:

import llm

model = llm.get_model("ggml-mpt-7b-chat")

conversation = model.conversation()
r1 = conversation.prompt("The biggest country in Europe?")
print(r1.text())

r2 = conversation.prompt("How many people live there?")
print(r2.text())

This and previous examples are taken from Simon Willison’s Web Blog. Check it out, I really love the blog!

Also check out our guide that I’m sure you’ll love!