How to Run Large Language Models (LLMs) in Your Command Line?

by
5/5 - (4 votes)

LLM is a command-line utility and Python library for interacting with large language models. In the latest release (v0.5), it offers support for self-hosted language models through plugins.

Installation

LLM can be installed using pip, pipx, or Homebrew. The syntax for each is as follows:

  • With pip: pip install llm
  • With pipx: pipx install llm
  • With Homebrew: brew install simonw/llm/llm

Key Features

  1. Plugins: Users can install plugins that add support for additional models. This includes 17 models from the GPT4All project, Mosaic’s MPT-30B self-hosted model, and Google’s PaLM 2 (via their API).
  2. Model Installation: With the new plugin system, users can install models directly on their machine. For instance, the llm-gpt4all plugin can be installed with llm install llm-gpt4all.
  3. Running Prompts: Users can run prompts against a model. For instance, to run the prompt "The capital of Germany?" against the ggml-vicuna-7b-1 model, use llm -m ggml-vicuna-7b-1 "The capital of Germany?".
  4. Logging: All prompts and responses are logged to a SQLite database. Users can view the most recent record with llm logs -n 1.

You can see the possible models by running the following command:

llm models list

Using LLM from Python

The new version also supports usage as a Python library. Here’s an example:

import llm

model = llm.get_model("gpt-3.5-turbo")
model.key = 'YOUR_API_KEY_HERE'

response = model.prompt("Ten names for a new programming language")
print(response.text())

You can also use conversations and send multiple prompts to the model within the same context:

import llm

model = llm.get_model("ggml-mpt-7b-chat")

conversation = model.conversation()
r1 = conversation.prompt("The biggest country in Europe?")
print(r1.text())

r2 = conversation.prompt("How many people live there?")
print(r2.text())

This and previous examples are taken from Simon Willison’s Web Blog. Check it out, I really love the blog!

Also check out our guide that I’m sure you’ll love!

👨‍💻 Recommended: 26 Insane Auto-GPT and Baby AGI Alternatives You Ought to Try in 2023