Fine-Tuning GPT-3.5 Turbo – How to Craft Your Own Proprietary LLM

The much-awaited feature for GPT-3.5 Turbo is here: fine-tuning. And guess what? GPT-4 is next in line this autumn. Dive in to discover how this can revolutionize your applications and user experiences.

What’s New?

OpenAI now empowers you to tailor GPT-3.5 Turbo with your data, ensuring the model aligns perfectly with your specific needs. Preliminary results? A fine-tuned GPT-3.5 Turbo can rival, and sometimes even surpass, the base GPT-4 in specialized tasks. And here’s a cherry on top: the data you use remains yours. OpenAI respects your privacy and won’t use it for other model training.

Why Fine-Tune?

Ever since GPT-3.5 Turbo hit the scene, there’s been a clamor for a more personalized touch. Here’s what fine-tuning brings to the table:

Steerability Boost: Want the model to follow instructions to the T? Fine-tuning is your answer. For instance, if you need the model to always reply in German, fine-tuning ensures it does just that.
Consistent Formatting: If you’re into tasks like code completion or API call composition, fine-tuning ensures the model’s responses are formatted just the way you want. Imagine converting user prompts into precise JSON snippets seamlessly.
Customized Tone: Every brand has its voice. With fine-tuning, GPT-3.5 Turbo can echo the unique tone of your brand, ensuring consistency across interactions.

Added Bonuses

Shorter Prompts, Same Performance: Fine-tuning means you can trim your prompts and still get top-notch results.
More Tokens: GPT-3.5 Turbo, when fine-tuned, can now manage 4k tokens, a whopping double from before. Some early birds have even slashed their prompt sizes by up to 90%, making API calls faster and more cost-effective.

Maximizing Fine-Tuning: The real magic happens when you blend fine-tuning with techniques like prompt engineering, information retrieval, and function calling. Hungry for more insights? OpenAI’s fine-tuning guide is your go-to resource.

You can stay updated on new developments by subscribing to our tech newsletter by downloading the following Python cheat sheet:

Step-by-Step Guide to Fine-Tuning GPT-3.5 Turbo

Step 1: Data Preparation

Before you start, you need to prepare your data in a specific format. This data will guide the model on how to behave. For instance, if you want the model to act as an assistant that occasionally misspells words, your data would look like this:

{
  "messages": [
    { "role": "system", "content": "You are an assistant that occasionally misspells words" },
    { "role": "user", "content": "Tell me a story." },
    { "role": "assistant", "content": "One day a student went to schoool." }
  ]
}

Here, the system instructs the assistant’s behavior, the user provides a prompt, and the assistant responds accordingly.

Step 2: Uploading Your Data

Once your data is ready, you need to upload it to OpenAI. Use the following curl command:

curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F "purpose=fine-tune" \
  -F "file=@path_to_your_file"

Replace path_to_your_file with the path to your prepared data file. Ensure your OpenAI API key is correctly set in the $OPENAI_API_KEY environment variable.

Step 3: Initiating the Fine-Tuning Job

With your data uploaded, it’s time to create a fine-tuning job. Use this curl command:

curl https://api.openai.com/v1/fine_tuning/jobs \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
  "training_file": "TRAINING_FILE_ID",
  "model": "gpt-3.5-turbo-0613"
}'

Replace TRAINING_FILE_ID with the ID you received after uploading your data in Step 2.

Once the model completes the fine-tuning, it’s ready for production use. It will have the same rate limits as the base model.

Step 4: Deploying the Fine-Tuned Model

To use your freshly fine-tuned model, employ the following curl command:

curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
  "model": "ft:gpt-3.5-turbo:org_id",
  "messages": [
    {
      "role": "system",
      "content": "You are an assistant that occasionally misspells words"
    },
    {
      "role": "user",
      "content": "Hello! What is fine-tuning?"
    }
  ]
}'

Replace org_id with your organization’s ID.

Pricing

Pricing Breakdown:

Fine-tuning costs are categorized into training and usage:

Training: $0.008 per 1K Tokens
Usage Input: $0.012 per 1K Tokens
Usage Output: $0.016 per 1K Tokens

To illustrate, a gpt-3.5-turbo fine-tuning job with a 100,000 tokens training file, trained over 3 epochs, would cost approximately $2.40.

Updates on GPT-3 Models:

In July, OpenAI revealed that the original GPT-3 models (ada, babbage, curie, and davinci) would be phased out by January 4th, 2024. However, the good news is that babbage-002 and davinci-002 are now available as replacements. You can access these models via the Completions API.

Furthermore, these models can be fine-tuned using the new API endpoint /v1/fine_tuning/jobs. This endpoint is more versatile, supporting the API’s future growth. Transitioning from the old /v1/fine-tunes to the new endpoint is a breeze. More details are available in the updated fine-tuning guide.

☠️ Note: The old /v1/fine-tunes endpoint will be discontinued on January 4th, 2024.

The pricing for both base and fine-tuned GPT-3 models will be provided subsequently.

Source: https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates

Coming Soon: OpenAI is gearing up to launch a user-friendly fine-tuning UI. This will offer developers a more intuitive way to monitor ongoing fine-tuning tasks, access completed model versions, and much more. Stay tuned!

With these steps, you’re well on your way to customizing GPT-3.5 Turbo to your unique requirements. Happy fine-tuning!