Skip to main content

OpenAI LLMs

OpenAI took the world by storm with the launch of ChatGPT and GPT-4, at the point of this writing, they are still the smartest LLMs out there. To use them, first you will need to get an API key from OpenAI, and export it with:

export OPENAI_API_KEY=<your key here>

Make sure you have OpenAI library installed:

pip install openai

Then, LangStream provides two thin wrapper layers for their APIs:

Text Completion​

OpenAI has two modes of generating text with LLMs, the first one, simple and more, is text completion, using GPT-3 derived models:

from langstream import join_final_output
from langstream.contrib import OpenAICompletionStream

recipe_stream = OpenAICompletionStream[str, str](
    "RecipeStream",
    lambda recipe_name: f"Here is my {recipe_name} recipe: ",
    model="davinci",
)

await join_final_output(recipe_stream("instant noodles"))
#=> '\xa01. Boil water 2. Add noodles 3. Add seasoning 4.'

This model is not specialized for chat as ChatGPT and GPT-4, but for completion, consider this while writing your prompts. In this case, we are taking the user input and making it part of a sentence for the model to complete ("Here is my..."), instead of asking a question for the model to answer.

When you use OpenAICompletionStream instead of a regular Stream, the lambda function you pass should return the prompt you want for the model consume, and the stream of outputs will be generated by the LLM, given this prompt.

You also must specify which model to use, OpenAI has several variations of the GPT-3 text completion model, take a look at their page to see which ones are available.

You also have other parameters you can pass to the stream like temperature, which helps with development if you keep it at zero, and max_tokens, take a look at the reference to learn more.

Chat Completion​

The most popular and powerful OpenAI completion API, however, is the Chat Completion, which gives you access to gpt-3.5-turbo and gpt-4 models. It is a bit more work to work with, because it has defined roles, for system, user, assistant or function. You define an OpenAIChatStream like this:

from langstream import Stream, join_final_output
from langstream.contrib import OpenAIChatStream, OpenAIChatMessage, OpenAIChatDelta

recipe_stream: Stream[str, str] = OpenAIChatStream[str, OpenAIChatDelta](
    "RecipeStream",
    lambda recipe_name: [
        OpenAIChatMessage(
            role="system",
            content="You are ChefGPT, an assistant bot trained on all culinary knowledge of world's most proeminant Michelin Chefs",
        ),
        OpenAIChatMessage(
            role="user",
            content=f"Hello, could you write me a recipe for {recipe_name}?",
        ),
    ],
    model="gpt-3.5-turbo",
).map(lambda delta: delta.content)

await join_final_output(recipe_stream("instant noodles"))
#=> "Of course! Here's a simple and delicious recipe for instant noodles:\n\nIngredients:\n- 1 packet of instant noodles (your choice of flavor)\n- 2 cups of water\n- 1 tablespoon of vegetable oil\n- 1 small onion, thinly sliced\n- 1 clove of garlic, minced\n- 1 small carrot, julienned\n- 1/2 cup of sliced mushrooms\n- 1/2 cup of shredded cabbage\n- 2 tablespoons of soy sauce\n- 1 teaspoon of sesame oil\n- Optional toppings: sliced green onions, boiled egg, cooked chicken or shrimp, chili flakes\n\nInstructions:\n1. In a medium-sized pot, bring the water to a boil. Add the instant noodles and cook according to the package instructions until they are al dente. Drain and set aside.\n\n2. In the same pot, heat the vegetable oil over medium heat. Add the sliced onion and minced garlic, and sauté until they become fragrant and slightly caramelized.\n\n3. Add the julienned carrot, sliced mushrooms, and shredded cabbage to the pot. Stir-fry for a few minutes until the vegetables are slightly softened.\n\n4. Add the cooked instant noodles to the pot and toss them with the vegetables.\n\n5. In a small bowl, mix together the soy sauce and sesame oil. Pour this mixture over the noodles and vegetables, and toss everything together until well combined.\n\n6. Cook for an additional 2-3 minutes, stirring occasionally, to allow the flavors to meld together.\n\n7. Remove the pot from heat and divide the noodles into serving bowls. Top with your desired toppings such as sliced green onions, boiled egg, cooked chicken or shrimp, and chili flakes.\n\n8. Serve the instant noodles hot and enjoy!\n\nFeel free to customize this recipe by adding your favorite vegetables or protein. Enjoy your homemade instant noodles!"

This model is really optimized for answering questions and following guidance. If you look at the lambda function, we do not return a simple string for the prompt, but a list of OpenAIChatMessages, those hold the role and the content.

Then, if you look at the type signature of OpenAIChatStream, you will notice that it takes a str but it returns an OpenAIChatDelta, this is what OpenAI's chat completion API streams back to us, it holds also the role and the content, so before joining the stream, we need to do a map on the delta.content to get strings back.

Now, you just learned how to use OpenAI for text and chat completion, which is powerful enough, but there is one even more powerful feature, which we put on a separate guide, OpenAI Function Calling, check it out on the next guide.

Also, if instead of an API you want to run an LLM locally, check it out our guide on GPT4All.