A Generation is the output or answer produced by an LLM. A generation can be part of a Step, and contains both the input sent to an LLM and its completion.

Generations can be viewed on their own page in Literal AI. In a single generation, you can see all the data that was sent to and retrieved from the LLM model, which follows the Prompt template, including System messages, User messages, Assistant messages, and Tool messages.

You can add Tags and Scores to Generations. Additionally, you can add Generations to a Dataset, to use later on in for example Evaluation.

An example of a Generation

An example of a Generation on Literal AI

​
Get Generations

Generations are available via the SDKs (Python, TypeScript).

To get all generations of the application, you can use the following code:

import os
from literalai import LiteralClient

literal_client = LiteralClient(api_key=os.getenv("LITERAL_API_KEY"))

generations = literal_client.api.get_generations()

for generation in generations.data:
    print(generation.to_dict())

# Network requests by the SDK are performed asynchronously.
# Invoke flush_and_stop() to guarantee the completion of all requests prior to the process termination.
# WARNING: If you run a continuous server, you should not use this method.
client.flush_and_stop()

​
Create new Generation

To create a new Generation:

import os
from literalai import LiteralClient, ChatGeneration

literal_client = LiteralClient(api_key=os.getenv("LITERAL_API_KEY"))

# Use ChatGeneration or CompletionGeneration to create a new generation

# @dataclass
# class ChatGeneration(BaseGeneration):
#     messages: List[GenerationMessage] = Field(default_factory=list)
#     type = GenerationType.CHAT

def create_generation():

    chat_generation = ChatGeneration(
        messages=[{"content": "Hello, this is my answer", "role": "system"}],
    )

    generation = literal_client.api.create_generation(
        generation = chat_generation
    )

    return generation

generation = create_generation()

# Network requests by the SDK are performed asynchronously.
# Invoke flush_and_stop() to guarantee the completion of all requests prior to the process termination.
# WARNING: If you run a continuous server, you should not use this method.
client.flush_and_stop()

Was this page helpful?

RunTags