LLM

class pipecat.services.google.llm.GoogleUserContextAggregator(context, *, params=None, **kwargs)

Bases: OpenAIUserContextAggregator

Parameters:

• context (OpenAILLMContext)

• params (LLMUserAggregatorParams | None)

async push_aggregation()

Pushes the current aggregation based on interruption strategies and conditions.

class pipecat.services.google.llm.GoogleAssistantContextAggregator(context, *, params=None, **kwargs)

Bases: OpenAIAssistantContextAggregator

Parameters:

• context (OpenAILLMContext)

• params (LLMAssistantAggregatorParams | None)

async handle_aggregation(aggregation)

Adds the given aggregation to the aggregator. The aggregator can use a simple list of message or a context. It doesn’t not push any frames.

Parameters:

aggregation (str)

async handle_function_call_in_progress(frame)

Handle a function call in progress.

Adds the function call to the context with an IN_PROGRESS status to track ongoing function execution.

Parameters:

frame (FunctionCallInProgressFrame) – Frame containing function call progress information.

async handle_function_call_result(frame)

Handle the result of a function call.

Updates the context with the function call result, replacing any previous IN_PROGRESS status.

Parameters:

frame (FunctionCallResultFrame) – Frame containing the function call result.

async handle_function_call_cancel(frame)

Handle a cancelled function call.

Updates the context to mark the function call as cancelled.

Parameters:

frame (FunctionCallCancelFrame) – Frame containing the function call cancellation information.

async handle_user_image_frame(frame)

Handle a user image frame from a function call request.

Marks the associated function call as completed and adds the image to the context for processing.

Parameters:

frame (UserImageRawFrame) – Frame containing the user image and request context.

class pipecat.services.google.llm.GoogleContextAggregatorPair(_user: pipecat.services.google.llm.GoogleUserContextAggregator, _assistant: pipecat.services.google.llm.GoogleAssistantContextAggregator)

Bases: object

Parameters:

• _user (GoogleUserContextAggregator)

• _assistant (GoogleAssistantContextAggregator)

user()

Return type:

GoogleUserContextAggregator

assistant()

Return type:

GoogleAssistantContextAggregator

class pipecat.services.google.llm.GoogleLLMContext(messages=None, tools=None, tool_choice=None)

Bases: OpenAILLMContext

Parameters:

• messages (List[dict] | None)

• tools (List[dict] | None)

• tool_choice (dict | None)

static upgrade_to_google(obj)

Parameters:

obj (OpenAILLMContext)

Return type:

GoogleLLMContext

set_messages(messages)

Parameters:

messages (List)

add_messages(messages)

Parameters:

messages (List)

get_messages_for_logging()

add_image_frame_message(*, format, size, image, text=None)

Parameters:

• format (str)

• size (tuple[int, int])

• image (bytes)

• text (str)

add_audio_frames_message(*, audio_frames, text='Audio follows')

Parameters:

• audio_frames (list[AudioRawFrame])

• text (str)

from_standard_message(message)

Convert standard format message to Google Content object.

Handles conversion of text, images, and function calls to Google’s format. System messages are stored separately and return None.

Parameters:

message –

Message in standard format: {

”role”: “user/assistant/system/tool”, “content”: str | [{“type”: “text/image_url”, …}] | None, “tool_calls”: [{“function”: {“name”: str, “arguments”: str}}]

}

Returns:

• role: “user” or “model” (converted from “assistant”)

  • parts: List[Part] containing text, inline_data, or function calls

Returns None for system messages.

Return type:

Content object with

to_standard_messages(obj)

Convert Google Content object to standard structured format.

Handles text, images, and function calls from Google’s Content/Part objects.

Parameters:

obj – Google Content object with: - role: “model” (converted to “assistant”) or “user” - parts: List[Part] containing text, inline_data, or function calls

Returns:

[

{

“role”: “user/assistant/tool”, “content”: [

{“type”: “text”, “text”: str} | {“type”: “image_url”, “image_url”: {“url”: str}}

]

}

]

Return type:

List of messages in standard format

class pipecat.services.google.llm.GoogleLLMService(*, api_key, model='gemini-2.0-flash', params=None, system_instruction=None, tools=None, tool_config=None, **kwargs)

Bases: LLMService

This class implements inference with Google’s AI models.

This service translates internally from OpenAILLMContext to the messages format expected by the Google AI model. We are using the OpenAILLMContext as a lingua franca for all LLM services, so that it is easy to switch between different LLMs.

Parameters:

• api_key (str)

• model (str)

• params (InputParams | None)

• system_instruction (str | None)

• tools (List[Dict[str, Any]] | None)

• tool_config (Dict[str, Any] | None)

adapter_class

alias of GeminiLLMAdapter

class InputParams(*, max_tokens=4096, temperature=None, top_k=None, top_p=None, extra=<factory>)

Bases: BaseModel

Parameters:

• max_tokens (int | None)

• temperature (float | None)

• top_k (int | None)

• top_p (float | None)

• extra (Dict[str, Any] | None)

max_tokens: int | None

temperature: float | None

top_k: int | None

top_p: float | None

extra: Dict[str, Any] | None

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

can_generate_metrics()

Return type:

bool

async process_frame(frame, direction)

Process a frame.

Parameters:

• frame (Frame) – The frame to process.

• direction (FrameDirection) – The direction of frame processing.

create_context_aggregator(context, *, user_params=LLMUserAggregatorParams(aggregation_timeout=0.5), assistant_params=LLMAssistantAggregatorParams(expect_stripped_words=True))

Create an instance of GoogleContextAggregatorPair from an OpenAILLMContext. Constructor keyword arguments for both the user and assistant aggregators can be provided.

Parameters:

• context (OpenAILLMContext) – The LLM context.

• user_params (LLMUserAggregatorParams, optional) – User aggregator parameters.

• assistant_params (LLMAssistantAggregatorParams, optional) – User aggregator parameters.

Returns:

A pair of context aggregators, one for the user and one for the assistant, encapsulated in an GoogleContextAggregatorPair.

Return type:

GoogleContextAggregatorPair