diff --git a/api-reference/server/services/llm/google-vertex.mdx b/api-reference/server/services/llm/google-vertex.mdx index c1deb075..f6b83379 100644 --- a/api-reference/server/services/llm/google-vertex.mdx +++ b/api-reference/server/services/llm/google-vertex.mdx @@ -124,6 +124,16 @@ Before using Google Vertex AI LLM services, you need: HTTP options for the Google AI client. + + How long to wait for the next chunk of a streamed response before giving up on + it. Bounds the wait when the API accepts a request and then stops producing + without closing the stream. This is a gap between chunks, not a limit on how + long a response may take overall. The first chunk is the slowest, since its + wait spans the whole round trip including any thinking the model does before + it emits anything; raise this for models configured to think at length. Set to + `None` to wait indefinitely. Inherited from `GoogleLLMService`. + + ## Usage ### Basic Setup diff --git a/api-reference/server/services/llm/google.mdx b/api-reference/server/services/llm/google.mdx index 520823a9..260861f5 100644 --- a/api-reference/server/services/llm/google.mdx +++ b/api-reference/server/services/llm/google.mdx @@ -100,6 +100,16 @@ Before using Google Gemini LLM services, you need: HTTP options for the Google API client. + + How long to wait for the next chunk of a streamed response before giving up on + it. Bounds the wait when the API accepts a request and then stops producing + without closing the stream. This is a gap between chunks, not a limit on how + long a response may take overall. The first chunk is the slowest, since its + wait spans the whole round trip including any thinking the model does before + it emits anything; raise this for models configured to think at length. Set to + `None` to wait indefinitely. + + ### Settings Runtime-configurable settings passed via the `settings` constructor argument using `GoogleLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details. @@ -272,10 +282,10 @@ await worker.queue_frame( `GoogleLLMService` supports the following event handlers, inherited from [LLMService](/api-reference/server/events/service-events): -| Event | Description | -| --------------------------- | --------------------------------------------------------------------------- | -| `on_completion_timeout` | Called when an LLM completion request times out (Google `DeadlineExceeded`) | -| `on_function_calls_started` | Called when function calls are received and execution is about to start | +| Event | Description | +| --------------------------- | ---------------------------------------------------------------------------------------------------------- | +| `on_completion_timeout` | Called when an LLM completion request times out (Google `DeadlineExceeded` or stream idle timeout reached) | +| `on_function_calls_started` | Called when function calls are received and execution is about to start | ```python @llm.event_handler("on_completion_timeout")