Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions docs/concepts/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ Used with `agent.with_tts()`. Each TTS vendor produces audio at a specific sampl
| `DeepgramTTS` | Deepgram | `api_key`, `model` | Configurable |
| `GradiumTTS` | Gradium | `api_key` | Configurable |
| `MistralTTS` | Mistral | `api_key` | — |
| `TypecastTTS` | Typecast | `api_key`, `voice_id`, `model` | — |
| `SarvamTTS` | Sarvam | `api_key` | — |
| `XaiTTS` | xAI | `api_key`, `language` | Configurable |

Expand Down Expand Up @@ -109,6 +110,8 @@ Used with `agent.with_stt()`.

Use `turn_detection.language` for Agora interaction language; it defaults to `en-US`. STT vendor `language` options are serialized under `asr.params` using each provider's own format. If `with_stt()` is omitted, AgentKit defaults to `AresSTT` for global clients and `FengmingSTT` for `Area.CN` clients. Ares does not take a provider language option; AgentKit uses `turn_detection.language` for REST `asr.language`.

`AresSTT` and `FengmingSTT` accept an optional `keywords: List[str]` hotword list.

| Class | Provider | Required Parameters |
|---|---|---|
| `SpeechmaticsSTT` | Speechmatics | `api_key`, `language` |
Expand Down Expand Up @@ -146,12 +149,14 @@ stt = DeepgramSTT(api_key='your-deepgram-key', language='en-US', model='nova-2')

Used with `agent.with_mllm()` for the [MLLM flow](../guides/mllm-flow.md). These handle audio input and output end-to-end.

| Class | Provider | Required Parameters |
|---|---|---|
| `OpenAIRealtime` | OpenAI Realtime | `api_key`; optional `turn_detection` |
| `GeminiLive` | Google Gemini Live API | `api_key`, `model`; optional `turn_detection` |
| `VertexAI` | Vertex AI (Gemini Live) | `model`, `project_id`, `location`, `adc_credentials_string`; optional `turn_detection` |
| `XaiGrok` | xAI Grok (`mllm.vendor`: `xai`) | `api_key`; optional `voice`, `language`, `sample_rate`, `turn_detection` |
| Class | Provider | Area | Required Parameters |
|---|---|---|---|
| `OpenAIRealtime` | OpenAI Realtime | Global | `api_key`; optional `turn_detection` |
| `AzureOpenAIRealtime` | Azure OpenAI Realtime | Global | `api_key`, `url`, `turn_detection`; optional `max_history` |
| `GeminiLive` | Google Gemini Live API | Global | `api_key`, `model`; optional `turn_detection` |
| `VertexAI` | Vertex AI (Gemini Live) | Global | `model`, `project_id`, `location`, `adc_credentials_string`; optional `turn_detection` |
| `XaiGrok` | xAI Grok (`mllm.vendor`: `xai`) | Global | `api_key`; optional `voice`, `language`, `sample_rate`, `turn_detection` |
| `QwenOmni` | Alibaba Cloud Qwen Omni Realtime | CN | `api_key`, `turn_detection`; optional `url` |

<!-- snippet: executable -->
```python
Expand All @@ -160,6 +165,12 @@ from agora_agent import OpenAIRealtime
mllm = OpenAIRealtime(api_key='your-openai-key', model='gpt-4o-realtime-preview')
```

Import the CN-only Qwen helper from `agora_agent.cn`:

```python
from agora_agent.cn import QwenOmni
```

## Avatar Vendors

Used with `agent.with_avatar()` in the cascading ASR + LLM + TTS pipeline. Some avatars require specific TTS sample rates — see [Avatar Integration](../guides/avatars.md).
Expand Down
6 changes: 4 additions & 2 deletions docs/guides/mllm-flow.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebar_position: 2
title: MLLM Flow (Multimodal)
description: Use OpenAI Realtime, Gemini Live, Vertex AI, or xAI for end-to-end audio processing.
description: Use supported global or CN realtime multimodal models for end-to-end audio processing.
---

# MLLM Flow (Multimodal)
Expand All @@ -11,9 +11,11 @@ The MLLM (Multimodal LLM) flow uses a single model to handle both audio input an
MLLM vendors supported by AgentKit:

- **OpenAI Realtime** — `gpt-4o-realtime-preview` and related models
- **Azure OpenAI Realtime** — global Azure-hosted realtime deployments
- **Gemini Live** — direct Google AI API access for audio-native Gemini models
- **Vertex AI** — Gemini Live through Google Cloud Vertex AI
- **xAI Grok** — xAI Realtime API
- **Qwen Omni** — Alibaba Cloud Qwen Omni Realtime for `Area.CN`

## Enable MLLM Mode

Expand Down Expand Up @@ -167,7 +169,7 @@ mllm = OpenAIRealtime(
|---|---|---|
| Latency | Lower — single model, no pipeline | Higher — three models in sequence |
| Voice control | Model-dependent | Full vendor choice for TTS |
| Vendor flexibility | Limited to supported MLLM providers (OpenAI Realtime, Gemini Live, Vertex AI, xAI Grok) | Mix and match LLM, TTS, and STT vendors |
| Vendor flexibility | Limited to supported global and CN MLLM providers | Mix and match LLM, TTS, and STT vendors |
| Audio understanding | Model hears tone, pacing, emotion | STT produces text only |

## Next Steps
Expand Down
8 changes: 4 additions & 4 deletions docs/guides/regional-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ client = Agora(
Bind `client` into `Agent(client=client, ...)` and construct vendors directly with SDK classes. The bound client selects `CNAgent` or `GlobalAgent` for IDE hints based on `area`, but does not restrict which vendor classes you can configure.
If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` clients and `AresSTT` for global clients.

| Client area | STT classes | LLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
| Client area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global client example:

Expand Down
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ The Agora Conversational AI Python SDK lets you build voice-powered AI agents on

**Cascading flow** uses ASR -> LLM -> TTS and supports the broadest set of vendor combinations.

**MLLM flow** uses a multimodal model such as OpenAI Realtime, Gemini Live, Vertex AI, or xAI Grok for end-to-end audio.
**MLLM flow** uses a multimodal model such as OpenAI Realtime, Azure OpenAI Realtime, Gemini Live, Vertex AI, xAI Grok, or Qwen Omni for end-to-end audio.

## Start here

- Start with [Quick Start](./getting-started/quick-start.md). It shows the baseline app-credentials setup and starts a cascading ASR -> LLM -> TTS agent.
- Use [MLLM Flow](./guides/mllm-flow.md) when your agent uses one realtime multimodal model, such as OpenAI Realtime, Gemini Live, Vertex AI, or xAI Grok.
- Use [MLLM Flow](./guides/mllm-flow.md) when your agent uses one supported realtime multimodal model.
- Use [Cascading Flow](./guides/cascading-flow.md) for more examples of the default ASR -> LLM -> TTS flow, including provider-specific configuration.

## How the SDK is organized
Expand All @@ -46,7 +46,7 @@ The Agora Conversational AI Python SDK lets you build voice-powered AI agents on
| [AgentSession](./concepts/session.md) | Manage the agent lifecycle |
| [Vendors](./concepts/vendors.md) | Browse all LLM, TTS, STT, MLLM, and Avatar providers |
| [Cascading Flow](./guides/cascading-flow.md) | Build an ASR -> LLM -> TTS pipeline |
| [MLLM Flow](./guides/mllm-flow.md) | Use OpenAI Realtime, Gemini Live, Vertex AI, or xAI Grok for end-to-end audio |
| [MLLM Flow](./guides/mllm-flow.md) | Use a global or CN realtime multimodal model for end-to-end audio |
| [Avatars](./guides/avatars.md) | Add a digital avatar with LiveAvatar, Akool, Anam, Generic Avatar, SenseTime, or Spatius |
| [Regional Routing](./guides/regional-routing.md) | Route requests to the nearest region |
| [Error Handling](./guides/error-handling.md) | Handle API errors with ApiError |
Expand Down
66 changes: 60 additions & 6 deletions docs/reference/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,18 @@ All vendor classes are available from `agora_agent`:

<!-- snippet: fragment -->
```python
from agora_agent import OpenAI, ElevenLabsTTS, DeepgramTTS, DeepgramSTT, OpenAIRealtime, XaiGrok, GenericAvatar
from agora_agent import AzureOpenAIRealtime, DeepgramSTT, OpenAI, TypecastTTS
from agora_agent.cn import QwenOmni
```

## Recommended vendors by area

Construct vendors directly from `agora_agent`, then bind a client with `Agent(client=client, ...)`. The bound client selects `CNAgent` or `GlobalAgent` for IDE hints based on `area`, but does not restrict which vendor classes you can configure.

| Area | STT classes | LLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
| Area | STT classes | LLM classes | MLLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `OpenAIRealtime`, `AzureOpenAIRealtime`, `GeminiLive`, `VertexAI`, `XaiGrok` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `TypecastTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `QwenOmni` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global example:

Expand Down Expand Up @@ -411,6 +412,16 @@ AgentKit serializes `credential_mode` at the top level of the Rime TTS configura
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Mistral TTS parameters, flattened into `params` |
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |

### `TypecastTTS`

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Typecast API key |
| `voice_id` | `str` | Yes | — | Typecast voice identifier |
| `model` | `str` | Yes | — | Typecast TTS model name, for example `ssfm-v30` |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Typecast parameters, flattened into `params` |
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |

### `MurfTTS`

| Parameter | Type | Required | Default | Description |
Expand Down Expand Up @@ -550,6 +561,7 @@ For `nova-2` and `nova-3`, omit `api_key` to use Agora-managed credentials. For

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `keywords` | `List[str]` | No | `None` | Hotwords that improve ASR accuracy |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional parameters |

### `SarvamSTT`
Expand Down Expand Up @@ -700,7 +712,9 @@ All CN TTS vendor classes support `skip_patterns` and `additional_params`.

#### `FengmingSTT`

No constructor parameters. Use `FengmingSTT()`.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `keywords` | `List[str]` | No | `None` | Hotwords that improve ASR accuracy |

#### `XfyunSTT`

Expand Down Expand Up @@ -789,6 +803,46 @@ No constructor parameters. Use `FengmingSTT()`.
| `params` | `Dict[str, Any]` | No | `None` | Additional parameters |
| `turn_detection` | `MllmTurnDetectionConfig` | No | `None` | MLLM turn detection configuration; overrides top-level `turn_detection` |

### `AzureOpenAIRealtime`

Global Azure OpenAI Realtime vendor (`mllm.vendor`: `"azure"`).

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Azure OpenAI API key |
| `url` | `str` | Yes | — | Azure OpenAI Realtime WebSocket URL |
| `model` | `str` | No | `None` | Model or deployment name |
| `voice` | `str` | No | `None` | Voice identifier |
| `instructions` | `str` | No | `None` | System instructions |
| `input_audio_transcription` | `Dict[str, Any]` | No | `None` | Audio transcription settings |
| `max_history` | `int` | No | `None` | Number of conversation history messages to cache; Azure-only MLLM field |
| `greeting_message` | `str` | No | `None` | Greeting message |
| `failure_message` | `str` | No | `None` | Message played when the model call fails |
| `output_modalities` | `List[str]` | No | `None` | Output modalities |
| `messages` | `List[Dict]` | No | `None` | Conversation messages |
| `params` | `Dict[str, Any]` | No | `None` | Additional Azure OpenAI parameters |
| `turn_detection` | `MllmTurnDetectionConfig` | Yes | — | Required MLLM turn detection configuration; overrides top-level `turn_detection` |

### `QwenOmni`

CN Alibaba Cloud Qwen Omni Realtime vendor (`mllm.vendor`: `"qwen_omni"`). Import it with `from agora_agent.cn import QwenOmni`.

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Alibaba Cloud Model Studio API key |
| `url` | `str` | No | `None` | Qwen Omni Realtime WebSocket URL |
| `model` | `str` | No | `None` | Qwen Omni Realtime model name |
| `voice` | `str` | No | `None` | Voice identifier |
| `instructions` | `str` | No | `None` | System instructions |
| `input_audio_transcription` | `Dict[str, Any]` | No | `None` | Audio transcription settings |
| `greeting_message` | `str` | No | `None` | Greeting message |
| `failure_message` | `str` | No | `None` | Message played when the model call fails |
| `input_modalities` | `List[str]` | No | `None` | Input modalities |
| `output_modalities` | `List[str]` | No | `None` | Output modalities |
| `messages` | `List[Dict]` | No | `None` | Conversation messages |
| `params` | `Dict[str, Any]` | No | `None` | Additional Qwen Omni parameters |
| `turn_detection` | `MllmTurnDetectionConfig` | Yes | — | Required MLLM turn detection configuration; overrides top-level `turn_detection` |

### `GeminiLive`

| Parameter | Type | Required | Default | Description |
Expand Down
6 changes: 6 additions & 0 deletions src/agora_agent/agentkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@
AresSTT,
AssemblyAISTT,
AzureOpenAI,
AzureOpenAIRealtime,
BaseAvatar,
BaseLLM,
BaseMLLM,
Expand Down Expand Up @@ -195,9 +196,11 @@
OpenAITTS,
GenericTTS,
RimeTTS,
QwenOmni,
SampleRate,
SarvamSTT,
SarvamTTS,
TypecastTTS,
SpeechmaticsSTT,
XaiSTT,
XaiTTS,
Expand Down Expand Up @@ -403,6 +406,7 @@
"MiniMaxTTS",
"MiniMaxCNTTS",
"MistralTTS",
"TypecastTTS",
"TencentTTS",
"BytedanceTTS",
"CosyVoiceTTS",
Expand All @@ -428,7 +432,9 @@
"XfyunDialectSTT",
"XfyunSTT",
"OpenAIRealtime",
"AzureOpenAIRealtime",
"GeminiLive",
"QwenOmni",
"VertexAI",
"XaiGrok",
"HeyGenAvatar",
Expand Down
14 changes: 13 additions & 1 deletion src/agora_agent/agentkit/regional_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import typing
from .agent import Agent
from .vendors.base import BaseAvatar, BaseLLM, BaseSTT, BaseTTS
from .vendors.base import BaseAvatar, BaseLLM, BaseMLLM, BaseSTT, BaseTTS
from .vendors.cn import (
AliyunLLM,
BytedanceDuplexTTS,
Expand All @@ -14,6 +14,7 @@
MiniMaxTTS as MiniMaxCNTTS,
MicrosoftSTT as MicrosoftCNSTT,
MicrosoftTTS as MicrosoftCNTTS,
QwenOmni,
SenseTimeAvatar,
SpatiusAvatar,
StepFunTTS,
Expand Down Expand Up @@ -47,6 +48,7 @@
OpenAI,
VertexAILLM,
)
from .vendors.mllm import AzureOpenAIRealtime, GeminiLive, OpenAIRealtime, VertexAI, XaiGrok
from .vendors.tts import (
AmazonTTS,
CartesiaTTS,
Expand All @@ -64,13 +66,15 @@
GenericTTS,
RimeTTS,
SarvamTTS,
TypecastTTS,
XaiTTS,
)
from .vendors.avatar import AkoolAvatar, AnamAvatar, GenericAvatar, HeyGenAvatar, LiveAvatarAvatar

CNSTT = typing.Union[TencentSTT, FengmingSTT, MicrosoftCNSTT, XfyunSTT, XfyunBigModelSTT, XfyunDialectSTT]
CNTTS = typing.Union[MiniMaxCNTTS, TencentTTS, BytedanceTTS, MicrosoftCNTTS, CosyVoiceTTS, BytedanceDuplexTTS, StepFunTTS, GenericTTS]
CNLLM = typing.Union[AliyunLLM, BytedanceLLM, DeepSeekLLM, TencentLLM]
CNMLLM = QwenOmni
CNAvatar = typing.Union[SenseTimeAvatar, SpatiusAvatar]

GlobalSTT = typing.Union[
Expand Down Expand Up @@ -103,6 +107,7 @@
DeepgramTTS,
GradiumTTS,
MistralTTS,
TypecastTTS,
]
GlobalLLM = typing.Union[
OpenAI,
Expand All @@ -115,6 +120,7 @@
Dify,
CustomLLM,
]
GlobalMLLM = typing.Union[OpenAIRealtime, AzureOpenAIRealtime, GeminiLive, VertexAI, XaiGrok]
GlobalAvatar = typing.Union[AkoolAvatar, LiveAvatarAvatar, AnamAvatar, GenericAvatar, HeyGenAvatar]


Expand All @@ -128,6 +134,9 @@ def with_llm(self, vendor: BaseLLM) -> "CNAgent":
def with_tts(self, vendor: BaseTTS) -> "CNAgent":
return typing.cast("CNAgent", super().with_tts(vendor))

def with_mllm(self, vendor: BaseMLLM) -> "CNAgent":
return typing.cast("CNAgent", super().with_mllm(vendor))

def with_avatar(self, vendor: BaseAvatar) -> "CNAgent":
return typing.cast("CNAgent", super().with_avatar(vendor))

Expand All @@ -142,6 +151,9 @@ def with_llm(self, vendor: BaseLLM) -> "GlobalAgent":
def with_tts(self, vendor: BaseTTS) -> "GlobalAgent":
return typing.cast("GlobalAgent", super().with_tts(vendor))

def with_mllm(self, vendor: BaseMLLM) -> "GlobalAgent":
return typing.cast("GlobalAgent", super().with_mllm(vendor))

def with_avatar(self, vendor: BaseAvatar) -> "GlobalAgent":
return typing.cast("GlobalAgent", super().with_avatar(vendor))

Expand Down
Loading
Loading