Zoom Avatar: Tocqueville and Kant

This project creates two conversational AI avatars for political science education:

• Alexis de Tocqueville, grounded in Democracy in America
• Immanuel Kant, grounded in Fundamental Principles of the Metaphysic of Morals and Perpetual Peace

You speak to an avatar through your microphone. The application transcribes the question, retrieves relevant passages from that thinker's books, generates a response with a local Mistral model, speaks it aloud, and animates the avatar. The avatars can also be shown in Zoom or Google Meet through OBS.

This guide assumes no previous Python or AI experience. Follow the sections in order.

What the application does

Your microphone
    -> faster-whisper speech recognition
    -> ChromaDB passage retrieval
    -> Ollama + Mistral response generation
    -> Piper text-to-speech
    -> animated PyQt5 avatar
    -> optional OBS Virtual Camera for Zoom or Google Meet

The standard setup is local-first. It does not require a paid API key.

Before you begin

You need:

1. A computer with a microphone and speakers.
2. An internet connection for the first installation and model downloads.
3. Approximately 8–12 GB of free disk space.
4. Python 3.11.
5. Git.
6. Ollama.

The first launch is slower because the application downloads its embedding, speech-recognition, and voice models. Later launches use the local copies.

Install the prerequisites

macOS

Install Homebrew from brew.sh if it is not already installed. Then open Terminal and run:

brew install python@3.11 git portaudio libsndfile
brew install --cask ollama

Start the Ollama application once from Applications, or run ollama serve in a Terminal window.

When macOS asks, allow Terminal or Python to use the microphone.

Windows

Install:

1. Python 3.11. During installation, select Add Python to PATH.
2. Git for Windows.
3. Ollama for Windows.

Use PowerShell for the Windows commands in this guide. Start Ollama from the Start menu before continuing.

Linux

Install Python, Git, PortAudio, libsndfile, and common Qt runtime libraries. On Ubuntu or Debian:

sudo apt update
sudo apt install python3 python3-venv python3-pip git portaudio19-dev libsndfile1 ffmpeg \
  libegl1 libxkbcommon-x11-0 libxcb-xinerama0

Confirm that python3 --version reports Python 3.11. Install Ollama using the instructions at ollama.com/download/linux, then start it:

ollama serve

Linux ARM systems may not have a compatible PyQt5 wheel. Linux x86-64 is the supported Linux target for the desktop avatar.

Step 1: Clone the repository

Open Terminal on macOS/Linux or PowerShell on Windows.

git clone https://github.com/shahtirth07/zoom-avatar.git
cd zoom-avatar

All commands below must be run from the zoom-avatar directory.

Step 2: Create a Python virtual environment

A virtual environment keeps this project's packages separate from the rest of your computer.

macOS or Linux

python3.11 -m venv venv
source venv/bin/activate

If python3.11 is not recognized but python3 --version reports 3.11, use:

python3 -m venv venv
source venv/bin/activate

Windows PowerShell

py -3.11 -m venv venv
.\venv\Scripts\Activate.ps1

If PowerShell blocks the activation script, run this once in the same window and try again:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

When activation succeeds, the command prompt begins with (venv).

Step 3: Install the Python packages

Run this on every platform:

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Installation can take several minutes. It installs the desktop interface, vector database, speech recognition, text-to-speech, and meeting-related Python packages.

For automated Google Meet browser launch, also install Chromium:

python -m playwright install chromium

Linux may additionally require:

python -m playwright install-deps chromium

Step 4: Create the configuration file

The example configuration uses local Ollama, local embeddings, and local Piper speech.

macOS or Linux

cp .env.example .env

Windows PowerShell

Copy-Item .env.example .env

No API key is needed for the standard setup. Do not place real keys in .env.example, and never commit your personal .env file.

The important defaults in .env are:

LLM_PROVIDER=ollama
OLLAMA_MODEL=mistral
EMBEDDINGS_PROVIDER=local
TTS_PROVIDER=piper
WHISPER_MODEL=base

Step 5: Download the Mistral language model

Make sure Ollama is running, then run:

ollama pull mistral

Verify the installation:

ollama list

You should see mistral in the output.

Step 6: Build the Tocqueville and Kant vector databases

The raw books are included in data/raw/. This command cleans the texts, divides them into passages, creates embeddings, and stores them in vectordb/:

python -m src.ingest.build_corpus

The first build downloads all-MiniLM-L6-v2 and can take several minutes. A successful build reports:

tocqueville=394
kant=180

Kant's 180 passages consist of 66 passages from Fundamental Principles of the Metaphysic of Morals and 114 from Perpetual Peace.

The build is safe to run again. It synchronizes the collections instead of adding duplicate passages.

Step 7: Check the installation

Run:

python -m src.cli doctor

Resolve any line marked FAIL before continuing. In particular:

• If Ollama fails, start Ollama and rerun ollama pull mistral.
• If .env fails, repeat Step 4.
• If a package fails, reactivate venv and repeat Step 3.

Step 8: Start the avatars

Make sure the virtual environment is active and run:

python -m src.cli start

This starts the FastAPI server in the background and opens the avatar interface.

Try:

• “Hey Alexis, what is the tyranny of the majority?”
• “Hey Kant, what is the categorical imperative?”

Allow microphone access if your operating system asks. Stop the foreground avatar with Ctrl+C.

To stop the background server:

python -m src.cli stop

Other useful commands:

python -m src.cli status   # Show Ollama and server status
python -m src.cli server   # Start only the API server
python -m src.cli voice    # Start server plus voice/avatar interface

On macOS and Linux, ./run.sh start, ./run.sh status, and ./run.sh stop may also be used. The python -m src.cli ... commands are preferred because they work consistently across macOS, Linux, and Windows.

Optional: Rhubarb lip sync

The application works without Rhubarb, but mouth shapes will be less precise.

Download Rhubarb Lip Sync from its release page. Put the executable somewhere on your PATH, or set its full path in .env:

RHUBARB_PATH=/full/path/to/rhubarb
ENABLE_LIP_SYNC=true

To run without Rhubarb:

ENABLE_LIP_SYNC=false

Piper voice models are downloaded automatically into models/piper/ on first use. If Piper cannot load, the application falls back to the operating system's pyttsx3 voice.

Use the avatars in Zoom or Google Meet

The dependable meeting workflow is to capture the avatar window with OBS and send it to the meeting as a virtual camera. The Python meeting commands prepare the avatar application; OBS and audio routing are still configured manually.

Current limitations:

• The project does not automatically create two independent Zoom participants.
• Zoom's --use-sdk path is incomplete.
• Google Meet Media API output is incomplete.
• Remote meeting speech is not automatically routed into the avatar's speech recognizer.
• Browser and direct virtual-camera modes are experimental.

Install meeting software

macOS

Install:

• OBS Studio
• BlackHole 2ch

In Audio MIDI Setup, create a Multi-Output Device containing your speakers and BlackHole. This lets you hear the avatar while the meeting receives its audio.

Windows

Install:

• OBS Studio
• VB-CABLE or VoiceMeeter

Route the avatar's output to the virtual cable and select that cable as the meeting microphone.

Linux

Install OBS and the virtual-camera backend:

sudo apt install obs-studio v4l2loopback-dkms

Create a virtual audio sink/source using PipeWire or PulseAudio. Configuration varies by Linux distribution.

Recommended OBS workflow

1. Start the avatar meeting mode:

   python -m src.cli meeting

2. Open OBS.

3. Add a Window Capture source and select the avatar window.

4. Click Start Virtual Camera in OBS.

5. Join Zoom or Google Meet normally.

6. Select OBS Virtual Camera as the meeting camera.

7. Select BlackHole, VB-CABLE/VoiceMeeter, or your Linux virtual source as the meeting microphone.

8. Say “Hey Alexis” or “Hey Kant” near the computer's physical microphone.

Zoom helper command

python -m src.cli zoom-meeting MEETING_ID

For a meeting password:

python -m src.cli zoom-meeting MEETING_ID --password PASSWORD

This starts both avatars and prints/manual-guides the Zoom setup. It does not itself transmit two native Zoom participant streams. Join the meeting in the Zoom desktop client and use the OBS workflow above.

Google Meet helper command

With OBS:

python -m src.cli google-meet abc-defg-hij

Experimental browser launch without OBS:

python -m src.cli google-meet abc-defg-hij --no-obs --name "Philosophy Avatar"

The no-OBS path launches Playwright Chromium and attempts to expose the first avatar through pyvirtualcam. A supported virtual-camera backend is still required, and the meeting host may need to admit the browser participant.

Test the text API without using a microphone

Start the server:

python -m src.cli server

Open http://localhost:8000/docs to use FastAPI's interactive documentation.

Example request:

curl -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"query":"What protects liberty in a democracy?","persona":{"key":"tocqueville"}}'

Use "key":"kant" to ask Kant.

Configuration options

Edit .env to change providers or models.

Use OpenAI

LLM_PROVIDER=openai
OPENAI_API_KEY=your_key_here
OPENAI_MODEL=gpt-4

Use Anthropic

LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=your_key_here
ANTHROPIC_MODEL=claude-3-opus-20240229

Use a smaller Whisper model

If transcription is slow:

WHISPER_MODEL=tiny

The base model provides a better default accuracy/speed balance.

Troubleshooting

python, python3.11, or py is not recognized

Reinstall Python 3.11 and ensure it is added to PATH. Close and reopen Terminal or PowerShell afterward.

Virtual environment is not active

Activate it again:

# macOS/Linux
source venv/bin/activate

# Windows
.\venv\Scripts\Activate.ps1

Ollama connection error

Start Ollama:

ollama serve

In another window:

ollama pull mistral

No module named ...

Reactivate the virtual environment and run:

python -m pip install -r requirements.txt

Empty vector database or answers without source passages

Rebuild:

python -m src.ingest.build_corpus

No speech detected

• Allow microphone access for Terminal, PowerShell, or Python.
• Confirm the correct microphone is selected as your system input.
• Speak close to the microphone.
• Close another application if it has exclusive control of the microphone.

No sound or Piper error

• Confirm speakers/headphones are selected as the system output.
• Rerun the app while connected to the internet so Piper can download its model.
• The system will attempt to use pyttsx3 if Piper fails.

PyQt5 or display error on Linux

Install the Qt runtime packages listed in the Linux prerequisites. The application requires a graphical desktop and will not run in a headless SSH terminal.

Google Meet browser does not start

Run:

python -m playwright install chromium

On Linux also run:

python -m playwright install-deps chromium

OBS camera is not available in Zoom or Meet

• Start OBS Virtual Camera before opening the meeting's device settings.
• Restart Zoom or the browser after installing OBS.
• On macOS, approve the OBS camera extension in System Settings.
• On Linux, confirm v4l2loopback is loaded.

See server errors

The background server writes errors to:

server.log

Project layout

zoom-avatar/
├── README.md
├── requirements.txt
├── .env.example
├── data/raw/                  # Tracked primary-source books
├── data/processed/            # Generated cleaned texts
├── vectordb/                  # Generated ChromaDB collections
├── avatar/frames/             # Tocqueville and Kant visuals
├── avatar/videos/             # Optional pre-rendered clips
├── models/piper/              # Downloaded voice models
├── src/cli.py                 # Cross-platform launcher
├── src/voice_interaction.py   # Main voice/avatar loop
├── src/avatar_window.py       # Desktop avatar interface
├── src/ingest/                # Corpus preprocessing and embedding
├── src/rag/                   # Retrieval, prompts, and LLM calls
├── src/server/                # FastAPI backend
└── src/meeting_platforms/     # Zoom and Google Meet helpers

Corpus and retrieval settings

• Embedding model: all-MiniLM-L6-v2
• Chunking: sentence-boundary groups of approximately 500 words
• Chunk overlap: none
• Normal Q&A retrieval: top 2 passages
• Debate retrieval: top 3 passages
• Tocqueville collection: 394 passages
• Kant collection: 180 passages

Stopping and starting later

After the first installation, you do not repeat all setup steps.

macOS or Linux

cd zoom-avatar
source venv/bin/activate
python -m src.cli start

Windows PowerShell

cd zoom-avatar
.\venv\Scripts\Activate.ps1
python -m src.cli start

When finished:

python -m src.cli stop