Skip to content

feat: migrate embeddings from Gunicorn service to Ollama / BYOK providers #146

Description

@jordanpartridge

Summary

Replace the custom Python/Gunicorn embedding service (knowledge-embeddings Docker container at :8001) with direct Ollama integration and support for bring-your-own-key (BYOK) providers like OpenAI.

Current State

  • Knowledge uses EmbeddingServiceInterface backed by a custom EmbeddingService that calls a Gunicorn Python server running BAAI/bge-large-en-v1.5 at http://localhost:8001/embed
  • This is the same model Ollama already serves locally (bge-large) on the same hardware
  • The Gunicorn container is a single point of failure with no scaling, no fallback, and extra operational overhead

Proposed Changes

  1. Add Ollama embedding provider — call Ollama's /api/embed endpoint directly (or via laravel/ai if feasible in Laravel Zero)
  2. Add OpenAI-compatible provider — support text-embedding-3-large or any OpenAI-compatible API with an API key
  3. Config-driven provider selection~/.knowledge/config.json or env vars to choose provider:
    {
      "embeddings": {
        "provider": "ollama",
        "model": "bge-large",
        "url": "http://100.68.122.24:11434"
      }
    }
  4. Kill knowledge-embeddings container — remove Docker service, update docs
  5. No vector re-indexing needed — same bge-large model produces identical 1024-dim vectors

Context

Acceptance Criteria

  • Embedding provider is configurable (ollama / openai / none)
  • Ollama provider works with existing bge-large model
  • OpenAI provider works with API key
  • knowledge-embeddings Docker container is removed
  • All existing tests pass, no vector migration needed

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions