EMBED_MODEL can be changed freely, but dims stays hardcoded at 384

[royalpinto007]

Desired outcome

Configuring a different embedding model fails loudly instead of silently producing wrong-width vectors.

Why it matters

app/embeddings.py hardcodes the dimension globally:

DIMS = 384  # all-MiniLM-L6-v2. If you change the model, change the schema's vector(384) too.

and LocalEmbedder claims it unconditionally:

class LocalEmbedder:
    dims = DIMS
    def __init__(self, model_name: str = "sentence-transformers/all-MiniLM-L6-v2") -> None:

but model_name is fully user-controlled. app/config.py exposes EMBED_MODEL as an env var, and app/cli.py passes it through at lines 43, 65, and 104. Set EMBED_MODEL to any other sentence-transformers model, say all-mpnet-base-v2 at 768 dimensions, and LocalEmbedder.dims still reports 384 while embed() returns 768-wide vectors. The failure surfaces far from the cause: an opaque pgvector error on insert against the vector(384) column in app/schema.sql, or, worse, a partially ingested corpus.

The comment already tells the reader what to keep in sync. Nothing enforces it.

Steps

1. In LocalEmbedder._load(), after constructing the SentenceTransformer, read the real width with model.get_sentence_embedding_dimension() and store it on the instance.
2. Raise a clear error when it does not match DIMS, naming the model, both dimensions, and the vector(384) column in app/schema.sql that must change too.
3. Make dims a property that returns the loaded value rather than the class constant, so callers cannot read a stale 384.
4. Add a note in README.md under configuration that changing EMBED_MODEL requires a schema change and a re-ingest.

Keep the lazy import: the module deliberately avoids pulling in torch at import time, and the check belongs inside _load() for that reason.

Claiming this

Comment below to claim it. A reply usually comes within a day.

[medlouaynjima]

Hi, I'd like to work on this issue. I plan to validate the embedding dimension when the SentenceTransformer model is loaded, expose the actual dimension through LocalEmbedder.dims, add regression tests for mismatched embedding dimensions, and update the configuration documentation as described.

Please let me know if you have any preferred implementation details or constraints before I start.

[github-actions]

It's yours, @medlouaynjima. Marked this as claimed.

Take your time, and shout in this thread if you get stuck. If there's no activity for 14 days the claim is released so someone else can pick it up.