Skip to content

fix(rag): apply channel scope before retrieval limit - #27

Merged
johnnyhuy merged 1 commit into
mainfrom
agent/issue-25-channel-retrieval
Nov 10, 2026
Merged

johnnyhuy merged 1 commit into
mainfrom
agent/issue-25-channel-retrieval

Conversation

@johnnyhuy

@johnnyhuy johnnyhuy commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Apply the existing channel restriction inside Chroma before selecting the top-k results, so globally higher-ranked messages from other channels cannot crowd out useful context. Keep the payload safeguard and preserve unscoped retrieval and current document visibility.

Part of #3
Closes #25

Verification used a cleared environment, MISE_NO_ENV=1, frozen dependencies after installation, and UV_PYTHON=3.13. No real Slack or model tokens or .env files were read or changed.

  • mise run install passed with Python 3.13. The initial automatic Python 3.14 selection lacked a compatible ONNX wheel; rerunning with the configured Python version succeeded.
  • mise run test-backend passed, including five new local Chroma regressions. Before the implementation, the fixture produced four failures and one pass; API and Slack paths both forwarded empty context.
  • mise run test-frontend passed its existing placeholder command (No frontend tests yet).
  • mise run test-runtime passed with local listener permission. The first sandboxed attempt could not bind loopback listeners.
  • mise run dev-backend started successfully on 127.0.0.1:18025, returned HTTP 200 for health and the fixture ask request, then stopped cleanly.
  • git diff --check passed. No schema changes or migrations.
======================= 72 passed, 93 warnings in 3.46s ========================
tests/test_retrieval.py::test_channel_filter_runs_before_top_k PASSED
tests/test_retrieval.py::test_unscoped_search_preserves_global_ranking_and_explicit_k PASSED
tests/test_retrieval.py::test_empty_channel_does_not_fall_back_to_global_results PASSED
tests/test_retrieval.py::test_rag_payload_uses_scoped_matches[api] PASSED
tests/test_retrieval.py::test_rag_payload_uses_scoped_matches[slack] PASSED
runtime tests 7, pass 7, fail 0

The RAG fixture uses fixed embeddings and temporary local Chroma storage. Two COTHER messages rank ahead globally; the CTARGET query still returns both CTARGET messages. A missing channel returns no results without falling back to global retrieval. Chroma telemetry is disabled in the tests.

The HTTP capture used temporary SQLite/Chroma storage, mocked Slack authentication, blocked Slack API calls and a mock runtime. The answer below is a fixture response, not model-generated proof.

curl --fail --silent --show-error http://127.0.0.1:18025/health
# {"status":"healthy"}
curl --fail --silent --show-error http://127.0.0.1:18025/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"How do I connect?","channel_id":"CTARGET","request_id":"fixture-api-proof"}'
{"answer":"Fixture only - context forwarded to the mock runtime.","sources":[{"channel_id":"CTARGET","source":"slack"},{"channel_id":"CTARGET","source":"slack"}],"confidence":"low"}

The captured payload forwarded to the mock runtime was

{"request_id":"fixture-api-proof","conversation_id":"request:fixture-api-proof","question":"How do I connect?","context":"From slack:\nTarget channel first\n\nFrom slack:\nTarget channel second","sources":[{"channel_id":"CTARGET","source":"slack"},{"channel_id":"CTARGET","source":"slack"}]}

No dashboard UI changed, so no screenshot was needed. No Slack posts, DMs, Socket Mode connections or live model calls were made. Live Slack delivery is unverified by design. Shared documents and uploads without channel metadata retain their existing visibility; this slice does not introduce a sharing policy or an authorisation boundary. The separate empty-collection indexing bug discovered during fixture setup is tracked in #26.

GitHub CI passed lint-and-test and all three image builds at 16da4d4. CodeQL and Socket Security pull request checks also passed; the Socket Security project report was skipped.

Co-authored-by: OpenAI Codex <noreply@openai.com>
@johnnyhuy
johnnyhuy merged commit 85e1f8d into main Nov 10, 2026
10 checks passed
@johnnyhuy
johnnyhuy deleted the agent/issue-25-channel-retrieval branch November 10, 2026 18:17
@github-actions github-actions Bot mentioned this pull request Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply channel scope before the retrieval top-k limit

1 participant