I noticed Cortex already uses RAG-style retrieval for chat. I think an optional advanced mode could make this more transparent by showing the retrieved chunks before generation and eventually letting users adjust or select the context.
Possible first version
- Add a backend command that retrieves the same chunks chat would use, without calling the LLM.
- Show those chunks in a read-only context panel in the chat UI.
- Include source name, location, retrieval score, and maybe whether it came from vector search or keyword search.
Possible future extensions
If the first version works well, we could later add more advanced settings:
One extension would be letting users adjust how much retrieved context is sent to the LLM. For example, if the answer feels too shallow, the user could increase the number of retrieved chunks; if the answer feels unfocused or polluted by irrelevant context, they could reduce it. This could start as simple presets like “focused”, “balanced”, and “broad”, and later expose the actual top-k retrieval parameter for advanced users.
Another extension would be letting users manually choose the chunks used for generation. After Cortex retrieves likely relevant chunks, the UI could show them with source names, locations, and readable previews. The user could then include or exclude specific chunks before asking the LLM to generate an answer, quiz, flashcards, etc. This would make the model’s context much more transparent and give users a way to recover when automatic retrieval picks the wrong material.
A third extension could apply to notes/sources themselves. When converting a note into a source, or ingesting especially technical material, the user could choose how detailed the chunking should be. Some sources might benefit from smaller/more numerous chunks, especially dense or highly technical notes, while simpler sources could use the default chunking.
Longer term, Cortex could also explore more content-aware or semantic chunking. I noticed the current implementation in ingest.rs chunks mostly by a fixed character target with overlap, rather than dynamically based on headings, paragraphs, topic boundaries, or the type/technicality of the content. Improving would probably improve retrieval quality, because the chunks being embedded would map more closely to real concepts.
A basic version could offer presets like “normal”, “detailed”, or “high importance”; a later advanced version could expose chunk size and overlap directly.
Over time, this could become an optional “advanced mode” setting. General users could keep the current simple flow, while power users could inspect retrieved context, adjust retrieval breadth, and control how sources are chunked/indexed when they want more reliable outputs.
If you’re open to this, could I take the first version? I’d start small with a backend retrieval-preview command and a read-only context panel in chat. I think the first version could be read-only, but ideally structured as the foundation for selectable context later: retrieved chunk cards now, with include/exclude controls added in a follow-up PR.
I noticed Cortex already uses RAG-style retrieval for chat. I think an optional advanced mode could make this more transparent by showing the retrieved chunks before generation and eventually letting users adjust or select the context.
Possible first version
Possible future extensions
If the first version works well, we could later add more advanced settings:
One extension would be letting users adjust how much retrieved context is sent to the LLM. For example, if the answer feels too shallow, the user could increase the number of retrieved chunks; if the answer feels unfocused or polluted by irrelevant context, they could reduce it. This could start as simple presets like “focused”, “balanced”, and “broad”, and later expose the actual top-k retrieval parameter for advanced users.
Another extension would be letting users manually choose the chunks used for generation. After Cortex retrieves likely relevant chunks, the UI could show them with source names, locations, and readable previews. The user could then include or exclude specific chunks before asking the LLM to generate an answer, quiz, flashcards, etc. This would make the model’s context much more transparent and give users a way to recover when automatic retrieval picks the wrong material.
A third extension could apply to notes/sources themselves. When converting a note into a source, or ingesting especially technical material, the user could choose how detailed the chunking should be. Some sources might benefit from smaller/more numerous chunks, especially dense or highly technical notes, while simpler sources could use the default chunking.
Longer term, Cortex could also explore more content-aware or semantic chunking. I noticed the current implementation in
ingest.rschunks mostly by a fixed character target with overlap, rather than dynamically based on headings, paragraphs, topic boundaries, or the type/technicality of the content. Improving would probably improve retrieval quality, because the chunks being embedded would map more closely to real concepts.A basic version could offer presets like “normal”, “detailed”, or “high importance”; a later advanced version could expose chunk size and overlap directly.
Over time, this could become an optional “advanced mode” setting. General users could keep the current simple flow, while power users could inspect retrieved context, adjust retrieval breadth, and control how sources are chunked/indexed when they want more reliable outputs.
If you’re open to this, could I take the first version? I’d start small with a backend retrieval-preview command and a read-only context panel in chat. I think the first version could be read-only, but ideally structured as the foundation for selectable context later: retrieved chunk cards now, with include/exclude controls added in a follow-up PR.