Skip to content

Backfill BM25/vector for pre-v3.1 events (v3.2 / 61-01) #41

Description

@RichardHightower

Tracking F5 / backfill from docs/plans/v3.2-prove-it-plan.md.

README BM25 row: events indexed before v3.1 have empty text_preview and there is no backfill command. admin rebuild-bm25 is a prune, not a rebuild (CHANGELOG / docs/UPGRADING.md).

An import path is a backfill path. March origin/gsd/phase-56-import-bootstrap already has streaming ExportBackup / ImportBackup plus memory backup / memory import. Cherry-pick by feature after 59-02 (see docs/plans/phase-59-orphan-branch-triage.md); do not merge the branch wholesale (it would regress memory-orchestrator and memory-bench).

Work (61-01)

  • memory-daemon admin backfill-index --index bm25|vector|all: replay events from RocksDB into the index, reset the index checkpoint, resumable, idempotent, progress on stderr.
  • Build on the import/bootstrap code if the 59-02 cherry-pick landed it.
  • Update the README status row and docs/UPGRADING.md to say backfill exists.

Success

An e2e test ingests events with an old daemon build's schema (fixture), runs backfill, and memory search returns previews. A pre-v3.1 store can be backfilled without data loss.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestv3.2v3.2 Prove It milestone

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions