Skip to content

feat(memory): learn from durable evidence and recover private sessions - #598

Open
jacoblee-io wants to merge 12 commits into
mainfrom
codex/private-space-oss
Open

jacoblee-io wants to merge 12 commits into
mainfrom
codex/private-space-oss

Conversation

@jacoblee-io

@jacoblee-io jacoblee-io commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Shared volumes tie private sessions to a Runtime, while unbounded recall can retain obsolete or forgotten instructions. Persist private session checkpoints in host-authorized object storage and replace legacy personal-memory indexes with bounded evidence learning in AgentBox. Users can reuse their own memory across Runtimes and resume completed sessions after Pod replacement.

Solution and upgrade

  • Keep ownership, writer leases, generation fencing, immutable object versions and idempotent commit receipts on the host. Restore the exact Pi entry tree and selected branch; unfinished execution requires explicit recovery and is never replayed automatically.
  • Learn from host-selected committed sources in a durable first stage, then organize evidence through an independent second-stage lease, revision and receipt. Preserve task goals, uncertainty, failed attempts and scoped corrections. Explicit remember/correct/forget operations retain source provenance and tombstones.
  • Supply small relevant briefs with a 1.5-second deadline, bounded body reads and context budgets; read longer evidence on demand. Recheck source authorization, expiry and supersession on return. Search exposure, explicit reads and final citations remain distinct usage signals.
  • Remove personal-memory embeddings, vector/FTS/BM25 indexes and old flush/watch/profile paths. Local and remote execution share retrieval contracts and ranking fixtures. Local memory uses per-user SQLite transactions; remote memory uses the host service without a local memory database. Memory cannot install Skills or grant tool permissions.

Deploy a compatible host API before Runtime/AgentBox. Export and verify historical state before enabling remote workspace mode. Object versions remain retained; forgetting learned memory does not erase source checkpoints. Existing spaces keep their storage placement; cross-bucket migration is not implemented. See the memory design and workspace operations guide.

Test Plan

  • Rebased on main at 479eb6b5; 85 targeted TypeScript test files / 1,540 tests and the TypeScript build passed.
  • Compatible host: 10 Go packages passed race tests; Web typecheck and four complete Linux amd64 image builds passed.
  • Deployed by image digest. Five real-model turns verified learning, owner-specific recall across two logical Runtimes, rejection of another user's checkpoint read (HTTP 403), and completed-session Pod replacement without PVCs or tool replay. All seven original Pi entries survived unchanged.
  • Earlier feature validation, before this rebase: 376 test files / 7,488 passed with one existing skip; 288 same-model synthetic evaluation trajectories passed. Those full suites were not rerun for this publication.

Both latest recall turns used one memory_search; the restored-session follow-up used no tools. This is not an official Codex/Claude product comparison or a geographical latency measurement. In-flight hard-kill recovery and earlier Helm coverage were not repeated in this deployment pass.

Architecture Checklist

  • Local and remote memory contracts have targeted coverage; live Pod recovery uses the remote workspace path.
  • No new shell execution authority or automatic Skill writer; source ownership and visibility remain host-enforced.
  • Personal-memory storage stays separate from Portal/Gateway configuration persistence.

General Checklist

  • One branch for private state persistence and evidence-backed memory.
  • No unrelated workspace changes or private deployment details included.

@nightmeng
nightmeng force-pushed the codex/private-space-oss branch from 427de6e to dbdff1a Compare September 13, 2026 15:12
Persist exact Pi branches and user files through a trusted object storage service, with owner-bound pods, fenced checkpoints and conservative personal memory. Keep configuration and Skill authority outside restored state, preserve workspace guards across rebuilds, and deliver SIGTERM to the application for final persistence.

Tested: full Vitest suite (7551 passed, 1 skipped), TypeScript/build, real MySQL/object storage recovery, two-user isolation and Linux Pod restart/shutdown checks.
Preserve failure classification across RPC and HTTP so temporary outages can recover before the existing local deadline. Keep terminal rejections and late replies fenced, bound stalled renewals, and enforce Linux file ownership for the actual process UID.

Tested: 7598 Vitest tests passed, 1 skipped; TypeScript check and build passed
Warn once per transient outage, report recovery and expiry, and keep private response bodies out of logs. Document the remaining 120-second server lease wait and its limits during crash recovery.

Tested: 7598 Vitest tests passed, 1 skipped; TypeScript check and build passed
Forward cancellation to HTTP and mTLS reads and bound missing heartbeats so a removed pod cannot leave the turn consumer waiting indefinitely. Keep Stop finalization intact when the transport aborts.

Tested: 23 related test files / 605 tests; TypeScript; build
… volumes

Without DAC_OVERRIDE, recursive chown cannot traverse a reused restricted directory assigned to the application group. Repair each credential type directory first so startup can retain the type-specific reader groups.
Keep remote recall selective and tied to literal source windows. Use independent search/read RPCs, shared turn budgets, and explicit remote backend requirements. Classify completed snapshots in coalesced background work so saved replies do not wait for learning.

Tested: 7617 Vitest cases, typecheck, build, live two-runtime isolation and source reads
Keep learning in AgentBox while the host selects committed evidence and fences publication. Bound recall and record explicit corrections and forgetting without installing skills or granting execution authority.

Tested: 7461 tests, typecheck, build, Helm matrix and two-runtime live checks
Keep the Portal capability picker and tool count aligned with the backend memory groups. Add a cross-layer contract assertion so future changes cannot silently drift.
Preserve task goals and failed attempts, organize cross-session topics with independent background work, and attribute only delivered evidence. Avoid redundant reads and bound optional recall so memory reduces repeated work without changing execution permissions.

Tested: 7488 tests passed, one existing skip; TypeScript; 288 real-model fixture runs; cross-runtime isolation, correction, forgetting, concurrent learning and completed-session Pod recovery.
@nightmeng
nightmeng force-pushed the codex/private-space-oss branch from a6e5ca3 to 7f25e2a Compare September 14, 2026 03:11
@jacoblee-io jacoblee-io changed the title feat(workspace): persist private sessions and memory in object storage feat(memory): learn from durable evidence and recover private sessions Sep 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant