From d340ddf6b0146b912895fa18bf540d986c101a8f Mon Sep 17 00:00:00 2001 From: Edvane Cristina Santos Daltro Date: Tue, 3 Nov 2026 05:51:59 -0400 Subject: [PATCH 1/2] Add scoped model binary receipt recovery and reconciliation --- README.md | 2 +- ROADMAP.md | 2 +- crates/cyber-app/tests/files.rs | 257 ++++++++++++++++ crates/cyber-core/src/binary/editor.rs | 179 ++++++++++- .../cyber-core/src/binary/editor/catalog.rs | 55 +++- crates/cyber-server/src/http/binary_files.rs | 11 + .../src/http/binary_files/model_recovery.rs | 218 ++++++++++++++ .../src/http/binary_files/recovery.rs | 8 +- crates/cyber-server/src/http/openapi.rs | 6 + docs/binary-editor.md | 38 ++- .../changes/add-binary-hex-editor/design.md | 18 ++ .../add-binary-hex-editor/implementation.md | 38 +++ .../specs/binary-editing/spec.md | 5 + sdk/openapi.json | 285 ++++++++++++++++-- sdk/typescript/src/binary.ts | 22 ++ sdk/typescript/src/client.ts | 4 +- sdk/typescript/src/generated/operations.ts | 69 +++++ sdk/typescript/test/binary.test.ts | 27 ++ 18 files changed, 1197 insertions(+), 47 deletions(-) create mode 100644 crates/cyber-server/src/http/binary_files/model_recovery.rs diff --git a/README.md b/README.md index d61ae70e8..1db81998a 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,7 @@ For independent patch attempts, host-recorded tests and a separate read-only jud Experimental [native computer use](docs/computer-use.md) adds macOS 14+ application/display screenshots and input with expiring owner grants, CLI/browser owner controls, private evidence and emergency Stop. Off by default; controlled macOS capture, input and Stop acceptance passed. Other layouts and signed distribution remain release checks. -The [binary workspace](docs/binary-editor.md) provides paged hex/ASCII viewing, literal hex/UTF-8 search, exact goto, bookmarks, byte copying and an LE/BE inspector. Open it with `cyber hex `, `/hex `, or `x` on an explorer file, without starting inference. Session read permissions apply. On admitted Linux/macOS workspaces, F2 prepares private staged edits with undo/redo; Ctrl+S reviews the exact byte changes, then explicitly saves. Reviewed Restore Last Save and retained draft recovery are also available through the owner API/SDK. Authorized Sessions can review shared saved history, and Session diffs show saved binary references independently of Git. The model-facing `binary` tool supports bounded inspect/read/search and exact reviewed patches in isolated invocation drafts. Checkpoint/retention/rewind adapters and remaining acceptance work are tracked in [#94](https://github.com/CyberdyneCorp/cybercode-cli/issues/94). +The [binary workspace](docs/binary-editor.md) provides paged hex/ASCII viewing, literal hex/UTF-8 search, exact goto, bookmarks, byte copying and an LE/BE inspector. Open it with `cyber hex `, `/hex `, or `x` on an explorer file, without starting inference. Session read permissions apply. On admitted Linux/macOS workspaces, F2 prepares private staged edits with undo/redo; Ctrl+S reviews the exact byte changes, then explicitly saves. Reviewed Restore Last Save and retained draft recovery are also available through the owner API/SDK. Authorized Sessions can review shared saved history, and Session diffs show saved binary references independently of Git. The model-facing `binary` tool supports bounded inspect/read/search and exact reviewed patches in isolated invocation drafts. Separate owner routes and SDK helpers discover and reconcile model-operation receipts. Checkpoint/retention/rewind adapters and remaining acceptance work are tracked in [#94](https://github.com/CyberdyneCorp/cybercode-cli/issues/94). ![Binary hex workspace with a shared hex/ASCII cursor and little-endian byte inspector](docs/screenshots/binary-hex.png) diff --git a/ROADMAP.md b/ROADMAP.md index 6904939b7..c9e56080b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -97,7 +97,7 @@ The [current inventory](docs/implementation/spec-doc-reconciliation.md) contains **Usage limit waits (`session-runtime`, `configuration`, `exec-mode`, `tui`, `web-client`, P1; `goals`, `loops-scheduling`, `workflows`, P2): Status: implemented** (change [`add-usage-limit-wait`](openspec/changes/add-usage-limit-wait/proposal.md)). An unattended run (`cyber exec`, a Goal, a loop iteration, a workflow agent, a teammate, or their subagents) that hits a subscription usage limit with a known reset waits for it and retries the Turn instead of failing: within `usage_limit.max_wait` (default 6 h), 10 s after the reset plus per-Session jitter, as one new attempt from the same history. The wait is durable (`session.usage_limit.waiting|resumed|abandoned.1`), cancelled by the normal interrupt, resumed after a restart for top-level Sessions and ended cleanly for owned ones; wall deadlines (Session, run, Goal and loop `max_wall_seconds`, `exec --timeout`) win over waiting. `exec` reports `usage_limit` stream-json lines, the TUI offers `/resume-at-reset` and a footer badge, and the dashboard shows the wait. The model fallback chain, once implemented, runs first. -**Binary hex editor (P1), in progress:** [`add-binary-hex-editor`](openspec/changes/add-binary-hex-editor/proposal.md) now provides the [live binary workspace](docs/binary-editor.md): paged hex/ASCII inspection, literal search, exact goto, bookmarks, LE/BE interpretation and scoped owner API/SDK access. The user prioritized issue #94 on 2026-11-01 ahead of the unfinished plugins/hooks and daily-coding work. The native core and owner API/SDK now provide durable staged patches, draft undo/redo, byte review and explicit recovery on admitted Linux/macOS workspaces. The terminal now stages edits with draft undo/redo and explicit reviewed save/Restore Last Save through the same owner API. Acknowledged saves feed the native FileChanged producer. Authorized Sessions can review the shared saved-version catalog and Session diffs include binary references. The model-facing `binary` tool supports bounded inspect/read/search and expected-digest patches through separate invocation drafts and the retained native save engine. Checkpoint/retention/rewind adapters and remaining acceptance are still open; this is not full editor acceptance. +**Binary hex editor (P1), in progress:** [`add-binary-hex-editor`](openspec/changes/add-binary-hex-editor/proposal.md) now provides the [live binary workspace](docs/binary-editor.md): paged hex/ASCII inspection, literal search, exact goto, bookmarks, LE/BE interpretation and scoped owner API/SDK access. The user prioritized issue #94 on 2026-11-01 ahead of the unfinished plugins/hooks and daily-coding work. The native core and owner API/SDK now provide durable staged patches, draft undo/redo, byte review and explicit recovery on admitted Linux/macOS workspaces. The terminal now stages edits with draft undo/redo and explicit reviewed save/Restore Last Save through the same owner API. Acknowledged saves feed the native FileChanged producer. Authorized Sessions can review the shared saved-version catalog and Session diffs include binary references. The model-facing `binary` tool supports bounded inspect/read/search and expected-digest patches through separate invocation drafts and the retained native save engine. Model-operation discovery/receipt reconciliation stays separate from editable manual handles. Checkpoint/retention/rewind adapters and remaining acceptance are still open; this is not full editor acceptance. **Untrusted-content defense (`untrusted-content`, `permissions-modes`, `system-context`, P1): Status: implemented** (change [`add-untrusted-content-defense`](openspec/changes/add-untrusted-content-defense/proposal.md)). It carries the `workspace-trust` provenance rule through every tool that returns third-party content. `webfetch`, `websearch`, MCP tool results, browser pages and MCP resources are returned in one escape-safe `` frame that replaces the MCP-resource frame. Hidden characters become visible markers, and a deterministic detector flags instruction-like text with a notice in the TUI, `exec` and the dashboard, without dropping content. After a flagged result, risky actions (network egress, writes outside the Location, classifier-allowed commands, secret reads) ask until the next user prompt, or are denied with no user attached; `bypass` shows the notice or refuses when configured. The base prompts cover the frame (version 3), and a labeled fixture set gates recall and false positives, with scripted end-to-end scenarios for secret reads, permission widening and provider redirection. The TUI and dashboard state each notice's consequence from the Location's effective `taint`/`bypass` settings (task 5.3). What remains: acceptance by hand on the real binary. diff --git a/crates/cyber-app/tests/files.rs b/crates/cyber-app/tests/files.rs index d2b65bcdc..ca0813682 100644 --- a/crates/cyber-app/tests/files.rs +++ b/crates/cyber-app/tests/files.rs @@ -1365,6 +1365,263 @@ async fn binary_immutable_read_only_open_refuses_preparation_even_with_edit_allo ); } +#[cfg(any(target_os = "macos", target_os = "linux"))] +fn retained_model(f: &Fixture, session: &str, id: &str, save: bool) { + use cyber_core::binary::{ + Document, Offset, + draft::{PatchBatch, Splice}, + editor::{DraftScope, Editor, Owner, SavePreparation}, + }; + let storage = f._app.runtime.binary_storage().unwrap(); + let owner = Owner { + session_id: session.into(), + directory: f.project.clone(), + root: f.project.clone(), + scope: DraftScope::Model { + operation: format!("{session}:{id}:canonical_payload"), + }, + }; + let mut document = Document::open(&f.project, "blob.bin").unwrap(); + let mut editor = Editor::model(&storage, id, owner, &mut document, || false).unwrap(); + let revision = editor.draft().revision(); + let before = editor.draft().page(&revision, Offset(0), 1).unwrap().bytes[0]; + editor + .patch( + "model_patch", + &PatchBatch { + revision, + splices: vec![Splice { + offset: Offset(0), + delete_length: Offset(1), + expected_hex: format!("{before:02x}"), + insert_hex: format!("{:02x}", before ^ 255), + }], + }, + ) + .unwrap(); + if save { + let review = editor.review(0, 128, || false).unwrap(); + let SavePreparation::Ready(prepared) = editor + .prepare_save("model_save", &review.fingerprint, || false) + .unwrap() + else { + panic!("new source save") + }; + assert_eq!( + editor.commit_save(*prepared).unwrap().outcome, + cyber_core::binary::receipts::Outcome::Acknowledged + ); + } +} + +#[cfg(any(target_os = "macos", target_os = "linux"))] +#[tokio::test] +async fn binary_model_receipt_recovery_stays_scoped_and_never_replays_source_edits() { + let f = Fixture::with(json!({"read":"allow", "edit":"allow"})).await; + let session = f.session().await; + retained_model(&f, &session, "bin_model_saved", true); + retained_model(&f, &session, "bin_model_unsaved", false); + let original = std::fs::read(f.project.join("blob.bin")).unwrap(); + let identity = cyber_core::binary::native::Target::capture(&f.project, Path::new("blob.bin")) + .unwrap() + .file; + let (status, listing) = f + .get(&format!( + "/files/binary/model/recovery?session_id={session}&limit=1" + )) + .await; + assert_eq!(status, 200, "{listing}"); + assert_eq!( + listing["data"]["documents"][0]["document_id"], + "bin_model_saved" + ); + let cursor = listing["data"]["next_cursor"].as_str().unwrap(); + let (status, next) = f + .get(&format!( + "/files/binary/model/recovery?session_id={session}&after={cursor}" + )) + .await; + assert_eq!(status, 200, "{next}"); + assert_eq!( + next["data"]["documents"][0]["document_id"], + "bin_model_unsaved" + ); + let (status, manual) = f + .get(&format!("/files/binary/recovery?session_id={session}")) + .await; + assert_eq!(status, 200, "{manual}"); + assert_eq!(manual["data"]["documents"], json!([])); + let path = "/files/binary/model/documents/bin_model_saved/operations/model_save"; + let (status, receipt) = f.get(&format!("{path}?session_id={session}")).await; + assert_eq!(status, 200, "{receipt}"); + assert_eq!(receipt["data"]["outcome"], "acknowledged"); + assert_eq!(receipt["data"]["file_effect"], "changed"); + let url = format!("{}/api/v1{path}?session_id={session}", f.base); + assert_eq!(f.http.get(&url).send().await.unwrap().status(), 401); + let authorized = f + .http + .get(&url) + .basic_auth("cyber", Some(PASSWORD)) + .header("x-cyber-directory", f.project.display().to_string()) + .send() + .await + .unwrap(); + assert_eq!(authorized.status(), 200); + assert!( + authorized.headers()["cache-control"] + .to_str() + .unwrap() + .contains("no-store") + ); + assert_eq!( + f.http + .get(&url) + .basic_auth("cyber", Some(PASSWORD)) + .header("x-cyber-directory", f.project.display().to_string()) + .header("Origin", "https://untrusted.invalid") + .send() + .await + .unwrap() + .status(), + 403 + ); + for _ in 0..2 { + let (status, reconciled) = f + .send( + "POST", + &format!("{path}/reconcile"), + Some(json!({"session_id":session})), + ) + .await; + assert_eq!(status, 200, "{reconciled}"); + assert_eq!(reconciled["data"], receipt["data"]); + } + let (status, absent) = f.get(&format!("/files/binary/model/documents/bin_model_unsaved/operations/model_save?session_id={session}")).await; + assert_eq!(status, 200, "{absent}"); + assert!(absent["data"].is_null()); + assert_ne!( + f.send( + "POST", + "/files/binary/documents/bin_model_saved/recover", + Some(json!({"session_id":session})) + ) + .await + .0, + 200 + ); + let opened = f.binary(&session, "blob.bin").await; + let manual_id = opened["document_id"].as_str().unwrap(); + let (status, prepared) = f + .send( + "POST", + &format!("/files/binary/documents/{manual_id}/prepare"), + Some(json!({"session_id":session,"revision":opened["revision"]})), + ) + .await; + assert_eq!(status, 200, "{prepared}"); + assert_eq!( + f.get(&format!( + "/files/binary/model/documents/{manual_id}/operations/model_save?session_id={session}" + )) + .await + .0, + 404 + ); + let other = f.session().await; + assert_eq!(f.get(&format!("{path}?session_id={other}")).await.0, 404); + let (status, hidden) = f + .get(&format!("/files/binary/model/recovery?session_id={other}")) + .await; + assert_eq!(status, 200, "{hidden}"); + assert_eq!(hidden["data"]["documents"], json!([])); + assert_eq!(std::fs::read(f.project.join("blob.bin")).unwrap(), original); + assert_eq!( + cyber_core::binary::native::Target::capture(&f.project, Path::new("blob.bin")) + .unwrap() + .file, + identity + ); + let mut config = f._model.config(); + config["permissions"] = json!({"read":"deny", "edit":"allow"}); + std::fs::write(f.root.join("config/cyber.json"), config.to_string()).unwrap(); + assert_eq!(f.get(&format!("{path}?session_id={session}")).await.0, 403); + assert_eq!( + f.send( + "POST", + &format!("{path}/reconcile"), + Some(json!({"session_id":session})) + ) + .await + .0, + 403 + ); + let (status, hidden) = f + .get(&format!( + "/files/binary/model/recovery?session_id={session}" + )) + .await; + assert_eq!(status, 200, "{hidden}"); + assert_eq!(hidden["data"]["documents"], json!([])); + assert!(f._model.requests().is_empty()); +} + +#[cfg(any(target_os = "macos", target_os = "linux"))] +#[tokio::test] +async fn binary_model_receipt_lookup_refreshes_read_authority_after_approval() { + for revoke in [false, true] { + let f = Fixture::with(json!({"read":"allow", "edit":"allow"})).await; + let session = f.session().await; + retained_model(&f, &session, "bin_model_review", true); + let mut config = f._model.config(); + config["permissions"] = json!({"read":"ask", "edit":"allow"}); + std::fs::write(f.root.join("config/cyber.json"), config.to_string()).unwrap(); + let (status, hidden) = f + .get(&format!( + "/files/binary/model/recovery?session_id={session}" + )) + .await; + assert_eq!(status, 200, "{hidden}"); + assert_eq!(hidden["data"]["documents"], json!([])); + let request = f.background("GET", format!("/files/binary/model/documents/bin_model_review/operations/model_save?session_id={session}"), None); + let mut asked = vec![]; + for _ in 0..500 { + asked = pending(&f, &session).await; + if !asked.is_empty() || request.is_finished() { + break; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert_eq!(asked.len(), 1, "{asked:?}"); + assert_eq!(asked[0]["action"], "read"); + if revoke { + config["permissions"] = json!({"read":"deny", "edit":"allow"}); + std::fs::write(f.root.join("config/cyber.json"), config.to_string()).unwrap(); + } + f.send( + "POST", + &format!( + "/sessions/{session}/permissions/{}/reply", + asked[0]["id"].as_str().unwrap() + ), + Some(json!({"reply":"once"})), + ) + .await; + let (status, result) = tokio::time::timeout(Duration::from_secs(5), request) + .await + .unwrap() + .unwrap(); + assert_eq!(status, if revoke { 403 } else { 200 }, "{result}"); + if !revoke { + assert_eq!(result["data"]["outcome"], "acknowledged"); + } + assert_eq!( + std::fs::read(f.project.join("blob.bin")).unwrap(), + [255, 159, 146, 150] + ); + assert!(f._model.requests().is_empty()); + } +} + #[cfg(any(target_os = "macos", target_os = "linux"))] #[tokio::test] async fn binary_closed_dirty_draft_is_recoverable_and_prevents_session_deletion() { diff --git a/crates/cyber-core/src/binary/editor.rs b/crates/cyber-core/src/binary/editor.rs index 3964e12b0..6e9311b63 100644 --- a/crates/cyber-core/src/binary/editor.rs +++ b/crates/cyber-core/src/binary/editor.rs @@ -68,7 +68,10 @@ impl BaseRef { } } fn open(&self, store: &Store) -> Result { - store.open_base(&self.name, self.length.0, &self.sha256) + self.open_cancellable(store, || false) + } + fn open_cancellable(&self, store: &Store, cancelled: impl Fn() -> bool) -> Result { + store.open_base_cancellable(&self.name, self.length.0, &self.sha256, cancelled) } } #[derive(Debug, Clone, Serialize, Deserialize)] @@ -212,7 +215,7 @@ impl Editor { if storage.resource(&owner, id)?.path() != document.resource().path() { return Err(Error::Changed); } - Self::resume(store, &owner) + Self::resume_for_resource(store, &owner, &document.resource(), cancelled) } Err(error) if error.kind() == std::io::ErrorKind::NotFound => { Self::create(store, owner, document, cancelled) @@ -268,19 +271,56 @@ impl Editor { record, }) } - pub fn resume(mut store: Store, owner: &Owner) -> Result { + pub fn resume(store: Store, owner: &Owner) -> Result { + Self::resume_cancellable(store, owner, || false) + } + pub fn resume_cancellable( + store: Store, + owner: &Owner, + cancelled: impl Fn() -> bool, + ) -> Result { + Self::resume_checked(store, owner, None, cancelled) + } + /// Bind the journal target to the exact resource already read-authorized by + /// the caller, before opening any potentially large private base image. + pub fn resume_for_resource( + store: Store, + owner: &Owner, + resource: &super::Resource, + cancelled: impl Fn() -> bool, + ) -> Result { + Self::resume_checked(store, owner, Some(resource), cancelled) + } + fn resume_checked( + mut store: Store, + owner: &Owner, + resource: Option<&super::Resource>, + cancelled: impl Fn() -> bool, + ) -> Result { + if cancelled() { + return Err(Error::Cancelled); + } let record: Record = serde_json::from_slice(&store.read_record("document.json")?) .map_err(|_| Error::Invalid)?; if record.version != 1 || &record.owner != owner || record.operations.len() > MAX_OPERATIONS || record.target.root != owner.root + || resource.is_some_and(|resource| { + record.target.root.join(&record.target.relative) != resource.path() + }) { return Err(Error::File); } store.ensure_history_lock()?; let pins = record.target.pins()?; - let draft = Draft::resume(record.base.open(&store)?, record.draft.clone())?; + let draft = Draft::resume( + record.base.open_cancellable(&store, &cancelled)?, + record.draft.clone(), + )?; + if cancelled() { + return Err(Error::Cancelled); + } Ok(Self { store, _pins: pins, @@ -711,18 +751,33 @@ impl Editor { let _effect = native::exchange(self.store.directory(), &intent.stage, &intent.prior); // Even an error may follow a native effect. Reconcile by identities and // content rather than retrying or attempting an unreviewed rollback. - self.settle(Some(prepared.after)) + self.settle(Some(prepared.after), || false) } pub fn reconcile(&mut self) -> Result { - self.settle(None) + self.reconcile_cancellable(|| false) } - fn settle(&mut self, after: Option>) -> Result { + /// Observe retained evidence only; cancellation never retries an exchange or + /// discards the unresolved intent. + pub fn reconcile_cancellable( + &mut self, + cancelled: impl Fn() -> bool, + ) -> Result { + self.settle(None, cancelled) + } + fn settle( + &mut self, + after: Option>, + cancelled: impl Fn() -> bool, + ) -> Result { self.store.validate()?; let intent = self.record.pending.as_ref().ok_or(Error::Invalid)?; let mut unknown = intent.operation.clone(); unknown.outcome = Outcome::Unknown; unknown.reason = Some("native_outcome_unknown_evidence_retained".into()); let verified = (|| { + if cancelled() { + return Err(Error::Cancelled); + } let installed_target = Target::capture(&intent.after.root, &intent.after.relative)?; if installed_target.file != intent.after.file || installed_target.parents != intent.after.parents @@ -736,8 +791,8 @@ impl Editor { let displaced_stamp = native::generation(&displaced)?; if native::id(&displaced)? != intent.prior.file || native::metadata(&displaced)? != intent.prior.metadata - || native::content(&installed, || false)? != intent.new_base.sha256 - || native::content(&displaced, || false)? != self.record.base.sha256 + || native::content(&installed, &cancelled)? != intent.new_base.sha256 + || native::content(&displaced, &cancelled)? != self.record.base.sha256 { return Err(Error::Changed); } @@ -757,12 +812,15 @@ impl Editor { let draft = match after { Some(draft) => draft, None => Draft::resume( - intent.new_base.open(&self.store)?, + intent.new_base.open_cancellable(&self.store, &cancelled)?, intent.after_draft.clone(), )?, }; // Preserve the prior intent if publishing this outcome fails. let (next, operation) = self.acknowledged_record(intent, installed_target, &draft); + if cancelled() { + return Ok(unknown); + } if persist(&mut self.store, &next).is_err() { return Ok(unknown); } @@ -1270,6 +1328,96 @@ mod tests { assert!(!editor.draft().can_undo()); } + #[test] + fn model_receipt_discovery_and_resume_are_separate_from_manual_drafts_and_other_locations() { + let (root, manual, _path, scope) = setup(); + let storage = + super::super::storage::StorageRoot::prepare(&root.path().join("data")).unwrap(); + let mut owner = scope.clone(); + owner.scope = DraftScope::Model { + operation: "call_one:payload".into(), + }; + let mut doc = Document::open(&scope.root, "bytes").unwrap(); + for id in ["bin_model_a", "bin_model_b"] { + let editor = Editor::model(&storage, id, owner.clone(), &mut doc, || false).unwrap(); + assert!(editor.lookup_operation("missing").unwrap().is_none()); + } + assert!(storage.model_owner(&scope, "bin_one").is_err()); + assert_eq!(storage.model_owner(&scope, "bin_model_a").unwrap(), owner); + let first = storage.list_model(&scope, None, 1).unwrap(); + assert_eq!(first.documents.len(), 1); + assert_eq!(first.documents[0].document_id, "bin_model_a"); + assert_eq!( + storage + .list_model(&scope, first.next_cursor.as_deref(), 1) + .unwrap() + .documents[0] + .document_id, + "bin_model_b" + ); + assert_eq!( + storage.list(&scope, None, 64).unwrap().documents[0].document_id, + "bin_one" + ); + assert!( + Editor::resume_cancellable( + storage.open_existing("bin_model_a").unwrap(), + &owner, + || true + ) + .is_err() + ); + assert!( + Editor::resume_cancellable( + storage.open_existing("bin_model_a").unwrap(), + &owner, + || false + ) + .is_ok() + ); + std::fs::write(scope.root.join("other"), b"other source").unwrap(); + let other = Document::open(&scope.root, "other").unwrap().resource(); + assert!( + Editor::resume_for_resource( + storage.open_existing("bin_model_a").unwrap(), + &owner, + &other, + || false + ) + .is_err() + ); + assert_foreign_model_scopes_are_refused(&storage, scope); + assert!(!manual.unresolved()); + } + + fn assert_foreign_model_scopes_are_refused( + storage: &super::super::storage::StorageRoot, + mut scope: Owner, + ) { + let original_session = scope.session_id.clone(); + scope.session_id = "ses_other".into(); + assert!(storage.model_owner(&scope, "bin_model_a").is_err()); + assert!( + storage + .list_model(&scope, None, 64) + .unwrap() + .documents + .is_empty() + ); + scope.session_id = original_session; + scope.directory = scope.root.join("nested"); + std::fs::create_dir(&scope.directory).unwrap(); + assert!(storage.model_owner(&scope, "bin_model_a").is_err()); + assert!( + storage + .list_model(&scope, None, 64) + .unwrap() + .documents + .is_empty() + ); + assert!(storage.model_owner(&scope, "../bin_model_a").is_err()); + } + #[test] fn owner_namespace_and_exact_location_cannot_be_rebound_on_resume() { let (root, editor, _path, owner) = setup(); @@ -1709,6 +1857,17 @@ mod tests { resumed.operation("save_cut").unwrap().outcome, Outcome::Unknown ); + assert_eq!( + resumed.reconcile_cancellable(|| true).unwrap().outcome, + Outcome::Unknown + ); + assert!(resumed.pending_operation().is_some()); + assert_eq!( + Target::capture(&owner.directory, Path::new("bytes")) + .unwrap() + .file, + installed + ); assert_eq!(resumed.reconcile().unwrap().outcome, Outcome::Acknowledged); assert_eq!(resumed.target().file, installed); assert_eq!(std::fs::read(path).unwrap(), [0, 255, 128, 27]); diff --git a/crates/cyber-core/src/binary/editor/catalog.rs b/crates/cyber-core/src/binary/editor/catalog.rs index 7338cce85..ff908c2e5 100644 --- a/crates/cyber-core/src/binary/editor/catalog.rs +++ b/crates/cyber-core/src/binary/editor/catalog.rs @@ -70,6 +70,24 @@ pub(super) fn publish_header( impl StorageRoot { /// Authenticate a small owner header before opening a potentially large draft. pub fn resource(&self, owner: &Owner, id: &str) -> Result { + let header = self.header(id)?; + if &header.owner != owner { + return Err(Error::File); + } + Ok(super::super::Resource(owner.root.join(header.relative))) + } + + /// Authenticate only this Session/Location's model namespace. The returned + /// owner is for receipt recovery, never promotion into a manual draft. + pub fn model_owner(&self, scope: &Owner, id: &str) -> Result { + let header = self.header(id)?; + if !model_scope(&header.owner, scope) { + return Err(Error::File); + } + Ok(header.owner) + } + + fn header(&self, id: &str) -> Result { if !id.starts_with("bin_") || id.len() > 100 || !id.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'_') @@ -87,7 +105,6 @@ impl StorageRoot { serde_json::from_slice(&small_record(&dir, "owner.json", HEADER_LIMIT)?) .map_err(|_| Error::Invalid)?; if header.version != 1 - || &header.owner != owner || header.relative.as_os_str().is_empty() || header .relative @@ -96,7 +113,7 @@ impl StorageRoot { { return Err(Error::File); } - Ok(super::super::Resource(owner.root.join(header.relative))) + Ok(header) } /// Deletion must not orphan dirty drafts or unresolved receipts, including closed @@ -137,6 +154,26 @@ impl StorageRoot { owner: &Owner, after: Option<&str>, limit: usize, + ) -> Result { + self.list_scoped(owner, after, limit, false) + } + + /// Bounded model receipt discovery reads small headers, never private drafts. + pub fn list_model( + &self, + scope: &Owner, + after: Option<&str>, + limit: usize, + ) -> Result { + self.list_scoped(scope, after, limit, true) + } + + fn list_scoped( + &self, + owner: &Owner, + after: Option<&str>, + limit: usize, + model: bool, ) -> Result { if limit == 0 || limit > 64 @@ -177,7 +214,12 @@ impl StorageRoot { let header: Header = serde_json::from_slice(&small_record(&dir, "owner.json", HEADER_LIMIT)?) .map_err(|_| Error::Invalid)?; - if header.version != 1 || &header.owner != owner { + let admitted = if model { + model_scope(&header.owner, owner) + } else { + &header.owner == owner + }; + if header.version != 1 || !admitted { continue; } if header.relative.as_os_str().is_empty() @@ -206,6 +248,13 @@ impl StorageRoot { } } +fn model_scope(owner: &Owner, scope: &Owner) -> bool { + owner.session_id == scope.session_id + && owner.directory == scope.directory + && owner.root == scope.root + && matches!(owner.scope, DraftScope::Model { .. }) +} + impl StorageRoot { /// Discover candidate namespaces by their small source header. No private /// command journal, base image or Session metadata is returned to the caller. diff --git a/crates/cyber-server/src/http/binary_files.rs b/crates/cyber-server/src/http/binary_files.rs index 49899d4d9..cc489fe06 100644 --- a/crates/cyber-server/src/http/binary_files.rs +++ b/crates/cyber-server/src/http/binary_files.rs @@ -24,6 +24,8 @@ pub(super) mod recovery; #[path = "binary_files/saving.rs"] pub(super) mod saving; +#[path = "binary_files/model_recovery.rs"] +pub(super) mod model_recovery; #[path = "binary_files/versions.rs"] pub(super) mod versions; @@ -75,6 +77,15 @@ pub struct BytePage { pub fn routes() -> Router { Router::new() + .route("/files/binary/model/recovery", get(model_recovery::list)) + .route( + "/files/binary/model/documents/{id}/operations/{operation}", + get(model_recovery::operation), + ) + .route( + "/files/binary/model/documents/{id}/operations/{operation}/reconcile", + post(model_recovery::reconcile), + ) .route("/files/binary/documents/{id}/versions", get(versions::list)) .route( "/files/binary/documents/{id}/versions/{document}/{version}/review", diff --git a/crates/cyber-server/src/http/binary_files/model_recovery.rs b/crates/cyber-server/src/http/binary_files/model_recovery.rs new file mode 100644 index 000000000..cc20a69f2 --- /dev/null +++ b/crates/cyber-server/src/http/binary_files/model_recovery.rs @@ -0,0 +1,218 @@ +//! Model receipt recovery never admits an invocation draft to the manual editor. +#[cfg(any(target_os = "macos", target_os = "linux"))] +use super::recovery::RecoveryEntry; +use super::recovery::{RecoveryList, RecoveryQuery}; +use super::*; +#[cfg(any(target_os = "macos", target_os = "linux"))] +use cyber_core::binary::{ + editor::{DraftScope, Editor, Owner}, + storage::StorageRoot, +}; + +#[derive(Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ScopeQuery { + pub session_id: String, +} + +#[cfg(any(target_os = "macos", target_os = "linux"))] +fn scope_owner(turn: &TurnContext, root: PathBuf) -> Result { + Ok(Owner { + session_id: turn.session_id.clone(), + directory: std::fs::canonicalize(&turn.directory).map_err(|_| Error::File)?, + root, + scope: DraftScope::Manual, + }) +} + +pub(super) async fn list( + State(state): State, + parts: Parts, + Query(query): Query, +) -> Result>, ApiError> { + #[cfg(not(any(target_os = "macos", target_os = "linux")))] + { + let _ = (state, parts, query); + Err(editing::unsupported()) + } + #[cfg(any(target_os = "macos", target_os = "linux"))] + { + let (turn, root, _) = files::scope(&state, &parts, Some(&query.session_id)).await?; + let scope = scope_owner(&turn, root.clone()).map_err(refused)?; + let authority = state.runtime.capture_child_admission(&turn.session_id)?; + let storage = state.runtime.binary_storage().map_err(refused)?; + let listing = work(&state, move || { + storage.list_model(&scope, query.after.as_deref(), query.limit.unwrap_or(64)) + }) + .await?; + let mut documents = Vec::new(); + for entry in listing.documents { + let decisions = state + .runtime + .file_decisions(&turn, "read", &[root.join(&entry.path)]) + .await; + if decisions.first() == Some(&FileDecision::Allow) { + documents.push(RecoveryEntry { + document_id: entry.document_id, + path: entry.path, + record_available: entry.record_available, + }); + } + } + authority.verify(&state.runtime, &turn.session_id)?; + let (fresh, fresh_root, _) = files::scope(&state, &parts, Some(&turn.session_id)).await?; + if fresh != turn || fresh_root != root { + return Err(ApiError::forbidden("Binary model recovery scope changed")); + } + // Recheck every disclosed name after bounded discovery, without prompts. + let paths: Vec<_> = documents + .iter() + .map(|entry| root.join(&entry.path)) + .collect(); + let decisions = state.runtime.file_decisions(&turn, "read", &paths).await; + documents = documents + .into_iter() + .zip(decisions) + .filter_map(|(entry, decision)| (decision == FileDecision::Allow).then_some(entry)) + .collect(); + Ok(Json(Located { + location: LocationInfo::of(FilePath::new(&turn.directory)), + data: RecoveryList { + documents, + next_cursor: listing.next_cursor, + }, + })) + } +} + +pub(super) async fn operation( + State(state): State, + parts: Parts, + Path((id, operation)): Path<(String, String)>, + Query(query): Query, +) -> Result>>, ApiError> { + inspect(state, parts, id, operation, query.session_id, false).await +} +pub(super) async fn reconcile( + State(state): State, + parts: Parts, + Path((id, operation)): Path<(String, String)>, + Json(input): Json, +) -> Result>>, ApiError> { + inspect(state, parts, id, operation, input.session_id, true).await +} + +#[cfg(any(target_os = "macos", target_os = "linux"))] +struct Admission { + owner: Owner, + read: ReadAuthority, + storage: Arc, + ancestor: crate::runtime::AdmissionAuthority, +} +#[cfg(any(target_os = "macos", target_os = "linux"))] +async fn admit( + state: &AppState, + parts: &Parts, + session: &str, + id: &str, +) -> Result { + let (turn, root, _) = files::scope(state, parts, Some(session)).await?; + let scope = scope_owner(&turn, root.clone()).map_err(refused)?; + let storage = state.runtime.binary_storage().map_err(refused)?; + let (owner, resource) = { + let (storage, id) = (storage.clone(), id.to_owned()); + work(state, move || { + let owner = storage.model_owner(&scope, &id)?; + let resource = storage.resource(&owner, &id)?; + Ok((owner, resource)) + }) + .await + .map_err(|_| ApiError::not_found("NotFoundError", "No such model binary document"))? + }; + let grant = permit_read(state, &turn, &resource).await?; + let read = ReadAuthority { + turn, + root, + grant, + resource, + }; + verify_read(state, parts, &read).await?; + let ancestor = state.runtime.capture_child_admission(session)?; + Ok(Admission { + owner, + read, + storage, + ancestor, + }) +} + +async fn inspect( + state: AppState, + parts: Parts, + id: String, + operation: String, + session: String, + reconcile: bool, +) -> Result>>, ApiError> { + #[cfg(not(any(target_os = "macos", target_os = "linux")))] + { + let _ = (state, parts, id, operation, session, reconcile); + Err(editing::unsupported()) + } + #[cfg(any(target_os = "macos", target_os = "linux"))] + { + let admission = admit(&state, &parts, &session, &id).await?; + let lifecycle = state.runtime.memory_mutation_lease().await?; + let cancel = tokio_util::sync::CancellationToken::new(); + let _cancel_on_disposal = cancel.clone().drop_guard(); + let runtime = state.runtime.clone(); + let (storage, owner, ancestor) = ( + admission.storage, + admission.owner, + admission.ancestor.clone(), + ); + let resource = admission.read.resource.clone(); + let result = work(&state, move || { + let _lifecycle = lifecycle; + ancestor + .verify(&runtime, &owner.session_id) + .map_err(|_| Error::Changed)?; + let mut editor = Editor::resume_for_resource( + storage.open_existing(&id)?, + &owner, + &resource, + || cancel.is_cancelled(), + )?; + editor.validate_read_resource()?; + runtime + .recover_binary_activity(editor.recovery_proof()?) + .map_err(|_| Error::Changed)?; + editor.lookup_operation(&operation)?; + if reconcile + && editor + .pending_operation() + .is_some_and(|pending| pending.id == operation) + { + editor.reconcile_cancellable(|| cancel.is_cancelled())?; + } + if cancel.is_cancelled() { + return Err(Error::Cancelled); + } + runtime.settle_binary_notification(&mut editor, &operation)?; + ancestor + .verify(&runtime, &owner.session_id) + .map_err(|_| Error::Changed)?; + editor.validate_read_resource()?; + // No callback dispatch or native replacement here. Existing unknown + // evidence and notification-pending state remain explicit. + editor.lookup_operation(&operation) + }) + .await?; + admission.ancestor.verify(&state.runtime, &session)?; + verify_read(&state, &parts, &admission.read).await?; + Ok(Json(Located { + location: LocationInfo::of(FilePath::new(&admission.read.turn.directory)), + data: result, + })) + } +} diff --git a/crates/cyber-server/src/http/binary_files/recovery.rs b/crates/cyber-server/src/http/binary_files/recovery.rs index 1a44f5413..5a0ff6642 100644 --- a/crates/cyber-server/src/http/binary_files/recovery.rs +++ b/crates/cyber-server/src/http/binary_files/recovery.rs @@ -130,9 +130,15 @@ pub(super) async fn recover( verify_read(&state, &parts, &authority).await?; let lifecycle = state.runtime.memory_mutation_lease().await?; let registry = state.runtime.clone(); + let resource = authority.resource.clone(); let result = work(&state, move || { let _lifecycle = lifecycle; - let editor = Editor::resume(storage.open_existing(&id)?, &owner)?; + let editor = Editor::resume_for_resource( + storage.open_existing(&id)?, + &owner, + &resource, + || false, + )?; editor.validate_read_resource()?; registry .recover_binary_activity(editor.recovery_proof()?) diff --git a/crates/cyber-server/src/http/openapi.rs b/crates/cyber-server/src/http/openapi.rs index d44857279..273e133e7 100644 --- a/crates/cyber-server/src/http/openapi.rs +++ b/crates/cyber-server/src/http/openapi.rs @@ -1591,6 +1591,12 @@ pub fn operations(g: &mut SchemaGenerator) -> Vec { .located().body::(g).ok::>(g), op("get", "/files/binary/recovery", "v1.fs.binaryRecovery", "Discover retained manual drafts in the exact Session Location") .located().query(&["session_id", "after", "limit"]).ok::>(g), + op("get", "/files/binary/model/recovery", "v1.fs.binaryModelRecovery", "Discover only this Session Location's retained model operation namespaces under allowed read rules") + .located().query(&["session_id", "after", "limit"]).ok::>(g), + op("get", "/files/binary/model/documents/{id}/operations/{operation}", "v1.fs.binaryModelOperation", "Read an invocation-bound receipt without opening its bytes in the manual editor") + .located().query(&["session_id"]).ok::>>(g), + op("post", "/files/binary/model/documents/{id}/operations/{operation}/reconcile", "v1.fs.binaryModelReconcile", "Observe retained native proof for a model operation; never replay a replacement") + .located().body::(g).ok::>>(g), op("post", "/files/binary/documents/{id}/recover", "v1.fs.binaryRecover", "Explicitly reopen a retained private draft; never replay a save or grant mutation authority") .located().body::(g).ok::>(g), op("get", "/files/binary/documents/{id}/operations/{operation}", "v1.fs.binaryOperation", "Query the durable receipt; absence is refused while journal storage is fenced") diff --git a/docs/binary-editor.md b/docs/binary-editor.md index 62a813f67..61010e697 100644 --- a/docs/binary-editor.md +++ b/docs/binary-editor.md @@ -251,6 +251,40 @@ The result is an explicit acknowledged/conflict/unknown operation receipt. The t is serialized and has no automatic retry. Its stable invocation/payload identity prevents reusing an operation for changed bytes. An unknown outcome must not be retried under a new call ID: preserve the document/operation identities for recovery. -The owner API does not admit model drafts as manual buffers; a dedicated model -recovery surface remains part of the remaining issue #94 acceptance work. Plugin +The owner API does not admit model drafts as manual buffers; model receipts are recovered through the separate endpoints below. Plugin registrations cannot invoke this built-in tool as a capability bypass. + + +### Recovering model operation receipts + +Model operation discovery and receipt lookup use a separate namespace: + +- `GET /api/v1/files/binary/model/recovery?session_id=…&after=…&limit=64` +- `GET /api/v1/files/binary/model/documents/{id}/operations/{operation}?session_id=…` +- `POST /api/v1/files/binary/model/documents/{id}/operations/{operation}/reconcile` + with `{ "session_id": "ses_…" }`. + +The exact owning Session and canonical Location must still match. Discovery returns +only model headers whose source has an explicit current read allowance; use a known +operation identity to request normal read approval when a rule says Ask. Manual +drafts are excluded, and these endpoints never return an editable document handle +or any private draft bytes. Authentication, Origin checks, current ancestor and read +permission checks and `no-store` remain effective. + +Lookup returns the durable receipt or explicit `null` for an absent operation in a +validated journal. Locked, missing or uncertain journals refuse absence instead. +Explicit reconciliation observes retained native evidence without another exchange. +It can acknowledge a proven installed result; an unprovable result stays unknown. +Cancellation preserves unresolved intent and source bytes. These reads do not +replay callbacks; a pending notification stays explicit until normal handoff settles. + +```ts +const listing = await client.binary.model.list(session.id); +const receipt = await client.binary.model.operation(session.id, documentId, operationId); +const observed = await client.binary.model.reconcile(session.id, documentId, operationId); +``` + +The SDK preserves the supplied identities and never retries reconciliation after a +transport failure. Retry lookup first. Discovery and reconciliation require neither +new model inference nor a source edit allowance. Source restore and abandoned model +draft cleanup remain separate from receipt recovery and from manual draft editing. diff --git a/openspec/changes/add-binary-hex-editor/design.md b/openspec/changes/add-binary-hex-editor/design.md index 604136868..ccfd99956 100644 --- a/openspec/changes/add-binary-hex-editor/design.md +++ b/openspec/changes/add-binary-hex-editor/design.md @@ -64,6 +64,24 @@ File I/O, hashing and encoding use bounded workers rather than blocking the serv Register one capability-owned model tool, `binary`, with `inspect`, `read`, `search` and `patch` operations. Read operations check ordinary `read`; patch additionally checks `edit`, code-restore protections and invocation-scoped write authority. Plan/read-only agents can inspect/search but cannot patch. The model receives bounded offset/hex/ASCII or match results, base fingerprints and explicit receipts; it cannot send a complete arbitrary file as unbounded base64 or invoke the owner's unsaved draft. A model patch owns a separate ephemeral draft, validates all expected bytes, passes ordinary permission review with its byte diff, then uses the same save engine and durable operation identity. No new provider request is required for manual editing. Hook/plugin code cannot bypass its declared filesystem capabilities by calling this API/tool. +### Model receipt recovery + +Model invocation namespaces remain distinct from manual document handles. Add +separate owner-authenticated model recovery discovery, operation lookup and explicit +reconciliation routes under `/files/binary/model`. Match the current Session, +canonical Location/root and a stored Model scope from the small private header +before loading its journal/base. Discovery discloses only currently allowed source +names; direct lookup uses normal read approval. Read/editor authority and ancestor +admission are refreshed before returning receipts. Never promote model commands +into the owner's manual editor or grant source write permission through recovery. + +Reconciliation observes retained evidence under the exclusive document lock, uses +bounded cancellable IO and never executes replacement again. Only matching trusted +filesystem activity may be settled by the existing borrowed document-lock proof; +generic native process activity remains untouched. Unknown results and pending +notification metadata stay explicit. Callback dispatch, source restore and cleanup +are separate actions, not side effects of receipt lookup. + ## Delivery and acceptance The user prioritized implementation on 2026-11-01 ahead of the unfinished first two workstreams. Preserve shared contracts and implement in this internal sequence: byte-service native identity/storage proof; bounded view/search and SDK; staged editing/save/recovery; themed TUI/direct CLI; model patch integration; snapshot/hooks and acceptance. Tests must cover every byte value, empty files, NUL/invalid UTF-8, offsets above 2^53, sparse files, EOF/overflow, same-size/same-mtime changes, denied reads/edits, partial nibbles, multi-splice atomicity, file growth/shrinkage, history, stale revisions and Session switches, lost acknowledgements, kill/restart around replacement and native alias races. Use real binary fixtures and hashes, TUI snapshots at wide/narrow/NO_COLOR widths plus real terminal captures, native platform positive/negative controls, generated schema/API/CLI checks and code/security review. See [implementation progress](implementation.md) for the unchanged-main browsing baseline, current viewer evidence and unperformed editor/native acceptance. diff --git a/openspec/changes/add-binary-hex-editor/implementation.md b/openspec/changes/add-binary-hex-editor/implementation.md index 14221a65e..7e1c35382 100644 --- a/openspec/changes/add-binary-hex-editor/implementation.md +++ b/openspec/changes/add-binary-hex-editor/implementation.md @@ -656,3 +656,41 @@ now lists the implemented tool. A real Plan-mode binary read golden retains byte bounds, fingerprint and source metadata, normalizing only the live revision and JSON key order. All 14 golden tests and strict OpenSpec validation (114 items) passed locally. Cross-platform CI must verify the corrected head before merge. + + +### Model operation receipt recovery (local follow-up) + +Separate `/files/binary/model` owner routes discover only the current Session and +canonical Location/root's model namespaces, query individual operation receipts and +explicitly reconcile retained native proof. Model headers do not enter manual +recovery or produce editable document handles. Discovery includes only explicitly +read-allowed names; known-ID lookup uses the ordinary read approval flow. Auth, +Origin, current read/ancestor scope, exclusive journal ownership, lifecycle/worker +capacity and response `no-store` remain effective. Absence is returned only from a +validated journal; locked/missing/fenced state is refused. + +Reconciliation never exchanges the source again or dispatches callbacks. Unknown +intent and notification-pending state remain explicit. Resume/reconciliation can +cancel bounded image/hash work while retaining proof. A shared resource-bound +resume checks the journal target against the already authorized header resource +before opening a base, for both manual recovery and model calls/recovery. + +The SDK adds `client.binary.model.list/operation/reconcile`, preserving identities +and the existing no-automatic-retry rule. Generated OpenAPI and SDK operations are +updated. Tests cover own/manual/other-Session/Location namespace isolation, paging, +missing operations, read denial and revocation after approval, auth/Origin/no-store, +repeated reconciliation with unchanged file identity/bytes, cancellation preserving +an installed unknown intent and explicit refusal of a different authorized resource. +The broader retention/checkpoint/rewind and remaining UI/platform/performance +acceptance work is still open; this increment does not close #94. + + +Local verification for model recovery: 20 production-router binary tests and 84 +binary-core tests passed (one process-death helper ignored in that selector). +Core/server/app all-target Clippy, formatting/diff checks and the current OpenAPI +artifact test passed. SDK typecheck and 127 unit/mock tests passed; ten configured +external SDK tests were skipped, and no new real-executable SDK run is claimed for +these recovery endpoints. Strict OpenSpec validation passed all 114 items. Source +review checked namespace/resource binding, permission waits, private-buffer +separation, cancellation and the existing filesystem-only recovery proof. Linux +and Windows recovery acceptance remain for CI after the preceding PR merges. diff --git a/openspec/changes/add-binary-hex-editor/specs/binary-editing/spec.md b/openspec/changes/add-binary-hex-editor/specs/binary-editing/spec.md index 1ea9447c3..be7846d9e 100644 --- a/openspec/changes/add-binary-hex-editor/specs/binary-editing/spec.md +++ b/openspec/changes/add-binary-hex-editor/specs/binary-editing/spec.md @@ -39,6 +39,11 @@ - **WHEN** a Session permits editing a path but denies reading its content - **THEN** binary preview, before/after disclosure and patch admission remain refused without leaking bytes +#### Scenario: Model receipt recovery stays outside the manual editor +- **WHEN** the owner queries a retained model operation after losing its response +- **THEN** lookup rechecks the same Session/Location and current source read authority, refuses manual or other-Session namespaces, and returns only operation metadata +- **AND** explicit reconciliation observes retained evidence without another source replacement, preserving unknown outcomes and cancellation evidence + ### Requirement: Binary draft and restore evidence retention (P1) Dirty drafts and unresolved saves SHALL survive disconnect/restart in owner-protected storage and SHALL require fresh scope/permission validation on resume. Clean abandoned drafts MAY be removed after 24 hours; unresolved mutation evidence SHALL NOT be expired by that rule. Every successful save SHALL retain byte-accurate preimage/receipt evidence and a separately reviewed conflict-aware Restore Last Save path, including Git/non-git files skipped by ordinary snapshot limits. Unknown effects SHALL remain unknown after local rewind; external checksums/signatures or remote effects SHALL NOT be repaired implicitly. diff --git a/sdk/openapi.json b/sdk/openapi.json index e762bbb04..4be451978 100644 --- a/sdk/openapi.json +++ b/sdk/openapi.json @@ -15611,22 +15611,6 @@ "record_available" ] }, - "BinaryRecoverRequest": { - "type": "object", - "additionalProperties": false, - "properties": { - "session_id": { - "type": "string" - }, - "read_only": { - "type": "boolean", - "default": false - } - }, - "required": [ - "session_id" - ] - }, "Located_Nullable_BinaryOperation": { "type": "object", "properties": { @@ -15649,6 +15633,34 @@ ], "description": "A Location-scoped response." }, + "BinaryReconcileRequest": { + "type": "object", + "additionalProperties": false, + "properties": { + "session_id": { + "type": "string" + } + }, + "required": [ + "session_id" + ] + }, + "BinaryRecoverRequest": { + "type": "object", + "additionalProperties": false, + "properties": { + "session_id": { + "type": "string" + }, + "read_only": { + "type": "boolean", + "default": false + } + }, + "required": [ + "session_id" + ] + }, "Located_BinaryVersions": { "type": "object", "properties": { @@ -15722,18 +15734,6 @@ "review" ] }, - "BinaryReconcileRequest": { - "type": "object", - "additionalProperties": false, - "properties": { - "session_id": { - "type": "string" - } - }, - "required": [ - "session_id" - ] - }, "Data_boolean": { "type": "object", "properties": { @@ -30154,6 +30154,235 @@ } } }, + "/api/v1/files/binary/model/recovery": { + "get": { + "operationId": "v1.fs.binaryModelRecovery", + "summary": "Discover only this Session Location's retained model operation namespaces under allowed read rules", + "tags": [ + "fs" + ], + "parameters": [ + { + "name": "session_id", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "after", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "location[directory]", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "x-cyber-directory", + "in": "header", + "required": false, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Located_BinaryRecoveryList" + } + } + } + }, + "default": { + "description": "Tagged error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + } + } + } + } + }, + "/api/v1/files/binary/model/documents/{id}/operations/{operation}": { + "get": { + "operationId": "v1.fs.binaryModelOperation", + "summary": "Read an invocation-bound receipt without opening its bytes in the manual editor", + "tags": [ + "fs" + ], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "operation", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "session_id", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "location[directory]", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "x-cyber-directory", + "in": "header", + "required": false, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Located_Nullable_BinaryOperation" + } + } + } + }, + "default": { + "description": "Tagged error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + } + } + } + } + }, + "/api/v1/files/binary/model/documents/{id}/operations/{operation}/reconcile": { + "post": { + "operationId": "v1.fs.binaryModelReconcile", + "summary": "Observe retained native proof for a model operation; never replay a replacement", + "tags": [ + "fs" + ], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "operation", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "location[directory]", + "in": "query", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "x-cyber-directory", + "in": "header", + "required": false, + "schema": { + "type": "string" + } + }, + { + "name": "Idempotency-Key", + "in": "header", + "required": false, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Located_Nullable_BinaryOperation" + } + } + } + }, + "default": { + "description": "Tagged error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorBody" + } + } + } + } + }, + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BinaryReconcileRequest" + } + } + } + } + } + }, "/api/v1/files/binary/documents/{id}/recover": { "post": { "operationId": "v1.fs.binaryRecover", diff --git a/sdk/typescript/src/binary.ts b/sdk/typescript/src/binary.ts index 2a198f37a..766b21ab1 100644 --- a/sdk/typescript/src/binary.ts +++ b/sdk/typescript/src/binary.ts @@ -6,6 +6,28 @@ import type { RequestOptions } from "./generated/operations.js"; const MAX_OFFSET = (1n << 64n) - 1n; const MAX_PAGE = 1024 * 1024; +/** Model receipts stay separate from editable manual document handles. */ +export function modelBinaryRecovery(client: CyberClient) { + const documentID = (id: string) => { + if (!/^bin_[a-zA-Z0-9_]{1,96}$/.test(id)) throw new TypeError("Invalid model binary document identity"); + }; + return { + async list(sessionID: string, after?: string, limit = 64, options?: RequestOptions) { + if (after !== undefined) documentID(after); + if (!Number.isSafeInteger(limit) || limit < 1 || limit > 64) throw new RangeError("Invalid model receipt page"); + return (await client.request("v1.fs.binaryModelRecovery", { query: { session_id: sessionID, after, limit } }, options)).data; + }, + async operation(sessionID: string, id: string, operation: string, options?: RequestOptions) { + documentID(id); operationID(operation); + return (await client.request("v1.fs.binaryModelOperation", { path: { id, operation }, query: { session_id: sessionID } }, options)).data; + }, + async reconcile(sessionID: string, id: string, operation: string, options?: RequestOptions) { + documentID(id); operationID(operation); + return (await client.request("v1.fs.binaryModelReconcile", { path: { id, operation }, body: { session_id: sessionID } }, options)).data; + }, + }; +} + /** Canonical wire offset. Numbers are deliberately excluded to prevent rounded offsets. */ export function byteOffset(value: bigint | string): string { if (typeof value !== "bigint" && typeof value !== "string") throw new TypeError("Use bigint or a decimal string for byte offsets"); diff --git a/sdk/typescript/src/client.ts b/sdk/typescript/src/client.ts index c197a1b9e..7cff5247c 100644 --- a/sdk/typescript/src/client.ts +++ b/sdk/typescript/src/client.ts @@ -1,4 +1,4 @@ -import { openBinaryDocument, recoverBinaryDocument, type BinaryDocument } from "./binary.js"; +import { openBinaryDocument, recoverBinaryDocument, modelBinaryRecovery, type BinaryDocument } from "./binary.js"; import { Connection, type ConnectOptions } from "./connection.js"; import { createGroups, @@ -62,6 +62,8 @@ export class CyberClient implements Caller { readonly #transport: Transport; readonly binary = { + /** Inspect retained invocation receipts without promoting private drafts. */ + model: modelBinaryRecovery(this), /** Reopen retained private commands/receipts without replaying a source save. */ recover: (sessionID: string, documentID: string, options: RequestOptions & { readOnly?: boolean } = {}): Promise => recoverBinaryDocument(this, sessionID, documentID, options), /** Open a live exact-byte view scoped to a Session and this client's Location. */ diff --git a/sdk/typescript/src/generated/operations.ts b/sdk/typescript/src/generated/operations.ts index 8de6bf665..ee98aab5f 100644 --- a/sdk/typescript/src/generated/operations.ts +++ b/sdk/typescript/src/generated/operations.ts @@ -256,6 +256,30 @@ export interface Operations { body: S.BinaryHistoryRequest; response: S.Located_BinaryEdited; }; + "v1.fs.binaryModelOperation": { + path: { id: string; operation: string }; + query: { + session_id?: QueryValue; + }; + body: undefined; + response: S.Located_Nullable_BinaryOperation; + }; + "v1.fs.binaryModelReconcile": { + path: { id: string; operation: string }; + query: Record; + body: S.BinaryReconcileRequest; + response: S.Located_Nullable_BinaryOperation; + }; + "v1.fs.binaryModelRecovery": { + path: Record; + query: { + session_id?: QueryValue; + after?: QueryValue; + limit?: QueryValue; + }; + body: undefined; + response: S.Located_BinaryRecoveryList; + }; "v1.fs.binaryOpen": { path: Record; query: Record; @@ -2213,6 +2237,42 @@ export const operations = { unwrap: false, stream: false, }, + "v1.fs.binaryModelOperation": { + tag: "fs", + name: "binaryModelOperation", + method: "GET", + path: "/api/v1/files/binary/model/documents/{id}/operations/{operation}", + pathParams: ["id", "operation"], + query: ["session_id"], + body: "none", + located: true, + unwrap: false, + stream: false, + }, + "v1.fs.binaryModelReconcile": { + tag: "fs", + name: "binaryModelReconcile", + method: "POST", + path: "/api/v1/files/binary/model/documents/{id}/operations/{operation}/reconcile", + pathParams: ["id", "operation"], + query: [], + body: "required", + located: true, + unwrap: false, + stream: false, + }, + "v1.fs.binaryModelRecovery": { + tag: "fs", + name: "binaryModelRecovery", + method: "GET", + path: "/api/v1/files/binary/model/recovery", + pathParams: [], + query: ["session_id", "after", "limit"], + body: "none", + located: true, + unwrap: false, + stream: false, + }, "v1.fs.binaryOpen": { tag: "fs", name: "binaryOpen", @@ -5253,6 +5313,15 @@ export function createGroups(caller: Caller) { /** Undo, redo or discard local draft commands without changing the source (`POST /api/v1/files/binary/documents/{id}/history`) */ binaryHistory: (id: string, body: Operations["v1.fs.binaryHistory"]["body"], options?: RequestOptions): Promise => caller.call("v1.fs.binaryHistory", { path: { id }, body }, options), + /** Read an invocation-bound receipt without opening its bytes in the manual editor (`GET /api/v1/files/binary/model/documents/{id}/operations/{operation}`) */ + binaryModelOperation: (id: string, operation: string, query?: Operations["v1.fs.binaryModelOperation"]["query"], options?: RequestOptions): Promise => + caller.call("v1.fs.binaryModelOperation", { path: { id, operation }, query }, options), + /** Observe retained native proof for a model operation; never replay a replacement (`POST /api/v1/files/binary/model/documents/{id}/operations/{operation}/reconcile`) */ + binaryModelReconcile: (id: string, operation: string, body: Operations["v1.fs.binaryModelReconcile"]["body"], options?: RequestOptions): Promise => + caller.call("v1.fs.binaryModelReconcile", { path: { id, operation }, body }, options), + /** Discover only this Session Location's retained model operation namespaces under allowed read rules (`GET /api/v1/files/binary/model/recovery`) */ + binaryModelRecovery: (query?: Operations["v1.fs.binaryModelRecovery"]["query"], options?: RequestOptions): Promise => + caller.call("v1.fs.binaryModelRecovery", { query }, options), /** Open a Session-owned live binary view; no eager whole-file copy or hash (`POST /api/v1/files/binary/documents`) */ binaryOpen: (body: Operations["v1.fs.binaryOpen"]["body"], options?: RequestOptions): Promise => caller.call("v1.fs.binaryOpen", { body }, options), diff --git a/sdk/typescript/test/binary.test.ts b/sdk/typescript/test/binary.test.ts index 4f5ed0610..1264231e9 100644 --- a/sdk/typescript/test/binary.test.ts +++ b/sdk/typescript/test/binary.test.ts @@ -116,6 +116,33 @@ test("binary mutation transport failure requires explicit recovery instead of au assert.equal(calls.length, 1, "lost open response allocated another document automatically"); }); +test("model binary receipt helpers keep Session scope separate from manual handles", async () => { + const receipt = { id: "save_once", kind: "save", payload_sha256: "a".repeat(64), outcome: "unknown", revision: "draft_a", file_effect: "unknown", reason: "native_outcome_requires_reconciliation", notification_pending: false }; + const { client, calls } = mockClient(req => json(200, { data: req.url.pathname.endsWith("/recovery") + ? { documents: [{ document_id: "bin_model", path: "fixture.bin", record_available: true }], next_cursor: null } : receipt })); + const listing = await client.binary.model.list("ses_one", undefined, 1); + assert.equal(listing.documents[0]!.document_id, "bin_model"); + assert.equal(calls[0]!.url.searchParams.get("session_id"), "ses_one"); + assert.equal(calls[0]!.url.searchParams.get("limit"), "1"); + assert.deepEqual(await client.binary.model.operation("ses_one", "bin_model", "save_once"), receipt); + assert.equal(calls[1]!.url.pathname.endsWith("/model/documents/bin_model/operations/save_once"), true); + assert.deepEqual(await client.binary.model.reconcile("ses_one", "bin_model", "save_once"), receipt); + assert.equal(calls[2]!.method, "POST"); + assert.deepEqual(JSON.parse(calls[2]!.body!), { session_id: "ses_one" }); + assert.equal(calls.some(call => call.url.pathname.endsWith("/recover") || call.url.pathname.endsWith("/save")), false); + const before = calls.length; + await assert.rejects(() => client.binary.model.operation("ses_one", "../bin_model", "save_once")); + await assert.rejects(() => client.binary.model.reconcile("ses_one", "bin_model", "../save_once")); + await assert.rejects(() => client.binary.model.list("ses_one", undefined, 65)); + assert.equal(calls.length, before); +}); + +test("model binary reconciliation never retries a lost response", async () => { + const { client, calls } = mockClient(() => { throw new TypeError("response lost"); }, { retry: { retries: 4, delayMs: 0 } }); + await assert.rejects(() => client.binary.model.reconcile("ses_one", "bin_model", "save_once")); + assert.equal(calls.length, 1); +}); + test("binary save and restore preserve explicit review fingerprints and update metadata", async () => { const status = { ...opened, length: "4", revision: "saved", source: "draft", read_only: false, prepared: true, dirty: false, can_undo: false, can_redo: false, base_sha256: "a".repeat(64), pending_operation: null, edit_supported: true, edit_decision: "allow" }; From 6cfb7f2b18e0fcff1279956ce9d0100922d6fc6a Mon Sep 17 00:00:00 2001 From: Edvane Cristina Santos Daltro Date: Tue, 3 Nov 2026 06:34:43 -0400 Subject: [PATCH 2/2] Measure binary editor latency and memory on a 16 MiB fixture --- crates/cyber-core/Cargo.toml | 4 + crates/cyber-core/benches/binary_editor.rs | 218 ++++++++++++++++++ docs/evidence/binary-editor-16mib/README.md | 64 +++++ .../binary-editor-16mib/apple-m2-dev-time.txt | 18 ++ .../binary-editor-16mib/apple-m2-dev.json | 133 +++++++++++ .../apple-m2-release-time.txt | 18 ++ .../binary-editor-16mib/apple-m2-release.json | 133 +++++++++++ docs/evidence/binary-view/README.md | 6 + .../add-binary-hex-editor/implementation.md | 15 ++ .../changes/add-binary-hex-editor/tasks.md | 16 +- 10 files changed, 617 insertions(+), 8 deletions(-) create mode 100644 crates/cyber-core/benches/binary_editor.rs create mode 100644 docs/evidence/binary-editor-16mib/README.md create mode 100644 docs/evidence/binary-editor-16mib/apple-m2-dev-time.txt create mode 100644 docs/evidence/binary-editor-16mib/apple-m2-dev.json create mode 100644 docs/evidence/binary-editor-16mib/apple-m2-release-time.txt create mode 100644 docs/evidence/binary-editor-16mib/apple-m2-release.json diff --git a/crates/cyber-core/Cargo.toml b/crates/cyber-core/Cargo.toml index 8fdde2cd2..8d6f564b0 100644 --- a/crates/cyber-core/Cargo.toml +++ b/crates/cyber-core/Cargo.toml @@ -54,5 +54,9 @@ windows-sys = { version = "0.61.2", features = ["Win32_System_Memory"] } name = "untrusted_detector" harness = false +[[bench]] +name = "binary_editor" +harness = false + [lints] workspace = true diff --git a/crates/cyber-core/benches/binary_editor.rs b/crates/cyber-core/benches/binary_editor.rs new file mode 100644 index 000000000..d1623067d --- /dev/null +++ b/crates/cyber-core/benches/binary_editor.rs @@ -0,0 +1,218 @@ +//! Reproducible core-only 16 MiB timings. No server, terminal or provider is used. +//! Build with `cargo build -p cyber-core --bench binary_editor --profile dev`; +//! run the resulting executable directly under an OS RSS measurement tool. +#[cfg(any(target_os = "macos", target_os = "linux"))] +mod native { + use cyber_core::binary::{ + Document, Error, Offset, SearchRequest, + draft::{ImmutableBase, PatchBatch, Splice, encode_hex}, + editor::{DraftScope, Editor, Owner, SavePreparation}, + receipts::Outcome, + storage::StorageRoot, + }; + use serde_json::{Value, json}; + use std::{ + cell::Cell, + fs::File, + io::{Seek, SeekFrom, Write}, + path::Path, + time::Instant, + }; + + const SIZE: u64 = 16 * 1024 * 1024; + const RUNS: usize = 5; + const NEEDLE: &[u8] = &[255, 0, 128, 27, 254, 31, 0]; + + fn ms(started: Instant) -> f64 { + started.elapsed().as_secs_f64() * 1000.0 + } + fn fixture(path: &Path) { + let mut file = File::create(path).unwrap(); + let chunk: Vec = (0..65536).map(|i| (i % 251) as u8).collect(); + for _ in 0..SIZE / chunk.len() as u64 { + file.write_all(&chunk).unwrap(); + } + file.seek(SeekFrom::End(-(NEEDLE.len() as i64))).unwrap(); + file.write_all(NEEDLE).unwrap(); + file.sync_all().unwrap(); + } + fn search(document: &mut Document) -> Value { + let mut request = SearchRequest { + revision: document.metadata().revision, + hex: encode_hex(NEEDLE), + offset: Offset(0), + }; + let (started, mut longest, mut slices, mut offsets) = + (Instant::now(), 0.0f64, 0, Vec::new()); + loop { + let slice = Instant::now(); + let result = document.search(&request, || false).unwrap(); + longest = longest.max(ms(slice)); + slices += 1; + offsets.extend(result.offsets); + let Some(next) = result.next_offset else { + break; + }; + assert!(next.0 > request.offset.0 && slices < 1024); + request.offset = next; + } + assert_eq!(offsets, vec![Offset(SIZE - NEEDLE.len() as u64)]); + json!({"total_ms":ms(started),"longest_slice_ms":longest,"slices":slices,"match_offset":offsets[0]}) + } + fn batch(editor: &Editor) -> PatchBatch { + let revision = editor.draft().revision(); + let splice = |offset, delete, inserted: &[u8]| Splice { + offset: Offset(offset), + delete_length: Offset(delete), + expected_hex: if delete == 0 { + String::new() + } else { + encode_hex( + &editor + .draft() + .page(&revision, Offset(offset), delete as usize) + .unwrap() + .bytes, + ) + }, + insert_hex: encode_hex(inserted), + }; + PatchBatch { + revision: revision.clone(), + splices: vec![ + splice(1024 * 1024, 2, &[255, 128]), + splice(8 * 1024 * 1024, 0, &[0, 255, 128]), + splice(12 * 1024 * 1024, 4, &[]), + ], + } + } + fn commit(editor: &mut Editor, preparation: SavePreparation) { + let operation = match preparation { + SavePreparation::Existing(operation) => operation, + SavePreparation::Ready(prepared) => editor.commit_save(*prepared).unwrap(), + }; + assert_eq!(operation.outcome, Outcome::Acknowledged); + } + fn one(index: usize) -> Value { + let temp = tempfile::tempdir().unwrap(); + let root = temp.path().canonicalize().unwrap(); + let project = root.join("project"); + std::fs::create_dir(&project).unwrap(); + let storage = StorageRoot::prepare(&root).unwrap(); + let path = project.join("bytes.bin"); + fixture(&path); + let started = Instant::now(); + let mut document = Document::open(&project, "bytes.bin").unwrap(); + assert_eq!( + document + .page(&document.metadata().revision, Offset(0), 65536) + .unwrap() + .bytes + .len(), + 65536 + ); + let first_page_ms = ms(started); + let search_result = search(&mut document); + let polls = Cell::new(0); + let started = Instant::now(); + let result = document.search( + &SearchRequest { + revision: document.metadata().revision, + hex: encode_hex(NEEDLE), + offset: Offset(0), + }, + || { + polls.set(polls.get() + 1); + polls.get() >= 3 + }, + ); + assert!(matches!(result, Err(Error::Cancelled))); + let cancellation_ms = ms(started); + let owner = Owner { + session_id: format!("ses_bench_{index}"), + directory: project.clone(), + root: project, + scope: DraftScope::Manual, + }; + let started = Instant::now(); + let mut editor = Editor::create( + storage.open(&format!("bin_bench_{index}")).unwrap(), + owner, + &mut document, + || false, + ) + .unwrap(); + let prepare_ms = ms(started); + let original_sha256 = editor.draft().base().sha256().to_owned(); + let patch = batch(&editor); + let started = Instant::now(); + editor.patch("patch", &patch).unwrap(); + let patch_ms = ms(started); + let started = Instant::now(); + editor + .history("undo", &editor.draft().revision(), false) + .unwrap(); + let undo_ms = ms(started); + let started = Instant::now(); + editor + .history("redo", &editor.draft().revision(), true) + .unwrap(); + let redo_ms = ms(started); + let started = Instant::now(); + let review = editor.review(0, 128, || false).unwrap(); + let review_ms = ms(started); + assert_eq!(review.draft.proposed_length, Offset(SIZE - 1)); + let started = Instant::now(); + let prepared = editor + .prepare_save("save", &review.fingerprint, || false) + .unwrap(); + commit(&mut editor, prepared); + let save_ms = ms(started); + let mut installed = Document::open(editor.target().root.as_path(), "bytes.bin").unwrap(); + assert_eq!( + installed.fingerprint(|| false).unwrap(), + review.draft.proposed_sha256 + ); + let started = Instant::now(); + let review = editor.restore_review(|| false).unwrap(); + let restore_review_ms = ms(started); + let started = Instant::now(); + let prepared = editor + .prepare_restore("restore", &review.fingerprint, || false) + .unwrap(); + commit(&mut editor, prepared); + let restore_ms = ms(started); + let mut restored = Document::open(editor.target().root.as_path(), "bytes.bin").unwrap(); + assert_eq!(restored.fingerprint(|| false).unwrap(), original_sha256); + assert_eq!(std::fs::metadata(path).unwrap().len(), SIZE); + json!({"first_page_ms":first_page_ms,"prepare_ms":prepare_ms,"patch_ms":patch_ms,"undo_ms":undo_ms,"redo_ms":redo_ms,"review_ms":review_ms,"save_ms":save_ms,"restore_review_ms":restore_review_ms,"restore_ms":restore_ms,"search":search_result,"cancellation_ms":cancellation_ms,"cancellation_polls":polls.get(),"restored_sha256":original_sha256}) + } + pub fn run() { + let runs: Vec = (0..RUNS).map(one).collect(); + let mut medians = serde_json::Map::new(); + for key in [ + "first_page_ms", + "prepare_ms", + "patch_ms", + "undo_ms", + "redo_ms", + "review_ms", + "save_ms", + "restore_review_ms", + "restore_ms", + "cancellation_ms", + ] { + let mut values: Vec<_> = runs.iter().map(|run| run[key].as_f64().unwrap()).collect(); + values.sort_by(f64::total_cmp); + medians.insert(key.into(), json!(values[RUNS / 2])); + } + println!("{}",serde_json::to_string_pretty(&json!({"schema":1,"fixture_bytes":Offset(SIZE),"runs":runs,"median_ms":medians,"scope":"Core engine only; excludes server, permissions, terminal rendering/input and provider inference. Fresh temporary files with warm OS executable/cache effects; no cold-cache claim."})).unwrap()); + } +} + +fn main() { + #[cfg(any(target_os = "macos", target_os = "linux"))] + native::run(); + #[cfg(not(any(target_os = "macos", target_os = "linux")))] + panic!("Native binary editing benchmark requires Linux or macOS"); +} diff --git a/docs/evidence/binary-editor-16mib/README.md b/docs/evidence/binary-editor-16mib/README.md new file mode 100644 index 000000000..4c72cc517 --- /dev/null +++ b/docs/evidence/binary-editor-16mib/README.md @@ -0,0 +1,64 @@ +# 16 MiB binary engine measurements + +Measured on 2026-11-03 on an Apple M2 with 24 GiB RAM, macOS arm64. Five serial +trials per profile create fresh synthetic 16 MiB files and private namespaces. +The named reference is `local-apple-m2-24gib`. Engine Rust sources match commit +`026533d4`; the new benchmark's exact source hash, executable hash, Cargo.lock hash, +platform and compiler are included in each JSON report. + +The benchmark streams fixture creation and byte IO. It stages a three-splice +batch (overwrite, insertion and deletion), checks undo/redo, reviews, saves and +restores through the actual native engine. Full proposed/restored SHA-256 and the +restored file length are verified in every trial. Timing excludes the additional +post-operation validation hashes. A literal search locates the unique final-byte +pattern using bounded continuation. Cancellation is requested on the third observed +poll; this measures cooperative core cancellation, not kernel IO preemption or +end-to-end Stop latency. + +| Operation, median milliseconds | Unoptimized dev | Optimized release | +| --- | ---: | ---: | +| Open + first 64 KiB page | 2.306 | 0.542 | +| Prepare immutable base | 998.119 | 140.748 | +| Stage overwrite/insert/delete batch | 6.083 | 6.873 | +| Draft undo | 7.045 | 7.021 | +| Draft redo | 7.916 | 6.962 | +| Byte review | 969.838 | 101.121 | +| Prepare + native save + receipt | 4877.749 | 552.926 | +| Restore review | 1447.474 | 149.698 | +| Prepare + native restore + receipt | 4863.623 | 545.187 | +| Cooperative search cancellation | 0.081 | 0.064 | + +- dev: median full search 2.744 ms; longest observed slice 2.302 ms; process peak RSS 8.30 MiB. +- release: median full search 3.115 ms; longest observed slice 7.037 ms; process peak RSS 7.45 MiB. + +Peak RSS comes from macOS `/usr/bin/time -l` around the benchmark executable +itself, including fixture generation and all five trials. Compilation/Cargo is +excluded. RSS excludes OS filesystem cache and other system memory. No memory +bound for the server or TUI is inferred from this measurement. + +These are core-only observations with uncontrolled background OS activity and +warm executable/filesystem cache effects. They are not a cold-cache claim, a p95 +latency guarantee, UI/permission overhead, first terminal viewport measurement, +server peak RSS, or Linux/Windows acceptance. No provider or inference is used. +No pass/fail performance threshold is invented from this machine's results. + +## Reproduction + +Build the standalone harness from the repository root: + +```sh +CARGO_INCREMENTAL=0 cargo build -p cyber-core --bench binary_editor --profile release --message-format=json > build.jsonl +``` + +Select the `compiler-artifact` record whose target name is `binary_editor` and use +its `executable` field. Run that exact executable directly (rather than timing +Cargo), for example on macOS: + +```sh +/usr/bin/time -l /path/from/build.jsonl > result.json 2> time.txt +``` + +Repeat with `--profile dev` and `CARGO_PROFILE_DEV_DEBUG=line-tables-only` for the +unoptimized comparison. Linux can use `/usr/bin/time -v`; its maximum RSS field is +in KiB rather than the macOS byte field. Windows native replacement is unsupported, +so the harness explicitly refuses to run there. diff --git a/docs/evidence/binary-editor-16mib/apple-m2-dev-time.txt b/docs/evidence/binary-editor-16mib/apple-m2-dev-time.txt new file mode 100644 index 000000000..403172d82 --- /dev/null +++ b/docs/evidence/binary-editor-16mib/apple-m2-dev-time.txt @@ -0,0 +1,18 @@ + 74.56 real 68.63 user 1.25 sys + 8699904 maximum resident set size + 0 average shared memory size + 0 average unshared data size + 0 average unshared stack size + 723 page reclaims + 0 page faults + 0 swaps + 0 block input operations + 0 block output operations + 0 messages sent + 0 messages received + 0 signals received + 984 voluntary context switches + 64371 involuntary context switches + 884779528449 instructions retired + 227233915260 cycles elapsed + 2769328 peak memory footprint diff --git a/docs/evidence/binary-editor-16mib/apple-m2-dev.json b/docs/evidence/binary-editor-16mib/apple-m2-dev.json new file mode 100644 index 000000000..083a986cc --- /dev/null +++ b/docs/evidence/binary-editor-16mib/apple-m2-dev.json @@ -0,0 +1,133 @@ +{ + "schema": 1, + "fixture_bytes": "16777216", + "runs": [ + { + "first_page_ms": 2.537583, + "prepare_ms": 1000.270833, + "patch_ms": 5.82275, + "undo_ms": 7.019958, + "redo_ms": 7.973292, + "review_ms": 965.975334, + "save_ms": 4904.886917, + "restore_review_ms": 1442.837584, + "restore_ms": 4978.800709, + "search": { + "total_ms": 3.9878750000000003, + "longest_slice_ms": 2.3022910000000003, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.096959, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 2.169459, + "prepare_ms": 1001.0186249999999, + "patch_ms": 5.877333999999999, + "undo_ms": 8.015167, + "redo_ms": 6.998584, + "review_ms": 969.5070000000001, + "save_ms": 4862.4958750000005, + "restore_review_ms": 1451.9885829999998, + "restore_ms": 4991.9024580000005, + "search": { + "total_ms": 3.264667, + "longest_slice_ms": 1.873041, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.079458, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 2.2022500000000003, + "prepare_ms": 998.118667, + "patch_ms": 7.9189169999999995, + "undo_ms": 9.032874999999999, + "redo_ms": 8.0095, + "review_ms": 976.922791, + "save_ms": 4877.748708, + "restore_review_ms": 1447.474417, + "restore_ms": 4863.623166, + "search": { + "total_ms": 2.4963339999999996, + "longest_slice_ms": 1.233542, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.073, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 2.572417, + "prepare_ms": 992.498583, + "patch_ms": 6.082667, + "undo_ms": 6.916208, + "redo_ms": 7.89, + "review_ms": 969.8384169999999, + "save_ms": 4860.362667, + "restore_review_ms": 1440.0701250000002, + "restore_ms": 4861.8974579999995, + "search": { + "total_ms": 2.744125, + "longest_slice_ms": 1.356167, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.081208, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 2.3060419999999997, + "prepare_ms": 994.357167, + "patch_ms": 6.97325, + "undo_ms": 7.044916, + "redo_ms": 7.915874999999999, + "review_ms": 972.768208, + "save_ms": 5025.832209, + "restore_review_ms": 1467.602791, + "restore_ms": 4862.539583, + "search": { + "total_ms": 2.663917, + "longest_slice_ms": 1.2745, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.107417, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + } + ], + "median_ms": { + "first_page_ms": 2.3060419999999997, + "prepare_ms": 998.118667, + "patch_ms": 6.082667, + "undo_ms": 7.044916, + "redo_ms": 7.915874999999999, + "review_ms": 969.8384169999999, + "save_ms": 4877.748708, + "restore_review_ms": 1447.474417, + "restore_ms": 4863.623166, + "cancellation_ms": 0.081208 + }, + "scope": "Core engine only; excludes server, permissions, terminal rendering/input and provider inference. Fresh temporary files with warm OS executable/cache effects; no cold-cache claim.", + "reference": "local-apple-m2-24gib", + "platform": "macOS-26.7-arm64-arm-64bit", + "cpu": "Apple M2", + "physical_ram_bytes": 25769803776, + "rustc": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "profile": "dev", + "engine_source_revision": "026533d49c19931e80229b20518a1aa8b8455fe4", + "benchmark_source_sha256": "cc44c262ca7c7b245f689f2dc66679a640c6b6948c60b7ef5d681418e39655de", + "executable_sha256": "2265c5e7a9c90fcc021df0a0be1fbc373986ce5775ccdf3d958f8dd3171765b1", + "cargo_lock_sha256": "907dfd81176064a8d26e9c20da6a948d4facb6bcb23dcb8b29b50e8f170f8e6f", + "measured_at_utc": "2026-11-03T10:08:16.411950+00:00", + "peak_process_rss_bytes": 8699904, + "median_search_total_ms": 2.744125, + "longest_search_slice_ms": 2.3022910000000003 +} diff --git a/docs/evidence/binary-editor-16mib/apple-m2-release-time.txt b/docs/evidence/binary-editor-16mib/apple-m2-release-time.txt new file mode 100644 index 000000000..7bd44d014 --- /dev/null +++ b/docs/evidence/binary-editor-16mib/apple-m2-release-time.txt @@ -0,0 +1,18 @@ + 9.00 real 6.91 user 0.62 sys + 7815168 maximum resident set size + 0 average shared memory size + 0 average unshared data size + 0 average unshared stack size + 665 page reclaims + 0 page faults + 0 swaps + 0 block input operations + 0 block output operations + 0 messages sent + 0 messages received + 0 signals received + 1045 voluntary context switches + 7409 involuntary context switches + 76906060002 instructions retired + 24333080716 cycles elapsed + 2818480 peak memory footprint diff --git a/docs/evidence/binary-editor-16mib/apple-m2-release.json b/docs/evidence/binary-editor-16mib/apple-m2-release.json new file mode 100644 index 000000000..8f77f968c --- /dev/null +++ b/docs/evidence/binary-editor-16mib/apple-m2-release.json @@ -0,0 +1,133 @@ +{ + "schema": 1, + "fixture_bytes": "16777216", + "runs": [ + { + "first_page_ms": 1.295958, + "prepare_ms": 156.121458, + "patch_ms": 6.137875, + "undo_ms": 6.8175, + "redo_ms": 6.007084, + "review_ms": 101.12075, + "save_ms": 552.925625, + "restore_review_ms": 150.57237500000002, + "restore_ms": 565.5014170000001, + "search": { + "total_ms": 10.799875, + "longest_slice_ms": 7.036833000000001, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.20775000000000002, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 0.5419160000000001, + "prepare_ms": 140.74758300000002, + "patch_ms": 7.093042, + "undo_ms": 7.759083, + "redo_ms": 8.867542, + "review_ms": 103.668666, + "save_ms": 573.240583, + "restore_review_ms": 150.154667, + "restore_ms": 545.1872079999999, + "search": { + "total_ms": 3.1150409999999997, + "longest_slice_ms": 1.586458, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.0785, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 0.48279099999999997, + "prepare_ms": 128.568291, + "patch_ms": 5.954875, + "undo_ms": 7.002416, + "redo_ms": 6.968959, + "review_ms": 99.800375, + "save_ms": 543.791875, + "restore_review_ms": 149.660125, + "restore_ms": 549.5097920000001, + "search": { + "total_ms": 2.944292, + "longest_slice_ms": 1.684708, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.063, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 0.501958, + "prepare_ms": 131.684291, + "patch_ms": 6.872667, + "undo_ms": 7.021125, + "redo_ms": 6.962000000000001, + "review_ms": 99.62941599999999, + "save_ms": 548.1025000000001, + "restore_review_ms": 149.69750000000002, + "restore_ms": 541.1866669999999, + "search": { + "total_ms": 2.864833, + "longest_slice_ms": 1.447167, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.06175, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + }, + { + "first_page_ms": 0.641333, + "prepare_ms": 142.73687500000003, + "patch_ms": 6.964292, + "undo_ms": 7.165875000000001, + "redo_ms": 6.8125, + "review_ms": 101.210041, + "save_ms": 554.5326670000001, + "restore_review_ms": 149.08029100000002, + "restore_ms": 544.492292, + "search": { + "total_ms": 3.511625, + "longest_slice_ms": 2.139125, + "slices": 3, + "match_offset": "16777209" + }, + "cancellation_ms": 0.064417, + "cancellation_polls": 3, + "restored_sha256": "3808005c4d1f0d8d401deb5b9b6177b40a313eb172d33fc959722cc283b95555" + } + ], + "median_ms": { + "first_page_ms": 0.5419160000000001, + "prepare_ms": 140.74758300000002, + "patch_ms": 6.872667, + "undo_ms": 7.021125, + "redo_ms": 6.962000000000001, + "review_ms": 101.12075, + "save_ms": 552.925625, + "restore_review_ms": 149.69750000000002, + "restore_ms": 545.1872079999999, + "cancellation_ms": 0.064417 + }, + "scope": "Core engine only; excludes server, permissions, terminal rendering/input and provider inference. Fresh temporary files with warm OS executable/cache effects; no cold-cache claim.", + "reference": "local-apple-m2-24gib", + "platform": "macOS-26.7-arm64-arm-64bit", + "cpu": "Apple M2", + "physical_ram_bytes": 25769803776, + "rustc": "rustc 1.99.1 (58cf8f9c3 2026-10-13)", + "profile": "release", + "engine_source_revision": "026533d49c19931e80229b20518a1aa8b8455fe4", + "benchmark_source_sha256": "cc44c262ca7c7b245f689f2dc66679a640c6b6948c60b7ef5d681418e39655de", + "executable_sha256": "fa3736710e9e8c76cf5b1be413d98d401825ada840ba649b6ec14f7c3fe9df92", + "cargo_lock_sha256": "907dfd81176064a8d26e9c20da6a948d4facb6bcb23dcb8b29b50e8f170f8e6f", + "measured_at_utc": "2026-11-03T10:08:16.460960+00:00", + "peak_process_rss_bytes": 7815168, + "median_search_total_ms": 3.1150409999999997, + "longest_search_slice_ms": 7.036833000000001 +} diff --git a/docs/evidence/binary-view/README.md b/docs/evidence/binary-view/README.md index b2848cf1d..5058c8517 100644 --- a/docs/evidence/binary-view/README.md +++ b/docs/evidence/binary-view/README.md @@ -15,3 +15,9 @@ server startup with an unoptimized line-tables build and warm OS executable cach RSS is one sample at first view, not peak memory. The temporary exploratory driver is not a portable benchmark harness. Editing, save/recovery, input/cancellation latency, the 16 MiB editing benchmark, and Linux/Windows native evidence remain open. + + +A later [16 MiB core-only editor measurement](../binary-editor-16mib/README.md) +covers prepare/edit/review/save/restore and cooperative search cancellation on the +Apple M2 reference. It does not turn this historical viewer capture into current +end-to-end terminal/editor or cross-platform performance acceptance. diff --git a/openspec/changes/add-binary-hex-editor/implementation.md b/openspec/changes/add-binary-hex-editor/implementation.md index 7e1c35382..14c2bbdf1 100644 --- a/openspec/changes/add-binary-hex-editor/implementation.md +++ b/openspec/changes/add-binary-hex-editor/implementation.md @@ -694,3 +694,18 @@ these recovery endpoints. Strict OpenSpec validation passed all 114 items. Sourc review checked namespace/resource binding, permission waits, private-buffer separation, cancellation and the existing filesystem-only recovery proof. Linux and Windows recovery acceptance remain for CI after the preceding PR merges. + + +### Named 16 MiB core performance evidence + +The reproducible `binary_editor` harness ran five serial 16 MiB trials on the +Apple M2/24 GiB reference, with full hash verification after native save and +restore. Both unoptimized and optimized results and process peak-RSS evidence are +in [the measurement record](../../../docs/evidence/binary-editor-16mib/README.md). +Optimized medians were 0.54 ms for open/first 64 KiB page, 141 ms for immutable +preparation, 101 ms for byte review, 553 ms for native save, 545 ms for native +restore and 0.064 ms for cooperative search cancellation. Optimized process peak +RSS was 7.45 MiB, excluding filesystem cache. These core-only observations exclude +server, permission, terminal rendering/input and provider work; they do not close +the broader UI/platform/performance acceptance task. The benchmark target passed +core all-target Clippy. diff --git a/openspec/changes/add-binary-hex-editor/tasks.md b/openspec/changes/add-binary-hex-editor/tasks.md index b54e2012f..0dbcfed6d 100644 --- a/openspec/changes/add-binary-hex-editor/tasks.md +++ b/openspec/changes/add-binary-hex-editor/tasks.md @@ -13,8 +13,8 @@ staged-edit/save/model/integration requirements below. ## 2. Byte service and native storage proof -- [ ] 2.1 Add typed byte-document/revision/offset/identity primitives with regular-file and project-root admission, exact integer transport and preserved permission ceilings. -- [ ] 2.2 Implement bounded pages, cancellable literal search and stable revision/continuation behavior without eager whole-file allocation or decoding. +- [x] 2.1 Add typed byte-document/revision/offset/identity primitives with regular-file and project-root admission, exact integer transport and preserved permission ceilings. +- [x] 2.2 Implement bounded pages, cancellable literal search and stable revision/continuation behavior without eager whole-file allocation or decoding. - [ ] 2.3 Prove native private immutable base preparation, same-filesystem atomic replacement with displaced preimage, metadata preservation and durable intent/outcome recovery; define unsupported outcomes honestly before write integration. - [ ] 2.4 Test native file/parent replacement, symlink/hardlink refusal, every byte value, NUL/invalid UTF-8, sparse/large files, offsets above 2^53 and resource exhaustion with positive controls. @@ -22,14 +22,14 @@ staged-edit/save/model/integration requirements below. - [x] 3.1 Add expected-revision overwrite/insert/delete splice batches, exact preimage checks, bounded payloads and atomic draft undo/redo. - [x] 3.2 Add stable byte-diff review with original offsets, base/proposed fingerprints, length/changed-range totals, metadata and effect disclosure. -- [ ] 3.3 Implement edit permission/Mode/checkout lease gates, capacity reservation and journaled save with operation idempotency and queryable acknowledged/conflict/unknown outcomes. -- [ ] 3.4 Preserve dirty drafts and unresolved receipts over disconnect/restart; add explicit discard and reviewed Restore Last Save without unguarded overwrite. +- [x] 3.3 Implement edit permission/Mode/checkout lease gates, capacity reservation and journaled save with operation idempotency and queryable acknowledged/conflict/unknown outcomes. +- [x] 3.4 Preserve dirty drafts and unresolved receipts over disconnect/restart; add explicit discard and reviewed Restore Last Save without unguarded overwrite. - [ ] 3.5 Test stale revision/review, same-size/same-mtime edits, all-or-nothing multi-splice changes, concurrent Sessions/external writers, cancel/kill cuts, retry identity and backup integrity. ## 4. Owner API, SDK and model tool -- [ ] 4.1 Implement authenticated `fs` document/page/search/patch/history/review/save/restore/operation/close routes with exact Session/Location scope and no raw-byte URLs/logging. -- [ ] 4.2 Generate OpenAPI/TypeScript types and shared client helpers with lossless offsets, bounded decoding, operation lookup and revision validation; preserve text content routes. +- [x] 4.1 Implement authenticated `fs` document/page/search/patch/history/review/save/restore/operation/close routes with exact Session/Location scope and no raw-byte URLs/logging. +- [x] 4.2 Generate OpenAPI/TypeScript types and shared client helpers with lossless offsets, bounded decoding, operation lookup and revision validation; preserve text content routes. - [x] 4.3 Register the capability-owned `binary` tool and implement bounded inspect/read/search/patch through the shared service, isolated from unsaved owner drafts. - [ ] 4.4 Enforce read+edit, plan/read-only/deny ceilings, existing output budgets, permission byte-diff review and plugin filesystem intersections; retain normal text-tool behavior. - [ ] 4.5 Test owner auth/Origin, cross-document/Session/Location access, stale generation, read-denied/write-allowed cases, malformed encodings, oversize payloads and secret-safe errors. @@ -38,13 +38,13 @@ staged-edit/save/model/integration requirements below. - [ ] 5.1 Build the virtualized themed hex/ASCII workspace, shared cursor/selection, adaptive rows, inspector and change rail with explicit view/edit/review/save/conflict states. - [ ] 5.2 Add goto, literal search, bookmarks, encoding-labelled text/hex input, partial-nibble cancellation, bounded copy/share, draft undo/redo and unsaved-change navigation. -- [ ] 5.3 Add explorer entry/palette/keybindings, `/hex` and `cyber hex` without inference, preserve Session selection and global interrupt/permission/native Stop priority; regenerate CLI documentation. +- [x] 5.3 Add explorer entry/palette/keybindings, `/hex` and `cyber hex` without inference, preserve Session selection and global interrupt/permission/native Stop priority; regenerate CLI documentation. - [ ] 5.4 Add wide/narrow/light/dark/NO_COLOR keyboard/mouse/accessibility snapshots and actual terminal captures; verify dirty-state, byte-diff totals and stale response fencing visually. ## 6. Snapshots, hooks and delivery - [ ] 6.1 Integrate durable binary preimages/receipts, byte diffs, conflict-aware restore and retention for Git/non-git Locations even when ordinary snapshots skip the file. -- [ ] 6.2 Publish only acknowledged successful file-change observations through the existing producer contract; preserve unknown effects and avoid duplicate notification after save retry. +- [x] 6.2 Publish only acknowledged successful file-change observations through the existing producer contract; preserve unknown effects and avoid duplicate notification after save retry. - [ ] 6.3 Benchmark first viewport, bounded memory, navigation/search responsiveness and cancellation on a named reference machine; publish fixture sizes and limitations. - [ ] 6.4 Run relevant native Rust/integration/real-binary/TUI tests, SDK typecheck/tests, generated artifact checks, strict OpenSpec/lint, formatting/Clippy and scoped platform CI. Record skipped/unperformed acceptance. - [ ] 6.5 Perform code/security review using the code-review skill; resolve findings, update README/feature/roadmap/operator guides with actual scope and deliver reviewable PRs. Do not claim format repair or arbitrary external-writer atomicity.