diff --git a/README.md b/README.md index cac63acd5..d61ae70e8 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,7 @@ For independent patch attempts, host-recorded tests and a separate read-only jud Experimental [native computer use](docs/computer-use.md) adds macOS 14+ application/display screenshots and input with expiring owner grants, CLI/browser owner controls, private evidence and emergency Stop. Off by default; controlled macOS capture, input and Stop acceptance passed. Other layouts and signed distribution remain release checks. -The [binary workspace](docs/binary-editor.md) provides paged hex/ASCII viewing, literal hex/UTF-8 search, exact goto, bookmarks, byte copying and an LE/BE inspector. Open it with `cyber hex `, `/hex `, or `x` on an explorer file, without starting inference. Session read permissions apply. On admitted Linux/macOS workspaces, F2 prepares private staged edits with undo/redo; Ctrl+S reviews the exact byte changes, then explicitly saves. Reviewed Restore Last Save and retained draft recovery are also available through the owner API/SDK. Authorized Sessions can review shared saved history, and Session diffs show saved binary references independently of Git. Model binary operations, checkpoint/retention/rewind adapters and remaining acceptance work are tracked in [#94](https://github.com/CyberdyneCorp/cybercode-cli/issues/94). +The [binary workspace](docs/binary-editor.md) provides paged hex/ASCII viewing, literal hex/UTF-8 search, exact goto, bookmarks, byte copying and an LE/BE inspector. Open it with `cyber hex `, `/hex `, or `x` on an explorer file, without starting inference. Session read permissions apply. On admitted Linux/macOS workspaces, F2 prepares private staged edits with undo/redo; Ctrl+S reviews the exact byte changes, then explicitly saves. Reviewed Restore Last Save and retained draft recovery are also available through the owner API/SDK. Authorized Sessions can review shared saved history, and Session diffs show saved binary references independently of Git. The model-facing `binary` tool supports bounded inspect/read/search and exact reviewed patches in isolated invocation drafts. Checkpoint/retention/rewind adapters and remaining acceptance work are tracked in [#94](https://github.com/CyberdyneCorp/cybercode-cli/issues/94). ![Binary hex workspace with a shared hex/ASCII cursor and little-endian byte inspector](docs/screenshots/binary-hex.png) diff --git a/ROADMAP.md b/ROADMAP.md index 2dd63a827..6904939b7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -97,7 +97,7 @@ The [current inventory](docs/implementation/spec-doc-reconciliation.md) contains **Usage limit waits (`session-runtime`, `configuration`, `exec-mode`, `tui`, `web-client`, P1; `goals`, `loops-scheduling`, `workflows`, P2): Status: implemented** (change [`add-usage-limit-wait`](openspec/changes/add-usage-limit-wait/proposal.md)). An unattended run (`cyber exec`, a Goal, a loop iteration, a workflow agent, a teammate, or their subagents) that hits a subscription usage limit with a known reset waits for it and retries the Turn instead of failing: within `usage_limit.max_wait` (default 6 h), 10 s after the reset plus per-Session jitter, as one new attempt from the same history. The wait is durable (`session.usage_limit.waiting|resumed|abandoned.1`), cancelled by the normal interrupt, resumed after a restart for top-level Sessions and ended cleanly for owned ones; wall deadlines (Session, run, Goal and loop `max_wall_seconds`, `exec --timeout`) win over waiting. `exec` reports `usage_limit` stream-json lines, the TUI offers `/resume-at-reset` and a footer badge, and the dashboard shows the wait. The model fallback chain, once implemented, runs first. -**Binary hex editor (P1), in progress:** [`add-binary-hex-editor`](openspec/changes/add-binary-hex-editor/proposal.md) now provides the [live binary workspace](docs/binary-editor.md): paged hex/ASCII inspection, literal search, exact goto, bookmarks, LE/BE interpretation and scoped owner API/SDK access. The user prioritized issue #94 on 2026-11-01 ahead of the unfinished plugins/hooks and daily-coding work. The native core and owner API/SDK now provide durable staged patches, draft undo/redo, byte review and explicit recovery on admitted Linux/macOS workspaces. The terminal now stages edits with draft undo/redo and explicit reviewed save/Restore Last Save through the same owner API. Acknowledged saves feed the native FileChanged producer. Authorized Sessions can review the shared saved-version catalog and Session diffs include binary references. Model operations, checkpoint/retention/rewind adapters and remaining acceptance are still open; this is not full editor acceptance. +**Binary hex editor (P1), in progress:** [`add-binary-hex-editor`](openspec/changes/add-binary-hex-editor/proposal.md) now provides the [live binary workspace](docs/binary-editor.md): paged hex/ASCII inspection, literal search, exact goto, bookmarks, LE/BE interpretation and scoped owner API/SDK access. The user prioritized issue #94 on 2026-11-01 ahead of the unfinished plugins/hooks and daily-coding work. The native core and owner API/SDK now provide durable staged patches, draft undo/redo, byte review and explicit recovery on admitted Linux/macOS workspaces. The terminal now stages edits with draft undo/redo and explicit reviewed save/Restore Last Save through the same owner API. Acknowledged saves feed the native FileChanged producer. Authorized Sessions can review the shared saved-version catalog and Session diffs include binary references. The model-facing `binary` tool supports bounded inspect/read/search and expected-digest patches through separate invocation drafts and the retained native save engine. Checkpoint/retention/rewind adapters and remaining acceptance are still open; this is not full editor acceptance. **Untrusted-content defense (`untrusted-content`, `permissions-modes`, `system-context`, P1): Status: implemented** (change [`add-untrusted-content-defense`](openspec/changes/add-untrusted-content-defense/proposal.md)). It carries the `workspace-trust` provenance rule through every tool that returns third-party content. `webfetch`, `websearch`, MCP tool results, browser pages and MCP resources are returned in one escape-safe `` frame that replaces the MCP-resource frame. Hidden characters become visible markers, and a deterministic detector flags instruction-like text with a notice in the TUI, `exec` and the dashboard, without dropping content. After a flagged result, risky actions (network egress, writes outside the Location, classifier-allowed commands, secret reads) ask until the next user prompt, or are denied with no user attached; `bypass` shows the notice or refuses when configured. The base prompts cover the frame (version 3), and a labeled fixture set gates recall and false positives, with scripted end-to-end scenarios for secret reads, permission widening and provider redirection. The TUI and dashboard state each notice's consequence from the Location's effective `taint`/`bypass` settings (task 5.3). What remains: acceptance by hand on the real binary. diff --git a/crates/cyber-app/src/host.rs b/crates/cyber-app/src/host.rs index 41c6a4757..df40627bb 100644 --- a/crates/cyber-app/src/host.rs +++ b/crates/cyber-app/src/host.rs @@ -257,6 +257,15 @@ impl ToolHost for AppHost { self.builtin.claim_location(info, creating, cancel) } + fn claim_binary_location<'a>( + &'a self, + info: &'a cyber_server::runtime::SessionInfo, + document: &'a str, + cancel: tokio_util::sync::CancellationToken, + ) -> BoxFuture<'a, Result> { + self.builtin.claim_binary_location(info, document, cancel) + } + fn prepare_child_continuation<'a>( &'a self, parent: &'a cyber_server::runtime::SessionInfo, diff --git a/crates/cyber-app/src/host/tests.rs b/crates/cyber-app/src/host/tests.rs index 5ea4c7bb2..55264e8d1 100644 --- a/crates/cyber-app/src/host/tests.rs +++ b/crates/cyber-app/src/host/tests.rs @@ -60,6 +60,35 @@ fn invocation(turn: &TurnContext) -> Invocation { } } +#[tokio::test] +async fn app_host_delegates_independent_binary_worker_checkout_ownership() { + let scratch = tempfile::tempdir().unwrap(); + let app = application(scratch.path()).await; + let host = AppHost { + builtin: app.host.clone(), + remote: app.state.remote_tools.clone(), + peers: Arc::new(|_| Ok(None)), + peer_names: Arc::new(|| Ok(Vec::new())), + }; + let info = app + .runtime + .create_session(cyber_server::runtime::CreateSession { + directory: scratch.path().canonicalize().unwrap().display().to_string(), + model: "test/main".into(), + ..Default::default() + }) + .await + .unwrap(); + let lease = host + .claim_binary_location(&info, "bin_retained_worker", CancellationToken::new()) + .await + .unwrap(); + assert_eq!(lease.worktree_id, info.worktree_id); + let retained = lease.settle_retained().unwrap(); + drop(retained); + app.runtime.shutdown().await; +} + #[tokio::test] async fn agent_tool_restrictions_refuse_client_dispatch_without_sending_a_request() { let scratch = tempfile::tempdir().unwrap(); diff --git a/crates/cyber-core/src/binary/editor.rs b/crates/cyber-core/src/binary/editor.rs index 13f58714e..3964e12b0 100644 --- a/crates/cyber-core/src/binary/editor.rs +++ b/crates/cyber-core/src/binary/editor.rs @@ -194,6 +194,33 @@ fn stage_file(store: &Store, name: &str) -> Result { Ok(file) } impl Editor { + /// Resume only the exact invocation/payload owner, authenticated by its small + /// header before opening any private command journal or base image. + pub fn model( + storage: &super::storage::StorageRoot, + id: &str, + owner: Owner, + document: &mut Document, + cancelled: impl Fn() -> bool, + ) -> Result { + if !matches!(owner.scope, DraftScope::Model { .. }) { + return Err(Error::Invalid); + } + let store = storage.open(id)?; + match store.directory().symlink_metadata("document.json") { + Ok(_) => { + if storage.resource(&owner, id)?.path() != document.resource().path() { + return Err(Error::Changed); + } + Self::resume(store, &owner) + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + Self::create(store, owner, document, cancelled) + } + Err(_) => Err(Error::Io), + } + } + pub fn create( mut store: Store, owner: Owner, @@ -1288,6 +1315,52 @@ mod tests { .unwrap(); } + #[test] + fn model_resume_binds_payload_and_never_opens_the_manual_draft() { + let (root, mut manual, path, mut owner) = setup(); + patch(&mut manual, "private_manual_patch"); + let private_revision = manual.draft().revision(); + let storage = + super::super::storage::StorageRoot::prepare(&root.path().join("data")).unwrap(); + owner.scope = DraftScope::Model { + operation: "ses_one:call_one:payload_one".into(), + }; + let mut document = Document::open(&owner.root, "bytes").unwrap(); + assert!( + Editor::model(&storage, "bin_one", owner.clone(), &mut document, || false).is_err() + ); + let mut model = Editor::model(&storage, "bin_model", owner.clone(), &mut document, || { + false + }) + .unwrap(); + assert_eq!( + model + .draft() + .page(&model.draft().revision(), Offset(0), 8) + .unwrap() + .bytes, + b"original" + ); + patch(&mut model, "patch"); + let prepared = ready(&mut model, "save"); + let receipt = model.commit_save(*prepared).unwrap(); + drop(model); + let mut source = Document::open(&owner.root, "bytes").unwrap(); + let resumed = + Editor::model(&storage, "bin_model", owner.clone(), &mut source, || false).unwrap(); + assert_eq!( + resumed.lookup_operation("save").unwrap().unwrap().revision, + receipt.revision + ); + drop(resumed); + owner.scope = DraftScope::Model { + operation: "ses_one:call_one:changed_payload".into(), + }; + assert!(Editor::model(&storage, "bin_model", owner, &mut source, || false).is_err()); + assert_eq!(manual.draft().revision(), private_revision); + assert_eq!(std::fs::read(path).unwrap(), [0, 255, 128, 27]); + } + #[test] fn equivalent_patch_encodings_reuse_the_durable_receipt() { let (_root, mut editor, _path, _owner) = setup(); diff --git a/crates/cyber-core/src/binary/source.rs b/crates/cyber-core/src/binary/source.rs index e7994f8c0..9736e10d2 100644 --- a/crates/cyber-core/src/binary/source.rs +++ b/crates/cyber-core/src/binary/source.rs @@ -229,6 +229,34 @@ impl Document { Ok(()) } + /// Whole-file evidence for an editable-sized live source. This does not + /// prepare a draft or claim an immutable revision. Bytes stream in 64 KiB. + pub fn fingerprint(&mut self, cancelled: impl Fn() -> bool) -> Result { + if self.length > MAX_EDIT_SIZE { + return Err(Error::Limit); + } + self.validate()?; + let mut hash = Sha256::new(); + let mut at = 0; + while at < self.length { + if cancelled() { + return Err(Error::Cancelled); + } + let count = (self.length - at).min(DEFAULT_PAGE as u64) as usize; + let bytes = self.read_at(at, count)?; + if bytes.len() != count { + return Err(Error::Changed); + } + hash.update(bytes); + at += count as u64; + } + if cancelled() { + return Err(Error::Cancelled); + } + self.validate()?; + Ok(format!("{:x}", hash.finalize())) + } + fn check_revision(&self, revision: &str) -> Result<(), Error> { if revision != self.observed { return Err(Error::Changed); diff --git a/crates/cyber-core/src/binary/tests.rs b/crates/cyber-core/src/binary/tests.rs index fb4f7486f..231e07178 100644 --- a/crates/cyber-core/src/binary/tests.rs +++ b/crates/cyber-core/src/binary/tests.rs @@ -293,3 +293,17 @@ fn a_zero_length_virtual_file_cannot_return_bytes_beyond_its_observed_range() { assert!(page.bytes.is_empty()); assert!(page.eof); } + +#[test] +fn live_fingerprints_stream_exact_bytes_and_refuse_cancellation_or_changed_sources() { + use sha2::{Digest, Sha256}; + let bytes: Vec = (0..=255).cycle().take(128 * 1024 + 17).collect(); + let (temp, mut source) = fixture(&bytes); + assert_eq!( + source.fingerprint(|| false).unwrap(), + format!("{:x}", Sha256::digest(&bytes)) + ); + assert!(matches!(source.fingerprint(|| true), Err(Error::Cancelled))); + std::fs::write(temp.path().join("bytes"), b"changed").unwrap(); + assert!(matches!(source.fingerprint(|| false), Err(Error::Changed))); +} diff --git a/crates/cyber-core/src/worktrees/activity.rs b/crates/cyber-core/src/worktrees/activity.rs index 53966dcea..117c101e5 100644 --- a/crates/cyber-core/src/worktrees/activity.rs +++ b/crates/cyber-core/src/worktrees/activity.rs @@ -145,6 +145,18 @@ impl Repository { self.claim_owner(execution, managed, server_id).await } + /// A retained filesystem-only binary worker owns a lease independently of + /// the Session actor, whose lease may already be held by the active turn. + pub async fn claim_binary( + &self, + execution: &dyn GitExecution, + managed: &Managed, + document: &str, + ) -> io::Result { + safe_id(document, "bin")?; + self.claim_owner(execution, managed, document).await + } + async fn claim_owner( &self, execution: &dyn GitExecution, @@ -215,6 +227,8 @@ fn safe_owner_id(id: &str) -> io::Result<()> { "mcs" } else if crate::ids::has_prefix(id, "lsp") { "lsp" + } else if crate::ids::has_prefix(id, "bin") { + "bin" } else { "ses" }; diff --git a/crates/cyber-core/tests/worktree_repository/activity.rs b/crates/cyber-core/tests/worktree_repository/activity.rs index 0c183c293..195f14384 100644 --- a/crates/cyber-core/tests/worktree_repository/activity.rs +++ b/crates/cyber-core/tests/worktree_repository/activity.rs @@ -1,6 +1,31 @@ use super::*; use cyber_core::worktrees::CheckoutActivity; +#[test] +fn binary_worker_retains_an_independent_checkout_lease_through_source_commit() { + let fixture = Fixture::new(); + let repository = fixture.repository(); + let managed = fixture + .create( + &repository, + &Name::parse("binary-worker").unwrap(), + &Settings::default(), + ) + .unwrap(); + let actor = block_on(repository.claim(&fixture.execution, &managed, "ses_actor")).unwrap(); + assert!(block_on(repository.claim_binary(&fixture.execution, &managed, "ses_actor")).is_err()); + let worker = + block_on(repository.claim_binary(&fixture.execution, &managed, "bin_worker")).unwrap(); + actor.settle().unwrap(); + let retained = worker.settle_retained().unwrap(); + assert!( + block_on(repository.remove(&fixture.execution, &CheckoutActivity, &managed, false)) + .is_err() + ); + drop(retained); + block_on(repository.remove(&fixture.execution, &CheckoutActivity, &managed, false)).unwrap(); +} + #[test] fn lsp_owner_remains_independent_and_excludes_removal_through_retained_settlement() { let fixture = Fixture::new(); diff --git a/crates/cyber-server/src/runtime/binary_documents.rs b/crates/cyber-server/src/runtime/binary_documents.rs index dca7cbe6e..f302c8f13 100644 --- a/crates/cyber-server/src/runtime/binary_documents.rs +++ b/crates/cyber-server/src/runtime/binary_documents.rs @@ -67,7 +67,7 @@ pub(crate) struct EditAdmission { /// Retained host authority for the actual filesystem worker. Verification only /// refreshes trusted policy; prompts and hook/model execution belong before it. #[cfg(any(target_os = "macos", target_os = "linux"))] -pub(crate) struct BinaryWork { +pub struct BinaryWork { runtime: super::Runtime, executor: tokio::runtime::Handle, turn: super::TurnContext, @@ -76,6 +76,7 @@ pub(crate) struct BinaryWork { authority: super::AdmissionAuthority, cancel: tokio_util::sync::CancellationToken, post: Arc>>>, + invocation: Option, } #[cfg(any(target_os = "macos", target_os = "linux"))] impl BinaryWork { @@ -157,6 +158,9 @@ impl BinaryWork { .verify(&self.runtime, &self.turn.session_id) .map_err(|_| Error::Changed)?; self.executor.block_on(async { + if let Some(invocation) = &self.invocation { + self.runtime.verify_binary_invocation(invocation).await?; + } let current = self .runtime .file_turn( @@ -353,6 +357,73 @@ impl Registry { } impl super::Runtime { + /// Trusted host storage; callers must separately authenticate the document + /// owner and retain source read/edit authority. + pub fn binary_storage(&self) -> Result, Error> { + self.binary_documents().storage() + } + + /// Bind source disclosure/effects to the exact currently dispatched built-in + /// call. Stored IDs and an old invocation are never sufficient authority. + pub async fn verify_binary_invocation(&self, inv: &super::Invocation) -> Result<(), Error> { + let handle = self + .inner + .handle(&inv.session_id) + .await + .map_err(|_| Error::Changed)?; + let state = handle.state.lock().await; + let info = &state.info; + if inv.registration.is_some() + || inv.name != "binary" + || inv.operation_key != format!("{}:{}", inv.session_id, inv.call_id) + || info.directory != inv.directory + || info.agent != inv.agent + || info.mode != inv.mode + || info.rules != inv.rules + || !state.calls.get(&inv.call_id).is_some_and(|call| { + call.status == super::CallStatus::Dispatched + && call.name == inv.name + && call.message_id == inv.message_id + && call.attempt == inv.attempt + && call.input.as_ref() == Some(&inv.input) + }) + { + return Err(Error::Changed); + } + Ok(()) + } + + /// Retained filesystem worker for an active built-in binary tool call. + #[cfg(any(target_os = "macos", target_os = "linux"))] + pub async fn binary_tool_work( + &self, + invocation: super::Invocation, + document: String, + resource: cyber_core::binary::Resource, + grants: (super::FileDecision, super::FileDecision), + work: impl FnOnce(BinaryWork) -> T + Send + 'static, + ) -> Result { + self.verify_binary_invocation(&invocation) + .await + .map_err(|error| super::RuntimeError::Invalid(error.to_string()))?; + let turn = self + .file_turn( + Some(&invocation.session_id), + std::path::Path::new(&invocation.directory), + ) + .await?; + self.binary_native_work(turn, document, resource, grants, Some(invocation), work) + .await + } + + /// Capacity only: callers still establish source read authority separately. + pub fn binary_read_permit( + &self, + ) -> Result { + self.binary_documents() + .worker() + .map_err(|e| super::RuntimeError::Invalid(e.to_string())) + } /// Install only after bootstrap has prepared and hidden the existing root from /// all child launches. An unconfigured runtime keeps binary editing disabled. pub fn configure_binary_storage( @@ -382,13 +453,29 @@ impl super::Runtime { resource: cyber_core::binary::Resource, grants: (super::FileDecision, super::FileDecision), work: impl FnOnce(BinaryWork) -> T + Send + 'static, + ) -> Result { + self.binary_native_work(turn, document, resource, grants, None, work) + .await + } + + #[cfg(any(target_os = "macos", target_os = "linux"))] + async fn binary_native_work( + &self, + turn: super::TurnContext, + document: String, + resource: cyber_core::binary::Resource, + grants: (super::FileDecision, super::FileDecision), + invocation: Option, + work: impl FnOnce(BinaryWork) -> T + Send + 'static, ) -> Result { let edit_permit = self .binary_documents() .edit_worker() .map_err(|error| super::RuntimeError::Invalid(error.to_string()))?; let lifecycle = self.memory_mutation_lease().await?; - self.inner.ensure_idle(&turn.session_id)?; + if invocation.is_none() { + self.inner.ensure_idle(&turn.session_id)?; + } let handle = self.inner.handle(&turn.session_id).await?; let info = handle.state.lock().await.info.clone(); if self @@ -405,23 +492,12 @@ impl super::Runtime { } let authority = self.capture_child_admission(&turn.session_id)?; let mut activity = - super::activity::Scope::reserve_binary(&self.inner, &handle, document).await?; + super::activity::Scope::reserve_binary(&self.inner, &handle, document.clone()).await?; let cancel = activity.control.stop.clone(); let _cancel_on_disposal = cancel.clone().drop_guard(); let location = self - .inner - .claim_location(&info, false, cancel.clone()) + .binary_worker_location(&info, &document, cancel.clone(), invocation.is_some()) .await?; - if location.worktree_id != info.worktree_id { - return Err(super::RuntimeError::Conflict( - "Binary checkout binding changed".into(), - )); - } - // This scope owns no subprocess. The retained proof keeps the native - // checkout lock through source mutation and journal settlement, including crash. - let location = location - .settle_retained() - .map_err(super::RuntimeError::Invalid)?; let paths = tokio::select! { result = self.inner.tools.file_mutation_guard(&turn, &resource) => result.map_err(super::RuntimeError::Invalid)?, _ = cancel.cancelled() => return Err(super::RuntimeError::Invalid("Binary operation cancelled before mutation".into())), @@ -455,6 +531,7 @@ impl super::Runtime { authority, cancel, post: post.clone(), + invocation, }; tokio::task::spawn_blocking(move || { let (_lifecycle, _location, _paths, _permit, _edit_permit) = @@ -494,6 +571,34 @@ impl super::Runtime { })? } #[cfg(any(target_os = "macos", target_os = "linux"))] + async fn binary_worker_location( + &self, + info: &super::SessionInfo, + document: &str, + cancel: tokio_util::sync::CancellationToken, + active_call: bool, + ) -> Result, super::RuntimeError> { + let location = if active_call { + self.inner + .tools + .claim_binary_location(info, document, cancel) + .await + .map_err(super::RuntimeError::Invalid)? + } else { + self.inner.claim_location(info, false, cancel).await? + }; + if location.worktree_id != info.worktree_id { + return Err(super::RuntimeError::Conflict( + "Binary checkout binding changed".into(), + )); + } + // Only this filesystem-only worker is acknowledged, never the actor's + // generic process activity. Keep its actual lock until commit settles. + location + .settle_retained() + .map_err(super::RuntimeError::Invalid) + } + #[cfg(any(target_os = "macos", target_os = "linux"))] async fn verify_binary_grants( &self, turn: &super::TurnContext, diff --git a/crates/cyber-server/src/runtime/host.rs b/crates/cyber-server/src/runtime/host.rs index 248980602..cd576562d 100644 --- a/crates/cyber-server/src/runtime/host.rs +++ b/crates/cyber-server/src/runtime/host.rs @@ -837,6 +837,17 @@ pub trait ToolHost: Send + Sync { Box::pin(async { Ok(LocationLease::unmanaged()) }) } + /// Independently retain a filesystem-only binary worker's checkout. Hosts + /// must explicitly support this; the active actor's lease cannot be reused. + fn claim_binary_location<'a>( + &'a self, + _info: &'a SessionInfo, + _document: &'a str, + _cancel: CancellationToken, + ) -> BoxFuture<'a, Result> { + Box::pin(async { Err("Binary worker checkout ownership is unsupported".into()) }) + } + /// Prepare an existing child under its exclusive owner. Opening a client view grants no bypass. fn prepare_child_continuation<'a>( &'a self, diff --git a/crates/cyber-server/src/runtime/mod.rs b/crates/cyber-server/src/runtime/mod.rs index cdc0a241e..4f9b71e9c 100644 --- a/crates/cyber-server/src/runtime/mod.rs +++ b/crates/cyber-server/src/runtime/mod.rs @@ -52,6 +52,8 @@ mod hooks; mod installation; pub use installation::{InstallationNotice, UpdateStatus}; pub(crate) mod binary_documents; +#[cfg(any(target_os = "macos", target_os = "linux"))] +pub use binary_documents::BinaryWork; pub mod binary_effects; #[cfg(any(target_os = "macos", target_os = "linux"))] mod binary_recovery; diff --git a/crates/cyber-tools/src/host.rs b/crates/cyber-tools/src/host.rs index 232a8784a..7af75d7c2 100644 --- a/crates/cyber-tools/src/host.rs +++ b/crates/cyber-tools/src/host.rs @@ -1333,6 +1333,15 @@ impl ToolHost for BuiltinHost { Box::pin(self.claim_worktree_location(info, creating, cancel)) } + fn claim_binary_location<'a>( + &'a self, + info: &'a cyber_server::runtime::SessionInfo, + document: &'a str, + cancel: CancellationToken, + ) -> BoxFuture<'a, Result> { + Box::pin(self.claim_binary_worktree_location(info, document, cancel)) + } + fn prepare_child_continuation<'a>( &'a self, parent: &'a cyber_server::runtime::SessionInfo, @@ -1858,7 +1867,7 @@ fn offered( let by_mode = match (mode, name) { (Mode::Plan, "write" | "plan_exit" | "todo" | "skill" | "notify") => true, (Mode::Plan, "plan_enter") => false, - (Mode::Plan, "computer") => true, + (Mode::Plan, "computer" | "binary") => true, (Mode::Plan, _) => safety == cyber_server::runtime::RetrySafety::ReadOnly, (_, "plan_exit") => false, _ => true, @@ -2233,6 +2242,45 @@ impl Ctx<'_> { } } + pub(crate) async fn verify_binary_read(&self, path: &Path) -> Result<(), ToolError> { + if self.cancel.is_cancelled() { + return Err(ToolError::Aborted); + } + let profile = self + .host + .agent_profile(&self.location, &self.inv.agent) + .map_err(tools::failed)?; + if !profile_allows_tool(&profile, "binary") { + return Err(tools::failed( + "Binary tool is no longer allowed for this agent", + )); + } + if crate::sandboxing::hidden(self).denies(path) { + return Err(tools::failed("Binary source is no longer readable")); + } + let request = Request { + tool: Some(self.inv.name.clone()), + action: "read".into(), + resources: vec![self.resource(path)], + read_only: true, + ..Default::default() + }; + let fresh = self + .host + .policy(self.inv) + .await + .map_err(tools::failed)? + .decide(&request); + if matches!(fresh, Decision::Deny(_)) + || (fresh == Decision::Ask && self.policy.decide(&request) != Decision::Ask) + { + return Err(tools::failed( + "Binary read authority changed; repeat the operation", + )); + } + Ok(()) + } + pub async fn authorize( &self, req: Request, diff --git a/crates/cyber-tools/src/tools/binary.rs b/crates/cyber-tools/src/tools/binary.rs new file mode 100644 index 000000000..a67e466a4 --- /dev/null +++ b/crates/cyber-tools/src/tools/binary.rs @@ -0,0 +1,258 @@ +//! Bounded live byte operations. Owner drafts and private version images are +//! deliberately absent from this tool's input namespace. +use super::{Tool, ToolError, def, failed}; +use crate::{host::Ctx, permissions::Request}; +use cyber_core::binary::{Document, Offset, SearchRequest}; +use cyber_server::runtime::{RetrySafety, ToolDef}; +use futures::future::BoxFuture; +use serde::Deserialize; +use serde_json::{Value, json}; +use std::sync::{Arc, OnceLock}; +#[cfg(any(target_os = "macos", target_os = "linux"))] +mod patch; + +pub(crate) struct Binary; +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct Input { + operation: Operation, + path: String, + offset: Option, + limit: Option, + revision: Option, + hex: Option, + expected_base_sha256: Option, + splices: Option>, +} +#[derive(Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +enum Operation { + Inspect, + Read, + Search, + Patch, +} + +impl Tool for Binary { + fn def(&self) -> ToolDef { + def( + "binary", + "Inspect regular-file bytes, read bounded hex/ASCII pages, search literal hex, or apply an exact reviewed patch. Patch requires the inspected whole-file SHA-256 and expected bytes for every splice. Offsets are exact decimal strings. Inspect computes SHA-256 through 512 MiB; larger sources remain live view only. Patch never automatically retries a native replacement.", + json!({"type":"object","additionalProperties":false,"required":["operation","path"],"properties":{ + "operation":{"type":"string","enum":["inspect","read","search","patch"]},"path":{"type":"string"}, + "offset":{"type":"string","pattern":"^(0|[1-9][0-9]{0,19})$"},"limit":{"type":"integer","minimum":1,"maximum":4096}, + "revision":{"type":"string"},"hex":{"type":"string","maxLength":8192}, + "expected_base_sha256":{"type":"string","pattern":"^[0-9a-f]{64}$"}, + "splices":{"type":"array","minItems":1,"maxItems":128,"items":{"type":"object","additionalProperties":false,"required":["offset","delete_length","expected_hex","insert_hex"],"properties":{ + "offset":{"type":"string","pattern":"^(0|[1-9][0-9]{0,19})$"},"delete_length":{"type":"string","pattern":"^(0|[1-9][0-9]{0,19})$"},"expected_hex":{"type":"string","maxLength":131072},"insert_hex":{"type":"string","maxLength":131072}}}}}}), + RetrySafety::Never, + false, + ) + } + fn run<'a>(&'a self, ctx: &'a Ctx<'a>) -> BoxFuture<'a, Result> { + Box::pin(run(ctx)) + } +} +fn permit(ctx: &Ctx<'_>) -> Result { + if let Some(runtime) = ctx.host.runtime() { + return runtime + .binary_read_permit() + .map_err(|_| failed("Binary IO capacity reached")); + } + static WORKERS: OnceLock> = OnceLock::new(); + WORKERS + .get_or_init(|| Arc::new(tokio::sync::Semaphore::new(16))) + .clone() + .try_acquire_owned() + .map_err(|_| failed("Binary IO capacity reached")) +} +async fn run(ctx: &Ctx<'_>) -> Result { + if ctx.inv.registration.is_some() { + return Err(failed("Binary operations require the built-in capability")); + } + let input: Input = serde_json::from_value(ctx.inv.input.clone()) + .map_err(|_| failed("Invalid binary operation or byte arguments"))?; + if input.operation != Operation::Patch + && (input.expected_base_sha256.is_some() || input.splices.is_some()) + { + return Err(failed("Patch arguments require the patch operation")); + } + let authority = ctx + .host + .runtime() + .map(|runtime| { + runtime + .capture_child_admission(&ctx.inv.session_id) + .map(|authority| (runtime, authority)) + }) + .transpose() + .map_err(|_| failed("Binary invocation admission changed"))?; + if let Some((runtime, _)) = &authority { + runtime + .verify_binary_invocation(ctx.inv) + .await + .map_err(error)?; + } + let permit = permit(ctx)?; + let (directory, path) = (ctx.location.clone(), input.path.clone()); + let (document, permit) = tokio::task::spawn_blocking(move || { + let root = cyber_core::config::project_root(&directory) + .canonicalize() + .map_err(|_| failed("Binary project root unavailable"))?; + let canonical_directory = directory + .canonicalize() + .map_err(|_| failed("Binary Location unavailable"))?; + let requested = if std::path::Path::new(&path).is_absolute() { + let absolute = std::path::PathBuf::from(&path); + absolute + .strip_prefix(&directory) + .map_or_else(|_| absolute.clone(), |tail| canonical_directory.join(tail)) + } else { + canonical_directory.join(&path) + }; + let relative = requested + .strip_prefix(&root) + .map_err(|_| failed("Binary source is outside the project"))?; + let relative = relative + .to_str() + .ok_or_else(|| failed("Binary source requires a UTF-8 path"))?; + let document = Document::open(&root, relative).map_err(error)?; + Ok::<_, ToolError>((document, permit)) + }) + .await + .map_err(|_| failed("Binary source worker interrupted"))??; + let path = document.path(); + ctx.check_external(std::slice::from_ref(&path)).await?; + if crate::sandboxing::hidden(ctx).denies(&path) { + return Err(failed("Binary source is private")); + } + let resource = ctx.resource(&path); + ctx.authorize( + Request { + action: "read".into(), + resources: vec![resource.clone()], + read_only: true, + ..Default::default() + }, + vec![resource], + Value::Null, + ) + .await?; + ctx.verify_binary_read(&path).await?; + let result = if input.operation == Operation::Patch { + #[cfg(any(target_os = "macos", target_os = "linux"))] + { + patch::run(ctx, document, input, permit).await? + } + #[cfg(not(any(target_os = "macos", target_os = "linux")))] + { + let _ = (document, input, permit); + return Err(failed( + "Native binary patching is unsupported on this platform", + )); + } + } else { + let cancel = ctx.cancel.clone(); + tokio::task::spawn_blocking(move || { + let _permit = permit; + execute(document, input, || cancel.is_cancelled()) + }) + .await + .map_err(|_| failed("Binary source worker interrupted"))?? + }; + if let Some((runtime, authority)) = authority { + runtime + .verify_binary_invocation(ctx.inv) + .await + .map_err(error)?; + authority + .verify(&runtime, &ctx.inv.session_id) + .map_err(|_| failed("Binary invocation admission changed"))?; + } + ctx.verify_binary_read(&path).await?; + serde_json::to_string(&result).map_err(|_| failed("Binary result encoding failed")) +} +fn execute( + mut document: Document, + input: Input, + cancelled: impl Fn() -> bool, +) -> Result { + document.validate().map_err(error)?; + let metadata = document.metadata(); + if input + .revision + .as_ref() + .is_some_and(|revision| revision != &metadata.revision) + { + return Err(failed("Binary live revision changed; inspect again")); + } + if cancelled() { + return Err(ToolError::Aborted); + } + let data = match input.operation { + Operation::Inspect => { + let hash = if metadata.within_edit_size_limit { + Some(document.fingerprint(&cancelled).map_err(error)?) + } else { + None + }; + json!({"metadata":metadata,"base_sha256":hash}) + } + Operation::Read => { + let limit = input.limit.unwrap_or(256); + if limit == 0 || limit > 4096 { + return Err(failed("Binary read limit must be 1–4096 bytes")); + } + let page = document + .page(&metadata.revision, input.offset.unwrap_or(Offset(0)), limit) + .map_err(error)?; + let hex = page + .bytes + .iter() + .map(|byte| format!("{byte:02x}")) + .collect::(); + let ascii = page + .bytes + .iter() + .map(|byte| { + if (32..=126).contains(byte) { + char::from(*byte) + } else { + '.' + } + }) + .collect::(); + json!({"metadata":metadata,"offset":page.offset,"length":Offset(page.bytes.len() as u64),"hex":hex,"ascii":ascii,"page_sha256":page.sha256,"eof":page.eof}) + } + Operation::Search => { + let hex = input + .hex + .ok_or_else(|| failed("Binary search requires literal hex"))?; + let result = document + .search( + &SearchRequest { + revision: metadata.revision.clone(), + hex, + offset: input.offset.unwrap_or(Offset(0)), + }, + &cancelled, + ) + .map_err(error)?; + json!({"metadata":metadata,"result":result}) + } + Operation::Patch => return Err(failed("Binary patch requires retained native ownership")), + }; + if cancelled() { + return Err(ToolError::Aborted); + } + document.validate().map_err(error)?; + Ok(data) +} + +fn error(error: cyber_core::binary::Error) -> ToolError { + if matches!(error, cyber_core::binary::Error::Cancelled) { + ToolError::Aborted + } else { + failed(error.to_string()) + } +} diff --git a/crates/cyber-tools/src/tools/binary/patch.rs b/crates/cyber-tools/src/tools/binary/patch.rs new file mode 100644 index 000000000..177fd0da3 --- /dev/null +++ b/crates/cyber-tools/src/tools/binary/patch.rs @@ -0,0 +1,243 @@ +//! Invocation-owned staging. Only the retained native worker installs bytes. +use super::{Input, error, failed}; +use crate::{ + host::Ctx, + permissions::{Decision, Mode, Request}, + tools::ToolError, +}; +use cyber_core::binary::{ + Document, Error, + draft::{self, ImmutableBase, PatchBatch}, + editor::{DraftScope, Editor, Owner, SavePreparation}, + receipts::{Operation, SaveReview}, +}; +use cyber_server::runtime::FileDecision; +use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; + +enum Staged { + Existing(Operation), + Review(Box, Box), +} + +pub(super) async fn run( + ctx: &Ctx<'_>, + mut document: Document, + input: Input, + permit: tokio::sync::OwnedSemaphorePermit, +) -> Result { + if Mode::parse(&ctx.inv.mode) == Mode::Plan || ctx.inv.registration.is_some() { + return Err(failed( + "Binary patch requires a writable built-in invocation", + )); + } + if input.offset.is_some() + || input.limit.is_some() + || input.hex.is_some() + || input.revision.is_some() + { + return Err(failed( + "Binary patch accepts only expected_base_sha256 and splices", + )); + } + let expected = input + .expected_base_sha256 + .ok_or_else(|| failed("Binary patch requires expected_base_sha256"))?; + if expected.len() != 64 + || !expected + .bytes() + .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) + { + return Err(failed("Binary patch requires a lowercase SHA-256")); + } + let mut batch = draft::canonical(&PatchBatch { + revision: String::new(), + splices: input + .splices + .ok_or_else(|| failed("Binary patch requires exact splices"))?, + }) + .map_err(error)?; + let path = document.path(); + let resource = document.resource(); + crate::sandboxing::binary_edit_boundary(ctx, &path).map_err(failed)?; + let request = Request { + tool: Some("binary".into()), + action: "edit".into(), + resources: vec![ctx.resource(&path)], + file_edit: true, + mutates: vec![path.clone()], + ..Default::default() + }; + ctx.refuse_denied(request.clone()).await?; + // Refuse Mode, inherited and protected-path ceilings before copying a base; + // Ask is deferred until the exact byte review has been prepared. + decision(ctx.policy.decide(&request))?; + let runtime = ctx + .host + .runtime() + .ok_or_else(|| failed("Binary patch requires the retained runtime"))?; + runtime + .verify_binary_invocation(ctx.inv) + .await + .map_err(error)?; + let storage = runtime.binary_storage().map_err(error)?; + let key = format!("{:x}", Sha256::digest(ctx.inv.operation_key.as_bytes())); + let document_id = format!("bin_{key}"); + let operation_id = format!("save_{key}"); + let payload = serde_json::to_vec(&(&ctx.inv.operation_key, &expected, &batch, &path)) + .map_err(|_| failed("Binary payload encoding failed"))?; + let owner = Owner { + session_id: ctx.inv.session_id.clone(), + directory: ctx + .location + .canonicalize() + .map_err(|_| failed("Binary Location unavailable"))?, + root: document.root().to_owned(), + scope: DraftScope::Model { + operation: format!("{}:{:x}", ctx.inv.operation_key, Sha256::digest(&payload)), + }, + }; + let cancel = ctx.cancel.clone(); + let id = document_id.clone(); + let save = operation_id.clone(); + let (staged, permit) = tokio::task::spawn_blocking(move || { + let mut editor = Editor::model(&storage, &id, owner, &mut document, || { + cancel.is_cancelled() + }) + .map_err(error)?; + if let Some(receipt) = editor.lookup_operation(&save).map_err(error)? { + return Ok((Staged::Existing(receipt), permit)); + } + if editor.draft().base().sha256() != expected { + return Err(failed("Binary base SHA-256 changed; inspect again")); + } + if editor.lookup_operation("patch").map_err(error)?.is_none() { + batch.revision = editor.draft().revision(); + editor.patch("patch", &batch).map_err(error)?; + } + let review = editor + .review(0, 128, || cancel.is_cancelled()) + .map_err(error)?; + Ok::<_, ToolError>((Staged::Review(Box::new(editor), Box::new(review)), permit)) + }) + .await + .map_err(|_| failed("Binary staging worker interrupted"))??; + let Staged::Review(mut editor, review) = staged else { + let Staged::Existing(operation) = staged else { + unreachable!() + }; + return Ok(json!({"document_id":document_id,"operation":operation})); + }; + ctx.verify_binary_read(&path).await?; + ctx.authorize( + request.clone(), + request.resources.clone(), + json!({"byte_review":review,"binary_operation":"patch"}), + ) + .await?; + let grants = ( + decision(ctx.policy.decide(&Request { + tool: Some("binary".into()), + action: "read".into(), + resources: request.resources.clone(), + read_only: true, + ..Default::default() + }))?, + decision(ctx.policy.decide(&request))?, + ); + let host = ctx + .host + .weak + .upgrade() + .ok_or_else(|| failed("Binary host unavailable"))?; + let inv = ctx.inv.clone(); + let cancel = ctx.cancel.clone(); + let executor = tokio::runtime::Handle::current(); + let save = operation_id.clone(); + ctx.host.note_call_metadata( + &ctx.inv.session_id, + &ctx.inv.call_id, + json!({"binary_document_id":document_id,"binary_operation_id":operation_id}), + ); + let receipt = runtime + .binary_tool_work( + inv.clone(), + document_id.clone(), + resource, + grants, + move |work| { + let _permit = permit; + let prepared = editor.prepare_save(&save, &review.fingerprint, || { + work.is_cancelled() || cancel.is_cancelled() + })?; + let operation = match prepared { + SavePreparation::Existing(operation) => operation, + SavePreparation::Ready(prepared) => editor.commit_save_checked( + *prepared, + || work.is_cancelled() || cancel.is_cancelled(), + || { + work.verify()?; + executor.block_on(async { + let policy = host.policy(&inv).await.map_err(|_| Error::Changed)?; + let fresh = Ctx { + skill_paths: Default::default(), + loaded_skill: Default::default(), + provenance: Default::default(), + compiler_feedback: Default::default(), + hook_decision: None, + denial: Default::default(), + host: &host, + inv: &inv, + policy, + location: std::path::PathBuf::from(&inv.directory), + cancel: cancel.clone(), + }; + crate::sandboxing::binary_edit_boundary(&fresh, &path) + .map_err(|_| Error::Changed)?; + fresh + .verify_binary_read(&path) + .await + .map_err(|_| Error::Changed)?; + for (action, grant) in [("read", grants.0), ("edit", grants.1)] { + let mut check = request.clone(); + check.action = action.into(); + check.read_only = action == "read"; + check.file_edit = action == "edit"; + let current = fresh.policy.decide(&check); + if !matches!(current, Decision::Allow) + && !(current == Decision::Ask && grant == FileDecision::Ask) + { + return Err(Error::Changed); + } + } + if cancel.is_cancelled() { + return Err(Error::Cancelled); + } + Ok(()) + }) + }, + )?, + }; + work.resume_notifications(&mut editor); + Ok::<_, Error>(operation) + }, + ) + .await + .map_err(|error| { + failed(format!( + "Binary operation {operation_id} needs status review before any retry: {error}" + )) + })? + .map_err(error)?; + ctx.host.note_call_metadata(&ctx.inv.session_id, &ctx.inv.call_id, + json!({"binary_document_id":document_id,"binary_operation_id":operation_id,"binary_operation":receipt})); + Ok(json!({"document_id":document_id,"operation":receipt})) +} + +fn decision(value: Decision) -> Result { + match value { + Decision::Allow => Ok(FileDecision::Allow), + Decision::Ask => Ok(FileDecision::Ask), + Decision::Deny(_) => Err(failed("Binary file authority denied")), + } +} diff --git a/crates/cyber-tools/src/tools/mod.rs b/crates/cyber-tools/src/tools/mod.rs index 159374c5e..a9552b3d1 100644 --- a/crates/cyber-tools/src/tools/mod.rs +++ b/crates/cyber-tools/src/tools/mod.rs @@ -3,6 +3,7 @@ pub(crate) mod agent; mod background; pub(crate) mod bash; +mod binary; pub(crate) mod collaboration; pub(crate) mod computer; mod deliver; @@ -63,6 +64,7 @@ pub(crate) trait Tool: Send + Sync { pub(crate) fn all() -> Vec> { let tools: Vec> = vec![ Box::new(fs::Read), + Box::new(binary::Binary), Box::new(lsp::Lsp), Box::new(fs::Write), Box::new(fs::Edit), @@ -135,7 +137,7 @@ pub(crate) fn action_of(name: &str) -> &str { match name { "write" | "edit" | "apply_patch" | "notebook_edit" => "edit", "powershell" | "monitor" | "pty_start" | "pty_write" | "pty_read" => "bash", - "send_file" => "read", + "send_file" | "binary" => "read", "enter_worktree" | "exit_worktree" => "worktree", "workflow" => crate::permissions::WORKFLOW_RUN, "collab_send_message" => crate::permissions::COLLAB_MESSAGE_SEND, diff --git a/crates/cyber-tools/src/worktrees.rs b/crates/cyber-tools/src/worktrees.rs index 52c96e6a6..a2adf1e29 100644 --- a/crates/cyber-tools/src/worktrees.rs +++ b/crates/cyber-tools/src/worktrees.rs @@ -202,26 +202,28 @@ impl BuiltinHost { creating: bool, cancel: CancellationToken, ) -> Result { - let inv = Invocation { - tool_output: Default::default(), - registration: None, - session_id: info.id.clone(), - directory: info.directory.clone(), - agent: info.agent.clone(), - mode: info.mode.clone(), - rules: info.rules.clone(), - message_id: String::new(), - call_id: cyber_core::ids::new_id("call"), - operation_key: String::new(), - name: "worktree".into(), - input: serde_json::Value::Null, - attempt: 1, - asker: Asker::detached(), - }; - self.claim_location(&inv, info.worktree_id.as_deref(), creating, cancel) + let inv = location_invocation(info); + self.claim_location(&inv, info.worktree_id.as_deref(), creating, cancel, None) .await } + pub(crate) async fn claim_binary_worktree_location( + &self, + info: &SessionInfo, + document: &str, + cancel: CancellationToken, + ) -> Result { + let inv = location_invocation(info); + self.claim_location( + &inv, + info.worktree_id.as_deref(), + false, + cancel, + Some(document), + ) + .await + } + pub(crate) async fn claim_hook_test_location( &self, event: &cyber_core::hooks::HookEvent, @@ -248,7 +250,7 @@ impl BuiltinHost { attempt: 1, asker: Asker::detached(), }; - self.claim_location(&inv, None, true, cancel).await + self.claim_location(&inv, None, true, cancel, None).await } async fn claim_location( @@ -257,6 +259,7 @@ impl BuiltinHost { bound: Option<&str>, creating: bool, cancel: CancellationToken, + binary: Option<&str>, ) -> Result { use cyber_server::runtime::LocationLease; let activity = self @@ -304,8 +307,16 @@ impl BuiltinHost { let mut leases = Vec::with_capacity(locations.len()); for (repository, managed) in locations { leases.push( - retry_repository_busy(&ctx.cancel, || { - repository.claim(&execution, &managed, &inv.session_id) + retry_repository_busy(&ctx.cancel, || async { + if let Some(document) = binary { + repository + .claim_binary(&execution, &managed, document) + .await + } else { + repository + .claim(&execution, &managed, &inv.session_id) + .await + } }) .await .map_err(|error| error.to_string())?, @@ -1533,3 +1544,22 @@ fn authorize_owned_worktree( _ => Ok(()), } } + +fn location_invocation(info: &SessionInfo) -> Invocation { + Invocation { + tool_output: Default::default(), + registration: None, + session_id: info.id.clone(), + directory: info.directory.clone(), + agent: info.agent.clone(), + mode: info.mode.clone(), + rules: info.rules.clone(), + message_id: String::new(), + call_id: cyber_core::ids::new_id("call"), + operation_key: String::new(), + name: "worktree".into(), + input: serde_json::Value::Null, + attempt: 1, + asker: Asker::detached(), + } +} diff --git a/crates/cyber-tools/tests/binary_model.rs b/crates/cyber-tools/tests/binary_model.rs new file mode 100644 index 000000000..a58d1c0ef --- /dev/null +++ b/crates/cyber-tools/tests/binary_model.rs @@ -0,0 +1,231 @@ +//! Real runtime dispatch into the shared native binary engine, with no provider. +#![cfg(any(target_os = "macos", target_os = "linux"))] +mod support; +use cyber_core::binary::storage::StorageRoot; +use cyber_server::runtime::{CallStatus, NoSnapshots, RetrySafety}; +use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; +use std::sync::Arc; +use support::{ + Fixture, + flow::{Flow, call, text}, +}; + +fn input(hash: &str) -> Value { + json!({"operation":"patch","path":"bytes.bin","expected_base_sha256":hash, + "splices":[{"offset":"1","delete_length":"2","expected_hex":"ff80","insert_hex":"00fe01"}]}) +} +fn flow(input: Value, config: Value) -> Flow { + interactive_flow(input, config, false) +} +fn interactive_flow(input: Value, config: Value, interactive: bool) -> Flow { + let f = Fixture::with_policy( + "bash", + cyber_sandbox::find_helper(), + Some("workspace-write".into()), + ); + f.set_config(config); + std::fs::write(f.repo.join("bytes.bin"), [0, 255, 128, 0, 10]).unwrap(); + let flow = Flow::with( + f, + vec![call("change", "binary", input), text("done")], + interactive, + Arc::new(NoSnapshots), + ); + flow.runtime + .configure_binary_storage(Arc::new(StorageRoot::prepare(flow.f.dir.path()).unwrap())) + .unwrap(); + flow +} + +#[tokio::test] +async fn binary_patch_approval_contains_exact_byte_review_and_fresh_revocation_wins() { + use cyber_server::runtime::{PendingKind, PermissionReply}; + let hash = format!("{:x}", Sha256::digest([0, 255, 128, 0, 10])); + for revoke in [false, true] { + let flow = interactive_flow( + input(&hash), + json!({"permissions":{"read":"allow","edit":"ask"}}), + true, + ); + let id = flow.session("default").await; + flow.prompt(&id, "review and apply").await; + let pending = flow.pending(&id).await; + let PendingKind::Permission(ask) = pending.kind else { + panic!("byte edit approval required") + }; + assert_eq!(ask.action, "edit"); + assert_eq!(ask.metadata["byte_review"]["draft"]["base_sha256"], hash); + assert_eq!( + ask.metadata["byte_review"]["draft"]["ranges"][0]["before_hex"], + "ff80" + ); + assert_eq!( + ask.metadata["byte_review"]["draft"]["ranges"][0]["after_hex"], + "00fe01" + ); + if revoke { + flow.f + .set_config(json!({"permissions":{"read":"allow","edit":"deny"}})); + } + flow.runtime + .reply_permission(&pending.id, PermissionReply::Once) + .await + .unwrap(); + flow.settle(&id).await; + let state = flow.runtime.state(&id).await.unwrap(); + assert_eq!( + state.calls["change"].status, + if revoke { + CallStatus::Error + } else { + CallStatus::Ok + }, + "{:?}", + state.calls["change"] + ); + assert_eq!( + std::fs::read(flow.f.repo.join("bytes.bin")).unwrap(), + if revoke { + vec![0, 255, 128, 0, 10] + } else { + vec![0, 0, 254, 1, 0, 10] + } + ); + flow.runtime.shutdown().await; + } +} + +#[tokio::test] +async fn stopping_during_binary_review_leaves_the_source_unchanged() { + let hash = format!("{:x}", Sha256::digest([0, 255, 128, 0, 10])); + let flow = interactive_flow( + input(&hash), + json!({"permissions":{"read":"allow","edit":"ask"}}), + true, + ); + let id = flow.session("default").await; + flow.prompt(&id, "review patch").await; + let _pending = flow.pending(&id).await; + flow.runtime.interrupt(&id).await.unwrap(); + flow.settle(&id).await; + assert_eq!( + std::fs::read(flow.f.repo.join("bytes.bin")).unwrap(), + [0, 255, 128, 0, 10] + ); + assert!( + flow.runtime + .binary_storage() + .unwrap() + .session_saved_references(&id, &flow.f.repo, || false) + .unwrap() + .is_empty() + ); + flow.runtime.shutdown().await; +} + +#[tokio::test] +async fn dispatched_plan_read_returns_bounded_bytes_without_a_private_draft() { + let flow = flow( + json!({"operation":"read","path":"bytes.bin","limit":3}), + json!({"permissions":{"read":"allow","edit":"deny"}}), + ); + let id = flow.session("plan").await; + flow.prompt(&id, "inspect bytes").await; + flow.settle(&id).await; + let state = flow.runtime.state(&id).await.unwrap(); + let call = &state.calls["change"]; + assert_eq!(call.status, CallStatus::Ok, "{call:?}"); + let page: Value = serde_json::from_str(call.output.as_ref().unwrap()).unwrap(); + assert_eq!(page["hex"], "00ff80"); + assert_eq!(page["length"], "3"); + assert!( + std::fs::read_dir(flow.runtime.binary_storage().unwrap().path()) + .unwrap() + .next() + .is_none() + ); + flow.runtime.shutdown().await; +} + +#[tokio::test] +async fn exact_model_binary_patch_installs_bytes_and_publishes_saved_history() { + let hash = format!("{:x}", Sha256::digest([0, 255, 128, 0, 10])); + let flow = flow( + input(&hash), + json!({"permissions":{"read":"allow","edit":"allow"}}), + ); + let id = flow.session("default").await; + flow.prompt(&id, "apply the exact binary patch").await; + flow.settle(&id).await; + let state = flow.runtime.state(&id).await.unwrap(); + let call = &state.calls["change"]; + assert_eq!(call.status, CallStatus::Ok, "{call:?}"); + assert_eq!(call.retry_safety, RetrySafety::Never); + let receipt: Value = serde_json::from_str(call.output.as_ref().unwrap()).unwrap(); + assert_eq!(receipt["operation"]["outcome"], "acknowledged"); + assert_eq!(receipt["operation"]["file_effect"], "changed"); + assert_eq!( + std::fs::read(flow.f.repo.join("bytes.bin")).unwrap(), + [0, 0, 254, 1, 0, 10] + ); + let storage = flow.runtime.binary_storage().unwrap(); + let refs = storage + .session_saved_references(&id, &flow.f.repo, || false) + .unwrap(); + assert_eq!(refs.len(), 1); + assert_eq!(refs[0].version.before_sha256, hash); + assert_eq!( + refs[0].version.after_sha256, + format!("{:x}", Sha256::digest([0, 0, 254, 1, 0, 10])) + ); + flow.runtime.shutdown().await; +} + +#[tokio::test] +async fn binary_model_patch_refuses_plan_denied_reads_denied_edits_stale_hash_and_wrong_bytes() { + let hash = format!("{:x}", Sha256::digest([0, 255, 128, 0, 10])); + let allow = json!({"permissions":{"read":"allow","edit":"allow"}}); + let mut wrong_bytes = input(&hash); + wrong_bytes["splices"][0]["expected_hex"] = json!("ffff"); + for (mode, config, patch) in [ + ("plan", allow.clone(), input(&hash)), + ( + "default", + json!({"permissions":{"read":"deny","edit":"allow"}}), + input(&hash), + ), + ( + "default", + json!({"permissions":{"read":"allow","edit":"deny"}}), + input(&hash), + ), + ("default", allow.clone(), input(&"0".repeat(64))), + ("default", allow, wrong_bytes), + ] { + let flow = flow(patch, config); + let id = flow.session(mode).await; + flow.prompt(&id, "apply the exact binary patch").await; + flow.settle(&id).await; + let state = flow.runtime.state(&id).await.unwrap(); + assert_eq!( + state.calls["change"].status, + CallStatus::Error, + "{mode}: {:?}", + state.calls["change"] + ); + assert_eq!( + std::fs::read(flow.f.repo.join("bytes.bin")).unwrap(), + [0, 255, 128, 0, 10] + ); + assert!( + flow.runtime + .binary_storage() + .unwrap() + .session_saved_references(&id, &flow.f.repo, || false) + .unwrap() + .is_empty() + ); + flow.runtime.shutdown().await; + } +} diff --git a/crates/cyber-tools/tests/golden.rs b/crates/cyber-tools/tests/golden.rs index 72224f670..3f0134a63 100644 --- a/crates/cyber-tools/tests/golden.rs +++ b/crates/cyber-tools/tests/golden.rs @@ -133,6 +133,37 @@ async fn advertised_tool_names_are_a_subset_of_the_canonical_registry() { ); } +#[tokio::test] +async fn binary_read_golden() { + let f = Fixture::new(); + f.set_config(json!({"permissions":{"read":"allow"}})); + std::fs::write(f.repo.join("bytes.bin"), [0, 255, 128, 27, 65, 10]).unwrap(); + let output = ok(f + .call( + "plan", + "binary", + json!({ + "operation":"read", "path":"bytes.bin", "offset":"1", "limit":4 + }), + ) + .await); + let mut page: serde_json::Value = serde_json::from_str(&output).unwrap(); + // The live observation includes file identity/mtime, so only that token is + // unstable. Preserve all byte, bounds, hash and source metadata contracts. + assert!( + page["metadata"] + .as_object_mut() + .unwrap() + .remove("revision") + .unwrap() + .is_string() + ); + // Workspace feature unification can enable preserve_order; JSON object key + // order is not part of this contract under either serde_json representation. + page.sort_all_objects(); + support::golden(&f, "binary", &serde_json::to_string_pretty(&page).unwrap()); +} + #[tokio::test] async fn filesystem_goldens() { let f = Fixture::new(); diff --git a/crates/cyber-tools/tests/goldens/binary.txt b/crates/cyber-tools/tests/goldens/binary.txt new file mode 100644 index 000000000..8c7b01f47 --- /dev/null +++ b/crates/cyber-tools/tests/goldens/binary.txt @@ -0,0 +1,14 @@ +{ + "ascii": "...A", + "eof": false, + "hex": "ff801b41", + "length": "4", + "metadata": { + "length": "6", + "path": "bytes.bin", + "source": "live", + "within_edit_size_limit": true + }, + "offset": "1", + "page_sha256": "b790daf98db2c34783564c4ab0c146dde11698376898461929725fe09b8fce1e" +} diff --git a/crates/cyber-tools/tests/tools.rs b/crates/cyber-tools/tests/tools.rs index 4b8920e21..b280c7e1b 100644 --- a/crates/cyber-tools/tests/tools.rs +++ b/crates/cyber-tools/tests/tools.rs @@ -1112,3 +1112,84 @@ async fn builtin_read_cannot_obtain_owner_api_password() { .contains("OWNER_PUBLIC_CONTROL") ); } + +#[tokio::test] +async fn binary_model_reads_inspects_and_searches_exact_bytes_in_plan_without_text_edits() { + use sha2::{Digest, Sha256}; + let f = Fixture::new(); + let bytes: Vec = (0..=255).collect(); + std::fs::write(f.repo.join("bytes.bin"), &bytes).unwrap(); + *f.config.lock().unwrap() = json!({"permissions":{"read":"allow"}}); + assert!( + f.tool_names("plan", false) + .iter() + .any(|name| name == "binary") + ); + let inspect: serde_json::Value = serde_json::from_str(&ok(f + .call( + "plan", + "binary", + json!({"operation":"inspect","path":"bytes.bin"}), + ) + .await)) + .unwrap(); + assert_eq!( + inspect["base_sha256"], + format!("{:x}", Sha256::digest(&bytes)) + ); + let read:serde_json::Value=serde_json::from_str(&ok(f.call("plan","binary",json!({"operation":"read","path":"bytes.bin","offset":"0","limit":256,"revision":inspect["metadata"]["revision"]})).await)).unwrap(); + assert_eq!( + read["hex"], + bytes.iter().map(|b| format!("{b:02x}")).collect::() + ); + assert_eq!(read["length"], "256"); + assert!(!read["ascii"].as_str().unwrap().contains('\x1b')); + let search: serde_json::Value = serde_json::from_str(&ok(f + .call( + "plan", + "binary", + json!({"operation":"search","path":"bytes.bin","hex":"feff","offset":"0"}), + ) + .await)) + .unwrap(); + assert_eq!(search["result"]["offsets"], json!(["254"])); + for input in [ + json!({"operation":"read","path":"bytes.bin","offset":1}), + json!({"operation":"read","path":"bytes.bin","offset":"01"}), + json!({"operation":"read","path":"bytes.bin","limit":4097}), + json!({"operation":"read","path":"bytes.bin","revision":"stale"}), + ] { + assert!(matches!( + f.call("plan", "binary", input).await, + ToolOutcome::Failed(_) + )); + } + *f.config.lock().unwrap() = json!({"permissions":{"read":"deny","edit":"allow"}}); + assert!( + !f.tool_names("default", false) + .iter() + .any(|name| name == "binary") + ); + let denied = failed( + f.call( + "default", + "binary", + json!({"operation":"read","path":"bytes.bin"}), + ) + .await, + ); + assert!(!denied.contains(read["hex"].as_str().unwrap())); + std::fs::write(f.dir.path().join("outside.bin"), b"private outside").unwrap(); + *f.config.lock().unwrap() = + json!({"permissions":{"read":"allow","external_directory":"allow"}}); + assert!(matches!( + f.call( + "default", + "binary", + json!({"operation":"read","path":f.dir.path().join("outside.bin")}) + ) + .await, + ToolOutcome::Failed(_) + )); + assert_eq!(std::fs::read(f.repo.join("bytes.bin")).unwrap(), bytes); +} diff --git a/docs/binary-editor.md b/docs/binary-editor.md index c3e3891a1..62a813f67 100644 --- a/docs/binary-editor.md +++ b/docs/binary-editor.md @@ -15,8 +15,8 @@ Location; the direct command refuses a target from another project. The terminal opens in view mode. On admitted Linux/macOS workspaces, F2 prepares an immutable private draft and enters edit mode. `--read-only` permanently disables -edit preparation for that document. Windows supports live viewing. Shared saved -history/snapshot integration, model operations and remaining acceptance work are +edit preparation for that document. Windows supports live viewing. Checkpoint/rewind integration, configured history +retention and remaining acceptance work are tracked under [issue #94](https://github.com/CyberdyneCorp/cybercode-cli/issues/94). In edit mode, type complete hex byte pairs to overwrite one byte; Esc cancels an @@ -213,3 +213,44 @@ saved references; larger histories use the paged source-history routes. Checkpoint/rewind integration remains unfinished. The core retention primitive exists, but the configured retention/checkpoint adapter remains unfinished; public history access does not silently delete data. + +## Model binary operations + +The capability-owned `binary` tool accepts `inspect`, `read`, `search` and `patch`. +It accesses source files within the current project, never a manual editor's private +draft. Reads remain available in Plan mode; patching requires a native Linux/macOS +workspace with current read and edit authority and an enforced writable sandbox. +The ordinary text read/edit tools retain their text semantics. + +Inspect streams a SHA-256 for sources up to 512 MiB; larger files remain live-view +only. Read defaults to 256 bytes and caps at 4096, with explicit hex, printable +ASCII, exact decimal-string offsets and page fingerprints. Search uses the same +bounded literal-hex search and continuation limits as the manual workspace. + +For example, after inspecting the current digest, the model may request: + +```json +{ + "operation": "patch", + "path": "fixtures/header.bin", + "expected_base_sha256": "<64 lowercase hexadecimal characters from inspect>", + "splices": [ + {"offset": "1", "delete_length": "2", "expected_hex": "ff80", "insert_hex": "00fe01"} + ] +} +``` + +The batch must match the whole prepared base and every deleted byte. The same +128-splice/64-KiB combined expected-and-inserted-byte limit applies. Approval receives +the exact byte review, final length/digest and native metadata/effect disclosure. +The actual worker rechecks current invocation, ancestor, read/edit and sandbox +authority immediately before durable save intent. It retains independent native +checkout and path ownership through completion after caller disposal. + +The result is an explicit acknowledged/conflict/unknown operation receipt. The tool +is serialized and has no automatic retry. Its stable invocation/payload identity +prevents reusing an operation for changed bytes. An unknown outcome must not be +retried under a new call ID: preserve the document/operation identities for recovery. +The owner API does not admit model drafts as manual buffers; a dedicated model +recovery surface remains part of the remaining issue #94 acceptance work. Plugin +registrations cannot invoke this built-in tool as a capability bypass. diff --git a/openspec/changes/add-binary-hex-editor/implementation.md b/openspec/changes/add-binary-hex-editor/implementation.md index 9a38801d2..14221a65e 100644 --- a/openspec/changes/add-binary-hex-editor/implementation.md +++ b/openspec/changes/add-binary-hex-editor/implementation.md @@ -608,3 +608,51 @@ Fresh read denial hides the reference. The focused catalog regression, 15 target TUI tests (including the shared diff overlay), OpenAPI artifact test, SDK typecheck and 125 unit/mock tests, core/server/TUI all-target Clippy and strict OpenSpec validation passed. No new database migration is required for this adapter. + + +### Model binary tool (follow-up to merged PR #103) + +The built-in `binary` tool now offers inspect/read/search and exact expected-digest +patches. Read pages default to 256 bytes and cap at 4096; inspect streams live +SHA-256 evidence through 512 MiB without preparing a private draft. Plan mode can +read/search, while patches additionally require edit authority and a native writable +workspace sandbox. Fully denied reads hide the tool. Text tools retain their +existing semantics. + +Each patch has a separate invocation/payload-bound private namespace. Its small +owner header is authenticated before a prior journal/base is loaded. Expected +whole-base and per-splice bytes, 128-splice/64-KiB limits and canonical payload +identity are enforced by the shared core. Permission approval receives the exact +byte review before mutation ownership. The retained filesystem worker validates +its dispatched built-in call, current Session/Location/ancestors, current tool and +read/edit policy and sandbox boundary immediately before intent. A dedicated +`bin_` checkout lease avoids reacquiring the active actor's Session lease and +cannot acknowledge generic subprocess activity. Native receipts and post-commit +notifications use the same save engine as the owner editor. The tool is serialized +and never automatically retries an uncertain replacement. + +Verification so far: 83 binary-core tests passed (one process-death helper ignored +in this selector); the independent checkout-owner regression, five real runtime +model-tool regressions, the Plan read/search regression and all 45 general tool tests passed. Coverage includes +exact growth bytes and saved history, digest/expected-byte refusal, read/edit denial, +Plan refusal, approval byte metadata, fresh policy revocation during approval, Stop +during approval and separation from the manual draft namespace. Strict OpenSpec +validation passed all 114 items. All six existing server-worker regressions and +core/server/tools all-target Clippy passed. Review identified and corrected the +production AppHost forwarding of the independent worker lease; its dedicated +regression and app all-target Clippy passed. Formatting and diff checks passed. + +This is not full #94 acceptance. Dedicated model receipt recovery, configured +retention/checkpoint/rewind adapters, remaining UI recovery/accessibility and named +performance/platform acceptance remain open. No external provider was called by +these model-tool tests. Linux and Windows verification for this follow-up must come +from CI; local verification is macOS only. + + +CI follow-up, 2026-11-03: Linux/macOS Rust and Windows build failed the +canonical tool-registry/golden coverage checks because the new tool was listed only +in the active proposal catalog and lacked a golden fixture. The canonical catalog +now lists the implemented tool. A real Plan-mode binary read golden retains bytes, +bounds, fingerprint and source metadata, normalizing only the live revision and +JSON key order. All 14 golden tests and strict OpenSpec validation (114 items) +passed locally. Cross-platform CI must verify the corrected head before merge. diff --git a/openspec/changes/add-binary-hex-editor/tasks.md b/openspec/changes/add-binary-hex-editor/tasks.md index 8850669f8..b54e2012f 100644 --- a/openspec/changes/add-binary-hex-editor/tasks.md +++ b/openspec/changes/add-binary-hex-editor/tasks.md @@ -30,7 +30,7 @@ staged-edit/save/model/integration requirements below. - [ ] 4.1 Implement authenticated `fs` document/page/search/patch/history/review/save/restore/operation/close routes with exact Session/Location scope and no raw-byte URLs/logging. - [ ] 4.2 Generate OpenAPI/TypeScript types and shared client helpers with lossless offsets, bounded decoding, operation lookup and revision validation; preserve text content routes. -- [ ] 4.3 Register the capability-owned `binary` tool and implement bounded inspect/read/search/patch through the shared service, isolated from unsaved owner drafts. +- [x] 4.3 Register the capability-owned `binary` tool and implement bounded inspect/read/search/patch through the shared service, isolated from unsaved owner drafts. - [ ] 4.4 Enforce read+edit, plan/read-only/deny ceilings, existing output budgets, permission byte-diff review and plugin filesystem intersections; retain normal text-tool behavior. - [ ] 4.5 Test owner auth/Origin, cross-document/Session/Location access, stale generation, read-denied/write-allowed cases, malformed encodings, oversize payloads and secret-safe errors. diff --git a/openspec/specs/builtin-tools/spec.md b/openspec/specs/builtin-tools/spec.md index d637d9b2b..15e5f48a7 100644 --- a/openspec/specs/builtin-tools/spec.md +++ b/openspec/specs/builtin-tools/spec.md @@ -24,6 +24,7 @@ Defines the built-in tools Cyber Code exposes to models: parameters, limits, per - `list_sessions`, `send_message`, `watch_session` (cross-session-messaging) - `channel_reply` (channels) - `memory` (memory) +- `binary` (binary-editing) - `wait_for_mcp`, `mcp_list_resources`, `mcp_list_resource_templates`, `mcp_read_resource` (mcp) - `publish_artifact` (session-sharing) - `advisor` (provider-catalog)