Stream Issue Scan without loading every file body

[samwdp]

Problem Statement

Issue Scan walks the Project Workspace tree and read_to_strings every file before looking for unlinked TODO/FIXME comments. Large repos hitch the worker, allocate huge strings, and duplicate git ls-files work. Capture on save already passes the in-memory buffer text and must stay non-blocking. Scan is the expensive path. Binary files are skipped only after a full read (\0 check).

Solution

Stream Issue Scan: for each candidate path, scan bytes for TODO/FIXME markers without keeping the whole file if no marker exists. Skip obvious binaries early. Reuse a repository file list cache when present (#72). Capture-on-save stays buffer-text based. Issues are still minted; comment rewrite rules stay the same.

User Stories

1. As a developer running Issue Scan on a large Project Workspace, I want it to finish without loading every file into RAM, so that the editor stays usable.
2. As a developer with a TODO in one file, I want that Issue Captured, so that streaming does not miss comments.
3. As a developer with HACK/XXX comments, I want them still ignored, so that marker rules do not change.
4. As a developer with a binary asset, I want it skipped without a full UTF-8 string, so that scans do not stall on media.
5. As a developer with files under issues/ or .scratch, I want them still excluded, so that the Issue Store is not scanned.
6. As a developer with target/ and node_modules, I want those directories still skipped.
7. As a developer whose tree walk fails, I want git listing fallback, so that scan still runs.
8. As a developer saving a file, I want Capture of that buffer’s text without a disk reread, so that after-save stays as today.
9. As a developer with an unlinked TODO that changed before rewrite applied, I want the existing “mint Issue, leave comment if mismatch” rule.
10. As a developer scanning while files change, I want stale paths dropped from Issues when comments disappear, never deleting the Issue (existing Scan semantics).
11. As a developer with a Code Reference whose Issue Id has no file, I want it reported, not auto-recreated.
12. As a developer in tests, I want capture_file fixtures unchanged; add a scan over a temp tree with one TODO and one large marker-free file that does not require keeping the large file’s full body (observable: scan still finds the TODO; worker completes).

Implementation Decisions

• Seam: Issue Scan file collection + capture_file. Do not change Issue markdown format or statuses.
• Prefer scanning a byte/line iterator; only retain full text when a marker is found and rewrite may be needed.
• Skip issues/, .git, target, node_modules, .scratch as today.
• Share repository path listing with the Lazy previews for Workspace Files picker #72 cache when that helper exists; if Lazy previews for Workspace Files picker #72 is not landed, still stream reads.
• Capture-after-save already sends buffer text + revision; do not switch it to disk.
• Keep Scan on the issues worker (async, non-blocking UI).
• Untracked files: keep tree walk as primary so untracked sources remain included.

Testing Decisions

• Good tests: capture_mints_and_rewrites_todo_and_fixme; capture_ignores_hack_and_xxx; capture_can_finish_after_caller_continues; board_hides_closed_by_default. Add scan over mixed tree (TODO file + skipped dir + binary-like \0).
• Primary seam: issues crate scan/capture.
• Secondary seam: shell issues worker enqueue/complete notifications.
• Prior art: Issue Scan complete/fail notifications; workspace-relative path helpers.

Out of Scope

• Changing Issue Status workflow
• Parsing TODO inside strings/comments via tree-sitter
• Workspace Files picker UI (Lazy previews for Workspace Files picker #72)
• Blocking the UI on Scan (must remain async)

Further Notes

• Domain: Issue, Capture, Issue Scan, Issue Store, Code Reference, Issue Id. Avoid “ticket/TODO” as the record.
• CONTEXT: Capture must not block save; Scan is the workspace-wide pass.