Skip to content

Cowork ingestion advances past a transcript record that is still being written #372

Description

@DivyamTalwar

Problem

Cowork can read a JSONL transcript while its writer has emitted only part of the final JSON record. The existing parse-failure path advances processedLines past that unterminated tail. Completing the same line later does not increase the line count, so future polls skip the message permanently.

Reproduced against main ce30de7ca94115cb73fa991538c6d2595ac2622a with synthetic data only.

Focused fix and ownership

Retain the watermark only for an unparseable final record without a terminating newline. Valid JSON without a final newline is still processed; malformed newline-terminated records retain the existing skip behavior. Earlier complete records and independent transcripts continue to progress.

I have prepared the implementation and checked the existing issue/PR inventory for overlapping work.

Reproduction and validation

All three new real-file regressions fail on unchanged production code and pass on the patch. They drive the public ingestion path through split writes and repeated polling, including complete no-newline JSON, CRLF/blank lines, a split UTF-8 sequence, independent transcript progress, and no duplicate upload of earlier records. Network and worker execution are mocked; actual filesystem and production parsing/watermarks are exercised.

The exact proposed source commit 655e40559e6d1822f4a8a1342050bb8ea7424755 passes the full Node 22/Linux suite with coverage: 5852 tests passed, zero failed, as well as typecheck/build and repository quality gates. Evidence: https://github.com/DivyamTalwar/hivemind/actions/runs/35666411820

Scope and limits

This does not redesign transcript rotation, truncate/rewrite recovery, cross-workspace queue identity, redaction, or real agent lifecycle behavior. It is independent of the existing redaction PR #363. An older equivalent branch in my fork is not being submitted separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions