Skip to content

refactor(remotefs): track the Windows file position in Go - #480

Merged
kke merged 1 commit into
mainfrom
refactor/rigrcp-go-side-seek
Sep 24, 2026
Merged

kke merged 1 commit into
mainfrom
refactor/rigrcp-go-side-seek

Conversation

@kke

@kke kke commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

winFile.Seek was a round trip to the rigrcp helper, and every later r or w relied on the handle's position staying where the previous command had left it. The position and size now live on winFile, the way PosixFile keeps them, and every r and w command carries the offset it applies to. Seek no longer contacts the host. The end Seek measures from is the size at open or the furthest offset written, whichever is greater, and an empty Write returns without a round trip so it cannot move that end. The handle also remembers whether the caller opened it for reading and for writing, and refuses an operation that access does not allow before contacting the host, as PosixFile does. So an empty write to a read-only file fails too. The access checked is the one the caller asked for, not the wider one some modes need on the host.

This also takes a round trip out of every append. An O_APPEND write used to send "s 0 End" before its "w". Now it sends its w at the size winFile already knows. That size is the real end: the helper opens the file with FileShare.None, so no other writer can grow the file while the handle is open.

The helper used to acknowledge a w before it had even read the payload, and sent nothing once the write was done. A failed write, such as a full disk, surfaced as the reply to whatever command came next, or not at all if none did. With Seek now local, Write would report success and SeekEnd would measure from an end that was never written. The helper now sends a second reply once the payload is in the file, and Write waits for it before moving the position or the end. A write the helper reports as failed ends the session: a failed FileStream.Write can leave part of the payload behind, so the handle no longer knows its position or its end. A read or copy that stops partway ends it too, since the rest of the payload would otherwise be read as the reply to the next command. Each Write now waits for one more reply, which is what it costs to know the data landed.

With that, the helper's "s" command has no caller, so it goes, along with code that never did anything: HexDump, which pasted a dump of the whole short-read buffer into the error message, a FileStream.EndOfStream check on a property FileStream does not have, and assignments nothing read.

Seek stays in the remotefs.File API. This is a step towards replacing the long-lived helper with one-shot commands without losing it.

Refs #475

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The size documentation and zero-length write handling must be corrected before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Refactors Windows remote file handling to track offsets locally, eliminating remote seek round trips.

Changes:

  • Tracks file position and size in Go.
  • Sends explicit offsets for reads and writes.
  • Removes obsolete helper seek logic and expands tests.
File Description Review
remotefs/​winfs_test.go Updates protocol fakes and seek coverage. No issues.
remotefs/​winfile.go Implements local position and size tracking. Critical (2 votes): Correct the inaccurate size-invariant documentation at lines 227 and 280. Moderate (1 vote): Update size only when written > 0 to prevent zero-length writes beyond EOF from creating unintended holes.
remotefs/​rigrcp.ps1 Applies explicit offsets and removes seek handling. No issues.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread remotefs/winfile.go Outdated
@kke
kke force-pushed the refactor/rigrcp-go-side-seek branch from 5f724d2 to c796a62 Compare September 24, 2026 08:21
@kke
kke requested a balanced review from Copilot September 24, 2026 08:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Two moderate write-state and access-mode issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread remotefs/winfile.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The critical write-acknowledgement flaw and moderate access-check gap must be resolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread remotefs/winfile.go Outdated
@kke
kke force-pushed the refactor/rigrcp-go-side-seek branch from c9cb8ec to d523193 Compare September 24, 2026 08:39
@kke
kke requested a balanced review from Copilot September 24, 2026 08:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Two critical and two moderate correctness issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity

Open (2)
Resolved since last review (1)

Comment thread remotefs/winfile.go
Comment thread remotefs/winfile.go Outdated
@kke
kke force-pushed the refactor/rigrcp-go-side-seek branch from d523193 to d06bfa1 Compare September 24, 2026 08:48
@kke
kke requested a balanced review from Copilot September 24, 2026 08:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Two moderate correctness issues remain in CopyFrom access validation and append-position handling.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread remotefs/winfile.go
@kke
kke force-pushed the refactor/rigrcp-go-side-seek branch from d06bfa1 to 2aa757a Compare September 24, 2026 08:56
@kke
kke requested a balanced review from Copilot September 24, 2026 08:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The read error path can leave the helper session desynchronized and unsafe to reuse.

Review effort: Balanced
Findings: None

Resolved since last review (1)

…p seek

winFile.Seek was a round trip to the rigrcp helper, and every later r or
w relied on the handle's position staying where the previous command had
left it. The position and size now live on winFile, the way PosixFile
keeps them, and every r and w command carries the offset it applies to.
Seek no longer contacts the host. The end Seek measures from is the size
at open or the furthest offset written, whichever is greater, and an
empty Write returns without a round trip so it cannot move that end. The
handle also remembers whether the caller opened it for reading and for
writing, and refuses an operation that access does not allow before
contacting the host, as PosixFile does. So an empty write to a read-only
file fails too. The access checked is the one the caller asked for, not
the wider one some modes need on the host.

The helper used to acknowledge a w before it had even read the payload,
and sent nothing once the write was done. A failed write, such as a full
disk, surfaced as the reply to whatever command came next, or not at all
if none did. With Seek now local, Write would report success and SeekEnd
would measure from an end that was never written. The helper now sends a
second reply once the payload is in the file, and Write waits for it
before moving the position or the end. A write the helper reports as
failed ends the session: a failed FileStream.Write can leave part of the
payload behind, so the handle no longer knows its position or its end. A
read or copy that stops partway ends it too, since the rest of the
payload would otherwise be read as the reply to the next command. Each
Write now waits for one more reply, which is what it costs to know the
data landed.

This also takes a round trip out of every append. An O_APPEND write
used to send "s 0 End" before its "w". Now it sends its w at the size
winFile already knows. That size is the real end: the helper opens the
file with FileShare.None, so no other writer can grow the file while
the handle is open.

With that, the helper's "s" command has no caller, so it goes, along
with code that never did anything: HexDump, which pasted a dump of the
whole short-read buffer into the error message, a FileStream.EndOfStream
check on a property FileStream does not have, and assignments nothing
read.

Seek stays in the remotefs.File API. This is a step towards replacing
the long-lived helper with one-shot commands without losing it.

Refs #475

Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
@kke
kke force-pushed the refactor/rigrcp-go-side-seek branch from 2aa757a to 9dc0371 Compare September 24, 2026 09:03
@kke
kke requested a balanced review from Copilot September 24, 2026 09:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The unresolved append correctness and protocol desynchronization issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread remotefs/winfile.go
@kke
kke requested a balanced review from Copilot September 24, 2026 09:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The stateful cross-language protocol changes warrant final human review despite no identified blocking issues.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@kke
kke marked this pull request as ready for review September 24, 2026 09:43
@kke
kke merged commit c53aefb into main Sep 24, 2026
19 checks passed
@kke
kke deleted the refactor/rigrcp-go-side-seek branch September 24, 2026 09:43
kke added a commit that referenced this pull request Sep 24, 2026
Uploads to Windows hosts went through winFile.CopyFrom, which used
io.Copy. Upload wraps its source in an io.TeeReader, which hides any
io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB
became a w command of its own: a reply to wait for before the payload,
and since #480, a completion reply after it too. A 100 MiB upload made
about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time,
which brings that down to about 25. It does so for every source,
including an *os.File, whose own WriteTo would also write 32 KiB at a
time.

WriteFile had the opposite problem. bytes.Reader implements
io.WriterTo, so its whole payload arrived in one Write and went out as a
single w. The helper reads a command's whole payload into memory before
writing it, and parses its length as a PowerShell [int]. So WriteFile
held the entire file in the helper's memory and failed above 2 GiB.
Write now splits a larger payload into commands of at most 4 MiB.

The idle timeout is unaffected. It is still measured per 32 KiB piece
of payload handed to the pipe, not per command, so a larger command
does not raise the transfer rate a slow link has to sustain.

Refs #475

Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
kke added a commit that referenced this pull request Sep 24, 2026
Uploads to Windows hosts went through winFile.CopyFrom, which used
io.Copy. Upload wraps its source in an io.TeeReader, which hides any
io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB
became a w command of its own: a reply to wait for before the payload,
and since #480, a completion reply after it too. A 100 MiB upload made
about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time,
which brings that down to about 25. It does so for every source,
including an *os.File, whose own WriteTo would also write 32 KiB at a
time.

WriteFile had the opposite problem. bytes.Reader implements
io.WriterTo, so its whole payload arrived in one Write and went out as a
single w. The helper reads a command's whole payload into memory before
writing it, and parses its length as a PowerShell [int]. So WriteFile
held the entire file in the helper's memory and failed above 2 GiB.
Write now splits a larger payload into commands of at most 4 MiB.

The idle timeout is unaffected. It is still measured per 32 KiB piece
of payload handed to the pipe, not per command, so a larger command
does not raise the transfer rate a slow link has to sustain.

Refs #475

Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
kke added a commit that referenced this pull request Sep 24, 2026
Uploads to Windows hosts went through winFile.CopyFrom, which used
io.Copy. Upload wraps its source in an io.TeeReader, which hides any
io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB
became a w command of its own: a reply to wait for before the payload,
and since #480, a completion reply after it too. A 100 MiB upload made
about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time,
which brings that down to about 25. It does so for every source,
including an *os.File, whose own WriteTo would also write 32 KiB at a
time.

WriteFile had the opposite problem. bytes.Reader implements
io.WriterTo, so its whole payload arrived in one Write and went out as a
single w. The helper reads a command's whole payload into memory before
writing it, and parses its length as a PowerShell [int]. So WriteFile
held the entire file in the helper's memory and failed above 2 GiB.
Write now splits a larger payload into commands of at most 4 MiB.

The idle timeout is unaffected. It is still measured per 32 KiB piece
of payload handed to the pipe, not per command, so a larger command
does not raise the transfer rate a slow link has to sustain.

Refs #475

Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants