Repository navigation
refactor(remotefs): track the Windows file position in Go - #480
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The size documentation and zero-length write handling must be corrected before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Refactors Windows remote file handling to track offsets locally, eliminating remote seek round trips.
Changes:
- Tracks file position and size in Go.
- Sends explicit offsets for reads and writes.
- Removes obsolete helper seek logic and expands tests.
| File | Description | Review |
|---|---|---|
remotefs/winfs_test.go |
Updates protocol fakes and seek coverage. | No issues. |
remotefs/winfile.go |
Implements local position and size tracking. | Critical (2 votes): Correct the inaccurate size-invariant documentation at lines 227 and 280. Moderate (1 vote): Update size only when written > 0 to prevent zero-length writes beyond EOF from creating unintended holes. |
remotefs/rigrcp.ps1 |
Applies explicit offsets and removes seek handling. | No issues. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
5f724d2 to
c796a62
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Two moderate write-state and access-mode issues must be addressed before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (1)
Resolved since last review (1)
c796a62 to
c9cb8ec
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The critical write-acknowledgement flaw and moderate access-check gap must be resolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Resolved since last review (1)
c9cb8ec to
d523193
Compare
d523193 to
d06bfa1
Compare
d06bfa1 to
2aa757a
Compare
…p seek winFile.Seek was a round trip to the rigrcp helper, and every later r or w relied on the handle's position staying where the previous command had left it. The position and size now live on winFile, the way PosixFile keeps them, and every r and w command carries the offset it applies to. Seek no longer contacts the host. The end Seek measures from is the size at open or the furthest offset written, whichever is greater, and an empty Write returns without a round trip so it cannot move that end. The handle also remembers whether the caller opened it for reading and for writing, and refuses an operation that access does not allow before contacting the host, as PosixFile does. So an empty write to a read-only file fails too. The access checked is the one the caller asked for, not the wider one some modes need on the host. The helper used to acknowledge a w before it had even read the payload, and sent nothing once the write was done. A failed write, such as a full disk, surfaced as the reply to whatever command came next, or not at all if none did. With Seek now local, Write would report success and SeekEnd would measure from an end that was never written. The helper now sends a second reply once the payload is in the file, and Write waits for it before moving the position or the end. A write the helper reports as failed ends the session: a failed FileStream.Write can leave part of the payload behind, so the handle no longer knows its position or its end. A read or copy that stops partway ends it too, since the rest of the payload would otherwise be read as the reply to the next command. Each Write now waits for one more reply, which is what it costs to know the data landed. This also takes a round trip out of every append. An O_APPEND write used to send "s 0 End" before its "w". Now it sends its w at the size winFile already knows. That size is the real end: the helper opens the file with FileShare.None, so no other writer can grow the file while the handle is open. With that, the helper's "s" command has no caller, so it goes, along with code that never did anything: HexDump, which pasted a dump of the whole short-read buffer into the error message, a FileStream.EndOfStream check on a property FileStream does not have, and assignments nothing read. Seek stays in the remotefs.File API. This is a step towards replacing the long-lived helper with one-shot commands without losing it. Refs #475 Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
2aa757a to
9dc0371
Compare
Uploads to Windows hosts went through winFile.CopyFrom, which used io.Copy. Upload wraps its source in an io.TeeReader, which hides any io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB became a w command of its own: a reply to wait for before the payload, and since #480, a completion reply after it too. A 100 MiB upload made about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time, which brings that down to about 25. It does so for every source, including an *os.File, whose own WriteTo would also write 32 KiB at a time. WriteFile had the opposite problem. bytes.Reader implements io.WriterTo, so its whole payload arrived in one Write and went out as a single w. The helper reads a command's whole payload into memory before writing it, and parses its length as a PowerShell [int]. So WriteFile held the entire file in the helper's memory and failed above 2 GiB. Write now splits a larger payload into commands of at most 4 MiB. The idle timeout is unaffected. It is still measured per 32 KiB piece of payload handed to the pipe, not per command, so a larger command does not raise the transfer rate a slow link has to sustain. Refs #475 Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
Uploads to Windows hosts went through winFile.CopyFrom, which used io.Copy. Upload wraps its source in an io.TeeReader, which hides any io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB became a w command of its own: a reply to wait for before the payload, and since #480, a completion reply after it too. A 100 MiB upload made about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time, which brings that down to about 25. It does so for every source, including an *os.File, whose own WriteTo would also write 32 KiB at a time. WriteFile had the opposite problem. bytes.Reader implements io.WriterTo, so its whole payload arrived in one Write and went out as a single w. The helper reads a command's whole payload into memory before writing it, and parses its length as a PowerShell [int]. So WriteFile held the entire file in the helper's memory and failed above 2 GiB. Write now splits a larger payload into commands of at most 4 MiB. The idle timeout is unaffected. It is still measured per 32 KiB piece of payload handed to the pipe, not per command, so a larger command does not raise the transfer rate a slow link has to sustain. Refs #475 Signed-off-by: Kimmo Lehto <klehto@mirantis.com>
Uploads to Windows hosts went through winFile.CopyFrom, which used io.Copy. Upload wraps its source in an io.TeeReader, which hides any io.WriterTo, so io.Copy fell back to its 32 KiB buffer and every 32 KiB became a w command of its own: a reply to wait for before the payload, and since #480, a completion reply after it too. A 100 MiB upload made about 3,200 of those exchanges. CopyFrom now fills 4 MiB at a time, which brings that down to about 25. It does so for every source, including an *os.File, whose own WriteTo would also write 32 KiB at a time. WriteFile had the opposite problem. bytes.Reader implements io.WriterTo, so its whole payload arrived in one Write and went out as a single w. The helper reads a command's whole payload into memory before writing it, and parses its length as a PowerShell [int]. So WriteFile held the entire file in the helper's memory and failed above 2 GiB. Write now splits a larger payload into commands of at most 4 MiB. The idle timeout is unaffected. It is still measured per 32 KiB piece of payload handed to the pipe, not per command, so a larger command does not raise the transfer rate a slow link has to sustain. Refs #475 Signed-off-by: Kimmo Lehto <klehto@mirantis.com>


winFile.Seek was a round trip to the rigrcp helper, and every later r or w relied on the handle's position staying where the previous command had left it. The position and size now live on winFile, the way PosixFile keeps them, and every r and w command carries the offset it applies to. Seek no longer contacts the host. The end Seek measures from is the size at open or the furthest offset written, whichever is greater, and an empty Write returns without a round trip so it cannot move that end. The handle also remembers whether the caller opened it for reading and for writing, and refuses an operation that access does not allow before contacting the host, as PosixFile does. So an empty write to a read-only file fails too. The access checked is the one the caller asked for, not the wider one some modes need on the host.
This also takes a round trip out of every append. An O_APPEND write used to send "s 0 End" before its "w". Now it sends its w at the size winFile already knows. That size is the real end: the helper opens the file with FileShare.None, so no other writer can grow the file while the handle is open.
The helper used to acknowledge a w before it had even read the payload, and sent nothing once the write was done. A failed write, such as a full disk, surfaced as the reply to whatever command came next, or not at all if none did. With Seek now local, Write would report success and SeekEnd would measure from an end that was never written. The helper now sends a second reply once the payload is in the file, and Write waits for it before moving the position or the end. A write the helper reports as failed ends the session: a failed FileStream.Write can leave part of the payload behind, so the handle no longer knows its position or its end. A read or copy that stops partway ends it too, since the rest of the payload would otherwise be read as the reply to the next command. Each Write now waits for one more reply, which is what it costs to know the data landed.
With that, the helper's "s" command has no caller, so it goes, along with code that never did anything: HexDump, which pasted a dump of the whole short-read buffer into the error message, a FileStream.EndOfStream check on a property FileStream does not have, and assignments nothing read.
Seek stays in the remotefs.File API. This is a step towards replacing the long-lived helper with one-shot commands without losing it.
Refs #475