Collect and serve PostgreSQL 18's measured I/O byte totals (#2655)

[erikdarlingdata]

Closes #2655.

The defect

PostgreSQL 18 removed op_bytes from pg_stat_io. PgIoStatsCollector already guarded that — it
substitutes NULL so a collection cannot fail on "column does not exist" — but both byte figures
get_pg_io_stats serves were derived from it:

block_bytes = r.OpBytes > 0 ? r.OpBytes : (long?)null,
read_bytes  = r.OpBytes > 0 ? r.Reads * r.OpBytes : (long?)null,

So on 18 both came back null with no note, no status, and nothing to separate a version change from a
collector that had stopped.

What 18 replaced it with is better than what was lost

op_bytes was the per-operation block size, which the read multiplies by a count to estimate volume.
read_bytes / write_bytes / extend_bytes are measured totals. 18 also introduced vectored reads, so
one entry in reads can cover several blocks and the old estimate undercounts.

Measured through the running service against a real 18.6 target — not reasoned about:

combination	reads	measured bytes	old estimate	undercount
background worker/bulkread	6,862	565,723,136	56,213,504	10.1×
client backend/bulkread	4,742	448,724,992	38,846,464	11.6×
autovacuum worker/vacuum	2,797	362,037,248	22,913,024	15.8×

An order of magnitude, not a rounding difference. That is why the column was removed rather than renamed,
and why the two quantities must never share a name without saying which is which.

The change

• V101 adds read_bytes / write_bytes / extend_bytes as numeric — what PostgreSQL declares
  them, verified on 18.6, beside counts it declares bigint. Nullable, no default, no backfill.
• The collector selects them on 18+ and NULL::numeric below, appended to the SELECT so no existing
  ordinal moves. Declared Decimal(28, 0): whole bytes, and 28 rather than 38 because C# decimal tops out
  near 29 significant digits and a declared width the runtime type cannot carry is a promise broken at the
  reader.
• The reader differences them like every other cumulative counter here — a plain SUM would report the
  total since stats_reset rather than the window.
• The read serves bytes_source per row and on the envelope: measured, estimated_from_block_size,
  or unavailable, plus a bytes_note saying the two are not comparable.

byte_counters_tracked is what stops a pre-18 zero reading as a measurement — GREATEST(NULL, 0) is 0
and the outer coalesce would flatten it anyway, so without the flag a pre-18 server reports a perfectly
credible zero bytes. Same guarantee write_counters_tracked already gives Aurora's write side. It probes
read_bytes, not extend_bytes: WAL rows legitimately report no extend_bytes (verified on 18.6), so
probing that column would answer "bytes are not measured here" for a row that simply does not extend, and
the read would serve a measured server as an unmeasured one.

Verification

Two live targets at once — 17.11 and 18.6 — through the real service, not fixtures:

• the registry stamped both majors (17 and 18) via PostgreSQL major version is never persisted, so no read can explain a version-absent column #2653's column
• the store split cleanly: 7,910 rows on 17 all carrying op_bytes and no read_bytes; 79 rows on
  18 with 56 carrying read_bytes and none carrying op_bytes
• get_pg_io_stats returned estimated_from_block_size with block_bytes: 8192 for the 17 target, and
  measured with block_bytes: null for the 18 target

Worth noting the empty case is already right and stayed right: before load, the 18 target answered
no_io_activity with an accurate explanation rather than a bare empty list.

Tests

Pg18IoBytesTests pins the rung, that the columns are numeric and never bigint, that all three are
differenced rather than summed as levels, and that the tracked flag probes read_bytes specifically — that
last one is the assertion with a real failure behind it. PgIoStatsCollectorDefinitionTests now pins both
arms of the version exchange: 18 loses op_bytes and gains the three, 17 and below the reverse, same shape
either way.

[claude]

This PR adds ReadBytes / WriteBytes / ExtendBytes / ByteCountersTracked to DarlingPgIoReader.PgIoRow, but only updates these test call sites to satisfy the new positional-record constructor — it doesn't add corresponding columns to PgDisplay.IoRow or entries in FoldedFields (both in ViewerPostgresTabsTests.cs / ViewerPostgresDisplay.cs).

That means:

1. EveryFieldTheSharedReadersReturn_ReachesAColumn_OrIsFoldedWithAReason (line ~744) should now fail — it reflects over PgIoRow's public properties and requires each to appear on PgDisplay.IoRow or in FoldedFields; none of the four new properties are listed anywhere.
2. Functionally, the WPF I/O tab (PgDisplay.Io in ViewerPostgresDisplay.cs) still only sets OpSize = Bytes(row.OpBytes). On a PostgreSQL 18 target row.OpBytes is now always 0 (the collector emits NULL::bigint for it and the reader's coalesce(MAX(op_bytes), 0) turns that into 0), and Bytes(0) renders "0 B" — not "not tracked"/N-A. So the Viewer will silently show a confident-looking zero-byte I/O reading on 18, which is exactly the "zero masquerading as measurement" failure this PR's own description calls out and fixes for the MCP tool (get_pg_io_stats), but leaves unfixed in Darling's own desktop grid.

Since the MCP surface and the Viewer are meant to show the same PostgreSQL data, this looks like an incomplete port rather than a deliberate omission — worth wiring read_bytes/write_bytes/extend_bytes/bytes_source (or an equivalent) into IoRow the same way WriteCountersTracked was folded into Writes/WriteTimeMs.

[claude]

read_bytes and write_bytes both fall back to a count * block_bytes estimate when ByteCountersTracked is false (lines 122-127), but extend_bytes here has no such fallback — it's unconditionally null unless the server is PG18+. That's inconsistent with the envelope's bytes_note a few lines down (line ~172): when bytesEstimated is true it says "Byte totals are ESTIMATED as count x block_bytes, which is exact below PostgreSQL 18..." — implying all three byte figures are estimated — but extend_bytes never receives an estimate, so a caller told "estimated" will see extend_bytes: null even though r.Extends and r.OpBytes (block size) are both available to compute r.Extends * r.OpBytes, the same way write_bytes uses r.Writes * r.OpBytes.

Worth either adding the estimate for parity with read/write, or narrowing the bytes_note wording so it doesn't overpromise for extends.

[claude]

Reviewed the change against CONTRIBUTING.md's style rules (migration schema-qualification, etc.) and the Lite/Darling parity requirement.

Correctness / migration: V101 is schema-qualified (collect.pg_io_stats), nullable/no-default/no-backfill as documented, and numeric matches what PostgreSQL 18.6 actually declares. The collector's SELECT-list ordinals, PayloadColumns, WritePayload, and the reader's positional GetDecimal/GetBoolean calls all line up correctly across PgIoStatsCollector.cs and DarlingPgIoReader.cs — I traced every ordinal by hand and found no shift. MapProbedSchemaVersion's new V101 arm and the ViewerDataService probe SQL/argument count are consistent (77 sentinels, 77 GetBoolean calls).

Two issues posted inline, both worth a look before merge:

1. Darling Viewer parity gap (on ViewerPostgresTabsTests.cs) — the new PgIoRow.ReadBytes/WriteBytes/ExtendBytes/ByteCountersTracked fields aren't surfaced anywhere in the WPF I/O tab (ViewerPostgresDisplay.cs's PgDisplay.Io/IoRow), and aren't added to the FoldedFields exemption list either. This repo has a pinned reflection-based test (EveryFieldTheSharedReadersReturn_ReachesAColumn_OrIsFoldedWithAReason) specifically to catch this class of gap, and it should now fail. Functionally, it also means the desktop grid still shows a bare "0 B" for OpSize on PG18 targets — the exact "zero masquerading as a measurement" bug this PR fixes for the MCP tool (get_pg_io_stats) but leaves in place for Darling's own UI.
2. extend_bytes estimate asymmetry (on DarlingMcpPgIoTools.cs) — read_bytes/write_bytes fall back to a count * block_bytes estimate pre-18, but extend_bytes doesn't, even though Extends and OpBytes are both available. The envelope's bytes_note text says byte totals are "estimated as count x block_bytes" without carving out this exception, so it overpromises for extend_bytes.

No SQL injection/security concerns — postgresMajorVersion driving the branching in BuildQueryText comes from the internal target registry, not user input, and the interpolated fragments are fixed literals (op_bytes/NULL::bigint/etc.), not caller-controlled strings. No missing-index recommendations made, per project convention.

[claude]

write_bytes's measured branch drops the write-tracked gate. When r.ByteCountersTracked is true (PG18+), this returns r.WriteBytes unconditionally. But WriteBytes is computed in DarlingPgIoReader.PgIoSql as coalesce(SUM(d_write_bytes), 0), and per that file's own comment, GREATEST(NULL, 0) flattens to 0 for any row where the server doesn't track writes at all (e.g. Aurora, where writes/write_time_ms come back NULL). byte_counters_tracked is derived solely from read_bytes IS NOT NULL — it says nothing about whether this row's write side is tracked.

So on a PG18 target with an untracked write side, this serves write_bytes: 0 instead of null — reintroducing exactly the zero-vs-untracked ambiguity this PR exists to fix for reads. The estimated branch (r.OpBytes > 0 && r.WriteCountersTracked) already gets this right; the measured branch should be r.ByteCountersTracked && r.WriteCountersTracked ? r.WriteBytes : ....

Worth checking extend_bytes on the next line too — though there's no ExtendCountersTracked flag anywhere in this codebase to gate it with, so that one may just be inheriting a pre-existing gap rather than a new one.

[claude]

Same gap as write_bytes in DarlingMcpPgIoTools.cs: WriteVolume's measured branch (row.ByteCountersTracked) never checks row.WriteCountersTracked. On a PG18 target where the write side isn't tracked for a row (Aurora), row.WriteBytes comes back as 0 (not null) from the reader's coalesce(SUM(d_write_bytes), 0), so this cell reads "0 B" instead of "not measured"/"not tracked" — silently passing an untracked counter off as a real measurement. The estimated branch three lines down correctly ANDs row.OpBytes > 0 && row.WriteCountersTracked; the measured branch needs the same row.WriteCountersTracked check.

Separately (minor): (long)row.ReadBytes / (long)row.WriteBytes are unguarded decimal→long casts. Explicit decimal-to-integral conversions are always checked in C# regardless of context, so a windowed SUM that exceeds long.MaxValue (representable in the numeric(28,0) source column) throws OverflowException and crashes the tab render instead of degrading — Bytes() already handles its own out-of-range sentinel gracefully, this cast doesn't.

[claude]

Review summary

Reviewed PR #2656 (PG18 pg_stat_io byte totals). No T-SQL is touched by this change, so the T-SQL style rules don't apply here; the C# side follows the codebase's existing conventions well (nullable-by-design counters, ordinal-matched SELECT/reader/writer, migration ladder pinned by tests).

Lite/Darling parity: no gap. The collector (PerformanceMonitor.Collectors/PgIoStatsCollector.cs) is shared and was updated correctly (with matching Lite.Tests coverage), but Lite has no PostgreSQL UI/MCP surface at all — it's SQL-Server-only by design (see Lite/Mcp/McpEngineCapability.cs) — so there's nothing on the Lite side that needed the same read_bytes/write_bytes/extend_bytes treatment given to Darling's viewer/MCP tool.

Correctness bug found (left as inline comments): both DarlingMcpPgIoTools.cs's write_bytes and ViewerPostgresDisplay.cs's WriteVolume gate their estimated branch on WriteCountersTracked (correctly), but their measured branch (ByteCountersTracked) does not. Since byte_counters_tracked is probed only from read_bytes IS NOT NULL and says nothing about a given row's write side, a PG18 target where writes aren't tracked for a combination (Aurora) will report write_bytes: 0 / "0 B" instead of "not tracked" — the exact zero-vs-unmeasured ambiguity this PR was written to eliminate, just reintroduced on the write side. Suggested fix: r.ByteCountersTracked && r.WriteCountersTracked (and the XAML equivalent).

Also flagged a minor, low-probability latent issue: the unguarded (long)row.ReadBytes/(long)row.WriteBytes casts in ViewerPostgresDisplay.cs throw OverflowException if a windowed sum ever exceeds long.MaxValue (the source is numeric(28,0)), unlike Bytes()'s own graceful handling of its negative sentinel.

Everything else — the V101 migration (schema-qualified, nullable, no default/backfill, precision matched to collector), the schema-version probe/mapping ladder, and the extensive test coverage for the new rung — looks solid.

[claude]

Reviewed the diff (PostgreSQL 18 measured I/O bytes, V101). Overall this is careful, well-tested work — the version-gating in the collector, the differencing/tracked-flag pattern in the reader, and the schema-generator pinning all line up correctly (SELECT ordinals, reader ordinals, and PayloadColumns all agree at 21 columns; bool_or(read_bytes IS NOT NULL) rather than probing extend_bytes is the right call given WAL rows legitimately omit it).

One correctness issue flagged inline: ViewerPostgresDisplay.cs's new ReadVolume/WriteVolume mapping does (long)row.ReadBytes / (long)row.WriteBytes on a decimal sourced from a numeric(28,0) column. A decimal → long cast in C# always range-checks (unlike integer narrowing), so a value beyond long.MaxValue throws OverflowException on the UI thread. It's also the only branch (ByteCountersTracked: true, i.e. the actual PG18 "measured" case) with no test coverage in ViewerPostgresTabsTests.cs — both new assertions there pin the pre-18/estimated branch only.

Everything else — Lite/Darling parity, the MCP tool's bytes_source/bytes_note envelope, the migration (nullable, no default, no backfill, CREATE+ALTER both updated), and the CHANGELOG entry — looks correct and in line with CONTRIBUTING.md.

Minor/non-blocking: six of the modified files (PgIoStatsCollector.cs, DarlingPgIoReader.cs, DarlingMcpPgIoTools.cs, PgIoStatsCollectorDefinitionTests.cs, Pg18IoBytesTests.cs, ViewerServerTab.xaml) picked up a leading UTF-8 BOM that wasn't there before this change (confirmed by diffing raw bytes against the base commit). Looks like an editor/tool side effect rather than an intentional change — worth normalizing back out so it doesn't show up as noise in future diffs.

[claude]

(long)row.ReadBytes / (long)row.WriteBytes narrow a numeric(28,0) value into a type that tops out at ~9.2×10¹⁸ (long.MaxValue). Unlike integer casts, an explicit decimal → long conversion in C# always range-checks regardless of checked/unchecked context, so a value beyond long.MaxValue throws OverflowException here — on the WPF UI thread, while rendering a grid row.

Given the care elsewhere in this PR about not letting the store's numeric promise get broken at the reader (the whole reason 28 was chosen over 38 for the collector's decimal precision), it's worth closing the same gap here: either widen Bytes(...) to accept a decimal directly (it only does simple division/formatting, no need to narrow first), or clamp/guard the cast.

Separately: this is also the only place in the diff where ByteCountersTracked: true (the PG18 "measured" branch) would actually exercise this cast, and no test in ViewerPostgresTabsTests.cs constructs a PgIoRow with ByteCountersTracked: true — the two new tests both pin the pre-18/estimated path (ByteCountersTracked: false). The measured branch of PgDisplay.Io currently has zero coverage.

[claude]

(long)row.ReadBytes / (long)row.WriteBytes narrow a numeric(28,0) value into a type that tops out at ~9.2×10¹⁸ (long.MaxValue). Unlike integer casts, an explicit decimal → long conversion in C# always range-checks regardless of checked/unchecked context, so a value beyond long.MaxValue throws OverflowException here — on the WPF UI thread, while rendering a grid row.

Given the care elsewhere in this PR about not letting the store's numeric promise get broken at the reader (the whole reason 28 was chosen over 38 for the collector's decimal precision), it's worth closing the same gap here: either widen Bytes(...) to accept a decimal directly (it only does simple division/formatting, no need to narrow first), or clamp/guard the cast.

Separately: this is also the only place in the diff where ByteCountersTracked: true (the PG18 "measured" branch) would actually exercise this cast, and no test in ViewerPostgresTabsTests.cs constructs a PgIoRow with ByteCountersTracked: true — the two new tests both pin the pre-18/estimated path (ByteCountersTracked: false). The measured branch of PgDisplay.Io currently has zero coverage.