CLI, benchmark harness and a worked example

[sahilkalgutkar]

The part you can actually run.

queryforge                          start the shell
queryforge -c "SELECT 1"            run one statement
queryforge examples/tour.sql        run a script
queryforge bench --rows 500000      build a dataset and measure the optimiser

Shell

Command handling lives in shell.rs, separate from the terminal loop, so every command is tested by handing it a line and reading the string it returns — no pty, no stdin, no mocking. repl.rs is the leftover: read a line, print what comes back, and it is excluded from coverage rather than tested through a fake terminal.

\dt, \d <table> (columns plus the zone-map statistics), \import (CSV with inferred types), \save (write to .qfc and start reading from it), \attach, \explain, \timing.

Benchmark

It measures the optimiser rather than asserting it. Every query runs twice against the same data in the same process — once on the bound plan, once on the optimised one — after a warm-up pass so the comparison is not timing the file cache. The run fails outright if the two plans disagree on the row count, because a flattering speedup from a wrong answer is worse than no number at all.

On 500k rows in 8192-row row groups:

query                        rows    optimised   speedup     unopt.     row groups
----------------------------------------------------------------------------------
selective range              9999       0.48ms    154.6x    74.72ms      3 read  59 pruned
point lookup                    1       0.65ms    120.1x    78.47ms      1 read  61 pruned
narrow projection               1      18.73ms      4.5x    85.20ms     62 read   0 pruned
group by                        4      83.59ms      1.4x   119.67ms     62 read   0 pruned
filtered group by               4      41.94ms      1.9x    81.29ms     62 read   0 pruned
order by with limit            10      55.32ms      2.4x   131.88ms     62 read   0 pruned
join                         4999       1.22ms    123.6x   151.20ms      2 read  60 pruned

The three-figure numbers are all zone-map pruning; the 4.5x on a full aggregate is projection pushdown alone, reading one column chunk per row group instead of four. queryforge bench regenerates the table.

A bug the example script found

examples/tour.sql walks through the feature set on ten rows of orders. Running it surfaced a genuine defect:

SELECT c.name, o.id FROM customers c
LEFT JOIN orders o ON o.customer_id = c.id AND o.status = 'cancelled'

returned one row instead of five. The join was applying the ON clause's non-equality part after deciding which probe rows had matched, so a preserved row whose only candidate failed the condition disappeared rather than coming back NULL-padded — and so did every customer with no orders at all, since NULL = 'cancelled' is unknown.

The fix settles matching before padding: candidates are gathered, the residual condition filters them, and only then is it decided which probe rows went unmatched. Regression tests cover the left and full cases at the operator level and through SQL. The existing 43 end-to-end tests did not catch this — every join test either had no residual condition or was an inner join — which is the argument for having an example you actually run.

Also fixed: a comment line above a statement swallowed it, because the script reader buffered the comment and then read the next line as a continuation.

Verification. 39 CLI tests. Workspace line coverage is 96.8%, with the optimiser's rewriting machinery brought up to 92% by tests that call it directly.

[codecov]

Codecov Report

❌ Patch coverage is 97.47748% with 28 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/qf-cli/src/main.rs	93.04%	13 Missing ⚠️
crates/qf-cli/src/bench.rs	96.00%	8 Missing ⚠️
crates/qf-cli/src/shell.rs	98.57%	4 Missing ⚠️
crates/qf-plan/src/optimizer.rs	98.51%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!