Vectorised execution engine

[sahilkalgutkar]

The half that actually runs the query. 43 end-to-end tests go SQL text in, rows out, through the same Session the CLI uses — nothing reaches past it into a convenient shortcut.

Pull-based operators

Each operator asks its input for a batch and returns one of its own. Pulling is not a style preference: it is what lets LIMIT 10 stop a scan after the first row group instead of running it to completion and throwing the rest away, and there is a test asserting exactly that (1 read out of ten row groups). Blocking operators — sort, hash join build, aggregate — buffer internally and stream their output, so nothing above them has to know the difference.

Vectorised evaluation

Every kernel takes whole columns. The operator is resolved once, outside the loop, and the loop runs over a contiguous buffer — the row-at-a-time alternative would walk the expression tree per row and spend most of its time on dispatch. Two cases get their own paths because they dominate real queries: column-against-literal (no constant column is materialised, and a literal on the left mirrors the comparison rather than copying the data) and same-type column pairs (no boxing through Value).

Three-valued logic is implemented properly, because getting it wrong silently changes which rows come back:

• false AND NULL is false, not NULL — the row cannot match whatever the unknown turns out to be. true OR NULL is true. Both asymmetric cases have their own test.
• A predicate that evaluates to unknown does not select its row.
• 5 IN (1, NULL) is unknown, not false: the NULL might have been 5.
• A NULL join key never matches, including against another NULL.
• x / 0 yields NULL rather than aborting — one bad row should not kill a scan of a million.

LIKE is a hand-written backtracking matcher rather than a regex, iterative so a pattern of many %s cannot blow the stack.

Scan: where pushdown becomes I/O

Before reading a row group the scan asks each pushed predicate whether the group's zone map could possibly satisfy it. If not, its column chunks are never read. A conjunction prunes when either half rules the group out; a disjunction only when both do; anything the zone maps cannot decide reads the group. Tests assert the exact read/pruned split, and one asserts the answer is identical whether the table is in memory or on disk — pruning that changes an answer is a bug, not an optimisation.

Sort: three modes

• Top-k when the sort feeds a LIMIT — memory bounded by k, not by the input. A test asserts it agrees with a full sort then truncate.
• In memory when everything fits.
• External otherwise: sorted runs spill to disk and merge. A test runs the same data through both paths at a 50-row threshold and asserts identical output — the answer must not depend on whether the input fitted in memory. Spill files are removed when the merge consumes them and when a query is abandoned mid-pull, which has its own test.

Ties keep input order (stable sort), so results are reproducible run to run.

Hash join

One side is built, the other streams past it. For inner and cross joins the smaller estimate is built — the build side is what has to fit in memory. For outer joins the choice is forced: the side that may be NULL-padded has to be the one that can be scanned for non-matches at the end, so the operator overrides whatever the planner suggested. Output column order is left-then-right either way, and a test asserts an inner join gives identical results with either build side.

All five join types work: inner, left, right, full and cross.

Fixed along the way

A projection producing no columns dropped its row count, which would have made SELECT count(*) return zero after column pruning removed every column. Caught by a test written for the case, not in passing.

Verification. 105 unit tests plus 43 end-to-end tests. Workspace line coverage is 96%.

[codecov]

Codecov Report

❌ Patch coverage is 98.06317% with 65 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/qf-exec/src/eval.rs	97.26%	26 Missing ⚠️
crates/qf-exec/src/engine.rs	92.30%	15 Missing ⚠️
crates/qf-exec/src/join.rs	97.53%	10 Missing ⚠️
crates/qf-exec/src/sort.rs	98.35%	7 Missing ⚠️
crates/qf-exec/src/scan.rs	98.43%	5 Missing ⚠️
crates/qf-exec/src/physical.rs	99.32%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!