Skip to content

Note that the benchmark table is a sample, not a fixed result - #7

Merged
sahilkalgutkar merged 1 commit into
mainfrom
fix/benchmark-reproducibility-note
Aug 28, 2026
Merged

sahilkalgutkar merged 1 commit into
mainfrom
fix/benchmark-reproducibility-note

Conversation

@sahilkalgutkar

Copy link
Copy Markdown
Owner

A clean-clone verification run produced 172.7x on the selective range where the README records 154.6x — same code, same data, same machine. Both are honest samples of the same measurement, but presenting one without saying so invites a reader to treat it as a guarantee and then wonder what they did wrong.

The row-group counts (3 read, 59 pruned) are stable, because they are a property of the data and the predicate rather than of the hardware — so the README now points at those as the number worth reading.

Re-running it on the same machine moves the speedups by tens of percent, so
presenting them without that caveat invites a reader to treat a sample as a
guarantee. The row-group counts are stable and are the number actually worth
reading.
@sahilkalgutkar
sahilkalgutkar merged commit cfd10f7 into main Aug 28, 2026
2 checks passed
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sahilkalgutkar
sahilkalgutkar deleted the fix/benchmark-reproducibility-note branch September 9, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant