Skip to content

Implement adaptive chunk sizes / adaptive speed benchmark results - #2427

Open
mr-creditcard wants to merge 13 commits into
hashtopolis:masterfrom
mr-creditcard:2295-adaptivechunks
Open

Implement adaptive chunk sizes / adaptive speed benchmark results#2427
mr-creditcard wants to merge 13 commits into
hashtopolis:masterfrom
mr-creditcard:2295-adaptivechunks

Conversation

@mr-creditcard

@mr-creditcard mr-creditcard commented Aug 7, 2026

Copy link
Copy Markdown

This PR is a continuation of the work done by @0xVavaldi in #729. I have cherry-picked their commits and added my own on top in an attempt to keep attribution in place. It addresses issues #551 & #2295.

The aforementioned PR has a number of requirements set by @s3inlc in the comments which have remained unaddressed thusfar, this PR aims to implement these. The changes include:

Did some quick tests with this code and it seems to work fine. Would appreciate someone else testing this as well though!

Closes #551 & #2295.

@mr-creditcard
mr-creditcard force-pushed the 2295-adaptivechunks branch 2 times, most recently from 3c4d5fc to 0bc5c8a Compare August 11, 2026 08:38
@mr-creditcard
mr-creditcard marked this pull request as ready for review August 11, 2026 08:42

@jessevz jessevz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks really good, apart from the small syntax error phpstan picked up and the postgres syntax it looks great. I have not tested the code yet but once these things are fixed I can do a test run. Maybe @s3inlc also want to review this

@@ -0,0 +1,2 @@
INSERT INTO `Config` (`configId`, `configSectionId`, `item`, `value`) VALUES

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some of the tests are failing because of the backticks, this is mysql syntax and not postgres syntax

// chunk calculation times for small chunks can be pretty inaccurate this is a pretty safe and reasonable
// multiplier. Keep in mind the chunk duration will not be tuned down, only up. So once overshot, you're
// stuck with that time for the remainder of the task.
$differenceToChunk = ($differenceToChunk > 1.5) ? 1.5 : $differenceToChunk;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bit nitpicky of me but the magic number 1.5 would be cleaner in a variable ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Adaptive chunk sizes

3 participants