Skip to content

Revert batch size - #404

Merged
Pl217 merged 1 commit into
developfrom
revert-batch-size
Jun 6, 2025
Merged

Pl217 merged 1 commit into
developfrom
revert-batch-size

Conversation

@Pl217

@Pl217 Pl217 commented Jun 6, 2025

Copy link
Copy Markdown
Contributor

Even though using the maximum theoretical limit (defined by PG_MAX_QUERY_PARAMS) works, it takes a significant performance toll. The change came after this comment, in which I suggested exploring the batch size increase.

I noticed the slowness when trying to use this query:

query {
  searchFlows(
    limit: 50
    sortField: "flow.updatedAt"
    sortOrder: "DESC"
    flowFilters: {activeStatus: true}
    flowObjectFilters: [{objectID: 4396, direction: "source", objectType: "organization"}]
    includeChildrenOfParkedFlows: true
  ) {
    total
  }
}

With batch size of 1000, this takes ~13 seconds and with maximum theoretical batch size it never completes (I let it for about 20 mins) and needed shm_size: 1g on PostgreSQL container.

Even though using the maximum theoretical limit
defined by `PG_MAX_QUERY_PARAMS` works, it takes
a significant performance toll
@Pl217
Pl217 requested a review from a team as a code owner June 6, 2025 16:23
@Pl217 Pl217 added the ready for review All comments have been addressed, and the Pull Request is ready for review label Jun 6, 2025

@unocha-hpc unocha-hpc left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks have passed and this pull request is ready for manual review

@manelcecs manelcecs added ready for merge Review and testing is complete. It is ready for merging as soon as CI has finished. and removed ready for review All comments have been addressed, and the Pull Request is ready for review labels Jun 6, 2025
@manelcecs manelcecs assigned Pl217 and unassigned manelcecs Jun 6, 2025
@Pl217
Pl217 merged commit 4bad351 into develop Jun 6, 2025
@Pl217
Pl217 deleted the revert-batch-size branch June 6, 2025 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for merge Review and testing is complete. It is ready for merging as soon as CI has finished.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants