-
-
Notifications
You must be signed in to change notification settings - Fork 62
Upcoming updates to tests #1370
Copy link
Copy link
Open
Labels
platformEverything related to benchmark exection, including the PR workflow.Everything related to benchmark exection, including the PR workflow.testsAbout test design, issues with tests etc.About test design, issues with tests etc.
Description
Metadata
Metadata
Assignees
Labels
platformEverything related to benchmark exection, including the PR workflow.Everything related to benchmark exection, including the PR workflow.testsAbout test design, issues with tests etc.About test design, issues with tests etc.
We've been redesigning and adding new tests as well as adding more load generators, as we add more tests it becomes clear that the suite is growing large and some old tests can be grouped up.
json and json-tls can be unified into a single test, maybe json-comp fits into the mix too?
static and static-tls can be unified into a single test.
baseline and limited-conn can be unified also into a single test, which also fix baseline request distribution.
pipelined test to be removed entirely, clearly the results are all over the place.
upload needs to be redesigned.
Overal we are also trying to move from "all out" type of tests where the load generator pushes the target server to the absolute maximum, into more controlled throughput tests, these are more stable between runs and do not consume as much power while delivering the same information as we can simply get the cpu/req which correlates to the all out results.