feat(nimbus): Run population sizing hourly - #9933
Open
yashikakhurana wants to merge 2 commits into
Open
yashikakhurana wants to merge 2 commits into
yashikakhurana wants to merge 2 commits into
Conversation
Moves experiment_population_estimates_v1 to a new bqetl_experiments_hourly DAG so a newly created or edited Draft experiment gets an eligible-client estimate within the hour rather than waiting for the next daily run. The client pools stay on bqetl_experiments_daily. They cover a 7-day rolling window so they do not need rebuilding hourly, and they are dropped from referenced_tables because a cross-DAG wait on a daily task would only clear once a day and defeat the hourly schedule. Most runs are a no-op: when nothing has changed since its last sizing, needsUpdate is false everywhere and the job exits before issuing a query.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
This PR adds a bqetl_experiments_hourly DAG and moves experiment_population_estimates_v1 onto it. It also drops the referenced_tables on the daily-built nimbus_sizing_clients_v1 pools, so the hourly run doesn't wait on a cross-DAG sensor. The DAG config and the decision to drop the dependency look reasonable. The main problem is that query.py writes with WRITE_TRUNCATE, and at hourly cadence each run wipes the estimates earlier runs wrote that day. Details are inline.
Each run only sizes the experiments flagged needsUpdate at that moment, so at hourly cadence WRITE_TRUNCATE on the day partition would drop estimates written by earlier runs the same day. Every row carries computed_at, so consumers take the latest row per slug. Also updates two docstrings that still described the client pools as being populated earlier in the same DAG run.
Contributor
Integration report
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves
experiment_population_estimates_v1onto a newbqetl_experiments_hourlyDAG, so a newly created or edited Draft experiment gets an estimate within the hour instead of waiting up to a day.The client pools stay daily — they cover a 7-day rolling window, so rebuilding them hourly would be pure cost. They are dropped from
referenced_tablesbecause a cross-DAG wait on a daily task only clears once a day and would defeat the hourly schedule.Cost should stay flat: the job exits before querying when nothing has
needsUpdateset, which is most hours, andbuild_querybatches all of an app's experiments into one COUNTIF, so a run is at most 3 queries.Needs mozilla/experimenter#17485 to land first — without it, experiments whose estimate comes back unchanged stay flagged and get re-queried every run.
Jira: EXP-7366