Repository navigation
ci: add a Blacksmith workflow for the server-log file backfill - #1312
Merged
Merged
Conversation
Run db:backfill-server-log-files on a Blacksmith runner by manual dispatch, for one run or a run-ID range, optionally with --force. Like the agentic recompute workflow, it targets production only from master or a named Neon child branch, and invalidates the production cache after a production run.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
db:backfill-server-log-fileshas only ever run locally. A local machine needs enough free disk for the largest extracted artifact, and the backfill needed after #1308 covers about 54 GiB of artifacts across 51 runs. This adds a manual-dispatch workflow that runs it onblacksmith-16vcpu-ubuntu-2404, followingrecompute-agentic-metrics.yml.Inputs:
run-id: one GitHub benchmark run.from-run: every DB run at or above this GitHub run ID (--all --from-run). Leaving both empty uses the script's 90-day default.force: revisit bundles already marked complete; only missing filenames are inserted.neon-branch: an existing Neon child branch. Without it the job writes to production, and only frommaster.It uses the same database selection as
recompute-agentic-metrics.yml, passesINFX_MAIN_PATsoghcan download InferenceX artifacts, and invalidates the production cache after a production run.Note
Medium Risk
Manual workflow can write production data and purge the live site cache; guards limit production runs to master and optional Neon branches.
Overview
Adds a manual GitHub Actions workflow so the existing
backfill-server-log-filesscript can run on Blacksmith (blacksmith-16vcpu-ubuntu-2404, 6h timeout) instead of only on a developer machine with enough disk for large benchmark artifacts.Dispatch inputs map to the CLI: single
run-id, rangefrom-run(mutually exclusive),forcefor incomplete bundles, and optionalneon-branchfor a Neon child DB. Without a branch, the job writes production and is allowed only frommaster; with a branch it resolves a write URL via the Neon API (child branch checks mirror the recompute workflow). The backfill step usesINFX_MAIN_PATfor GitHub artifact access and runspackages/db/src/backfill-server-log-files.tswith--yes.A follow-up
invalidatejob runs after production backfills and callsadmin:cache:invalidatefor the live site so cached data reflects new server-log file rows.Reviewed by Cursor Bugbot for commit ba0bb2c. Bugbot is set up for automated code reviews on this repo. Configure here.