fix: rework AWS private runner AMI build and module wiring - #24
Merged
Conversation
Packer rebuilt the AMI on every apply, because null_resource.packer_build triggered on timestamp(). That cost a full build each run and, since the AMI ID came straight from the build log, replaced the runner instance every time too. Build once per state instead. The ID is recorded in terraform_data.ami_id and read from there on later plans, so a fresh checkout or a CI runner no longer depends on packer_manifest.log being present. Set packer_config.rebuild_ami_token to any new value to build a fresh AMI; leaving it alone never rebuilds. It is a free-form string rather than a flag so unsetting it does not trigger another build. Also: - build_ami.sh now checks for the artifact line, since tee masks packer's exit status and a silent failure would otherwise be recorded as a successful build - ami_cleanup keys off the recorded ID, so destroy never deregisters an AMI this state did not build - new sg_runner.pre_release bakes the newest sg-runner pre-release into the AMI, falling back to latest stable when none is published - requires terraform >= 1.4 (opentofu >= 1.6) for terraform_data
The console URL was derived by stripping "api." out of the API host. That works for app.stackguardian.io and us.stackguardian.io but not for QA, where the API lives at testapi.qa.stackguardian.io and the console at dash.qa.stackguardian.io. The wrong host ended up in runner_group_url and, more importantly, in the S3 bucket's CORS allowed_origins, so browser uploads from the console were rejected. Map API host to console host explicitly instead. The console host is not derivable from the API host in every region, so an unlisted api_uri now fails the plan rather than silently producing a bad origin.
The module looked the runner group up itself through the stackguardian provider
data sources, which meant it could only be used against a runner group that
already existed, and it re-read the token on every plan.
Take runner_group_name and runner_group_token as inputs instead and drop the
data sources. The module can now be composed directly with runner_group in one
apply, with the token passed in memory as a sensitive value.
runner_group_token is required, so add it to the template schemas as well -
without it the no-code form offers no field for the token and the apply fails
on a missing variable. It renders as a password input and documents the
${secret::NAME} reference form, since the value should not be typed literally.
Deploying a runner meant three separate applies with outputs copied by hand between them. examples/aws/quickstart wires runner_group, packer, and single_runner into one root module: fill in the API key, org, VPC, and subnet, apply once, and the runner registers itself. The README covers the AMI-reuse behaviour, the VPC interface endpoint rule that otherwise makes jobs hang on plan with no error, and where the example stops - public subnet only, single runner, local state. Also ignore tfplan and .claude/settings.local.json. The existing pattern was *.tfplan, which does not match a plan file written as plain "tfplan", so a binary plan could be committed by accident.
joscheuerer
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes three defects in the AWS Private Runner modules — an AMI that rebuilt on every apply, a console URL derived incorrectly outside EU1/US1, and a
single_runnermodule that could not be composed withrunner_group— and adds a quickstart example that wires all three modules into a single apply.Motivation & Context
Deploying a private runner meant three separate applies with outputs copied by hand between them, and the result was not stable:
null_resource.packer_buildtriggered ontimestamp(), so every apply rebuilt the AMI from scratch and, because the AMI ID was read straight out of the build log, replaced the runner instance along with it. A no-op apply cost a full Packer build and a runner restart.Two smaller problems surfaced while building the combined example:
api.out of the API host. That happens to work forapi.app.stackguardian.ioandapi.us.stackguardian.io, but not for QA, where the API is attestapi.qa.stackguardian.ioand the console atdash.qa.stackguardian.io. The wrong host landed inrunner_group_urland, more consequentially, in the S3 bucket's CORSallowed_origins, so console-driven uploads were rejected.single_runnerlooked its runner group up through thestackguardianprovider data sources, so it could only target a group that already existed and re-read the token on every plan. It could not be composed withrunner_groupin one configuration.Changes Made
Packer AMI lifecycle (
aws/packer/)timestamp()trigger onnull_resource.packer_buildwithpacker_config.rebuild_ami_token, so the AMI is built once per state and reused thereafter. The token is a free-form string rather than a boolean, so unsetting it does not trigger another build.terraform_data.ami_idto record the built AMI ID in state. Plans no longer depend onpacker_manifest.logbeing present, which previously broke on a fresh checkout or a CI runner.data.external.packer_ami_idnow returns an empty ID instead of failing the plan when the log is missing, and resolves the log throughpath.module.build_ami.shchecks for theartifact,0,idline before succeeding.teemasks Packer's exit status, so a failed build previously recorded as successful and stuck until the token changed.null_resource.ami_cleanupkeys off the recorded ID, so a destroy never deregisters an AMI this state did not build.sg_runner.pre_releaseto bake the newest sg-runner pre-release into the AMI, falling back to the latest stable release when none is published.required_versionto>= 1.4.0forterraform_data.Runner group (
runner_group/)replace(api_uri, "api.", "")derivation with an explicit API-host-to-console-host map, used by bothrunner_group_urland the S3 CORSallowed_origins.Single runner (
aws/single_runner/)stackguardian_runner_group,stackguardian_runner_group_token, andaws_s3_bucketdata sources; the module now takesrunner_group_tokenas a sensitive input.runner_group_tokentoinput_schema.json(properties andrequired) andui_schema.jsonas a password input documenting the${secret::NAME}reference form. Without this the no-code form offered no field for a now-required variable.Quickstart example (
examples/aws/quickstart/)runner_group,packer, andsingle_runner. Four required values, one apply, and the runner registers itself.Repo hygiene
tfplanand.claude/settings.local.json. The existing pattern was*.tfplan, which does not match a plan file written as plaintfplan, so a binary plan could be committed by accident.Testing
tofu fmt -checkclean across the example and changed modulessingle_runnerschemas parse, andrequirednow matches exactly the variables that have no defaulttofu init && tofu applyinexamples/aws/quickstart/against a real VPCrebuild_ami_tokenbuilds exactly one new AMI and deregisters the superseded onetestapi.qa.stackguardian.io),runner_group_urland the S3 CORS origin both resolve todash.qa.stackguardian.io${secret::NAME}referenceRisks & Edge Cases
single_runnerconsumers.runner_group_tokenis required and has no default. Existing configurations and no-code deployments must supply it; previously the module discovered it on its own.api_uriis now constrained. The map lookup accepts only the three known platform hosts. Any other value fails the plan rather than silently producing a wrong console URL and CORS origin. This is intentional, but it is stricter than before.packer_buildtrigger switches fromtimestamp()to the token, which replaces the resource once. Expect a single AMI rebuild and one runner instance replacement, then stability.terraform_data.cleanup_amis_on_destroynow only ever touches AMIs this state built. Images previously created by repeated applies of older revisions are not adopted and will need manual cleanup —cleanup_commandsoutput has the CLI for it.Deployment Notes
single_runnerdeployments must addrunner_group_tokenbefore the next apply.