Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0b954d0
STAC-25142 Add GitHub Actions lint + unit-test workflow (agent CI mig…
LouisParkin Jul 10, 2026
d9ca269
STAC-25142 CI: run lint+unit on dedicated public runners (docker-publ…
LouisParkin Jul 22, 2026
51c513e
STAC-25142 CI: run workflow steps in bash (dash lacks set -o pipefail…
LouisParkin Jul 24, 2026
936010c
STAC-25142 CI: mark workspace safe.directory (git dubious ownership i…
LouisParkin Jul 24, 2026
adc6b70
STAC-25142 CI: full-depth checkout for unit tests so git describe fin…
LouisParkin Jul 24, 2026
c7d7a42
STAC-25142 Pass workspace dir to fix_branding.sh (CI_PROJECT_DIR is G…
LouisParkin Jul 24, 2026
c07d0c8
STAC-25142 Add GitHub Actions binary-build workflow (agent CI migrati…
LouisParkin Jul 27, 2026
ae94fa8
STAC-25429-godeps-cache-image: warm Go module cache via prebuilt cach…
LouisParkin Jul 28, 2026
7bedf74
STAC-25142 Add GitHub Actions omnibus DEB build workflow
LouisParkin Jul 28, 2026
229df15
STAC-25142 Add agent and cluster-agent container image builds
LouisParkin Jul 28, 2026
c45407d
STAC-25142 Read the PyPI index URL as a variable, not a secret
LouisParkin Jul 28, 2026
ecb9c60
STAC-25429 Push the Go dep cache to its own quay repo, not sts-ci-images
LouisParkin Jul 28, 2026
ffabcac
STAC-25429 Correct visibility comments: datadog_build base and cache …
LouisParkin Jul 28, 2026
e2e6e1c
STAC-25142 Restore the cluster-agent binary's exec bit after artifact…
LouisParkin Jul 28, 2026
7049264
STAC-25429 fix godeps cache image baking an empty module cache
LouisParkin Jul 28, 2026
4e986be
STAC-25429 keep only module zips in the godeps cache image
LouisParkin Jul 29, 2026
0ebc9f1
STAC-25459 wire inv check-mod-tidy into CI, gated on the godeps cache…
LouisParkin Jul 29, 2026
1996c90
Merge pull request #445 from StackVista/STAC-25429-godeps-cache-image
LouisParkin Jul 29, 2026
a9a92e0
Merge pull request #447 from StackVista/STAC-25459-mod-tidy-gate
LouisParkin Jul 29, 2026
a0e85fa
Merge remote-tracking branch 'origin/STAC-25142-agent-lint-unit' into…
LouisParkin Jul 29, 2026
9a2658b
STAC-25429 Build the DEB package in the warm Go module cache image
LouisParkin Jul 29, 2026
77780d0
Merge pull request #446 from StackVista/STAC-25142-agent-omnibus-deb
LouisParkin Jul 30, 2026
36d9298
STAC-25466 tidy the Go workspace manifests
LouisParkin Jul 30, 2026
272d975
STAC-25461 report release-branch build failures to Slack via Cerberus
LouisParkin Jul 30, 2026
b048ba9
STAC-25494 stagger godeps-cache builds instead of gating on concurrency
LouisParkin Jul 30, 2026
46a089e
STAC-25461 read SLACK_CI_REPORT_CHANNEL as a secret, not a variable
LouisParkin Jul 30, 2026
279f418
Merge pull request #450 from StackVista/STAC-25494-godeps-cache-no-ca…
LouisParkin Jul 30, 2026
5980757
STAC-25461 point the Cerberus prerequisite at the repo-level secrets PR
LouisParkin Jul 30, 2026
73f7ee7
Merge pull request #448 from StackVista/STAC-25466-workspace-tidy
LouisParkin Jul 31, 2026
72c17a1
STAC-25461 make Cerberus secrets optional so the missing-secret path …
LouisParkin Jul 31, 2026
2c64bf5
STAC-25498 Add the arm64 build lane to the GitHub Actions pipeline
LouisParkin Jul 31, 2026
1fe5bf3
Merge pull request #449 from StackVista/STAC-25461-cerberus-notify
LouisParkin Jul 31, 2026
e782f72
STAC-25498 Merge integration branch (Cerberus notify, push triggers) …
LouisParkin Jul 31, 2026
41f575b
STAC-25498 point cerberus-notify at the per-arch godeps-cache jobs
LouisParkin Jul 31, 2026
f32f73a
Merge pull request #451 from StackVista/STAC-25498-arm64-lane
LouisParkin Jul 31, 2026
de015d5
STAC-25461 Drop SLACK_CI_REPORT_CHANNEL from the Cerberus notify path
LouisParkin Jul 31, 2026
2678b05
Merge pull request #452 from StackVista/STAC-25461-drop-slack-channel
LouisParkin Jul 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/docker/godeps-cache/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Go dependency cache image for stackstate-agent CI (STAC-25429). Derives FROM the
# datadog_build CI image and bakes the workspace's external Go module
# downloads into GOMODCACHE, so consumer jobs get a warm module cache with no
# per-job `go mod download`/network round-trips. Rebuilt only when the module graph
# hash changes -- the image tag is content-addressed
# (see .github/scripts/agent-godeps-cache-metadata.sh).
#
# Only the module ZIPS ($GOMODCACHE/cache/download) are kept; the extracted module
# trees are deleted before the layer closes and Go re-extracts on demand. Measured on
# a full cache, the extracted trees are ~79% of the bytes but ~97% of the *files*, and
# image pull cost here is bound by filesystem metadata, not transfer: warm-proxy pulls
# of the full-cache image spent 0.7m downloading and 4.9m extracting. Shipping ~400k
# tiny files through overlayfs cost more than it saved (+3.0m pull against ~1.45m of
# avoided in-build downloading, so ~1.5m/job net loss).
#
# No `# syntax=` frontend directive and no BuildKit secret mounts: the base image is
# pulled by the daemon after `docker login` (creds stay in the daemon, not the
# build), so the build itself needs no secrets and no Docker Hub frontend pull.
ARG BASE_IMAGE
FROM ${BASE_IMAGE}

# `read -d ''` and `pipefail` are bashisms and RUN defaults to /bin/sh (dash), where
# the download loop silently iterates zero times and publishes an empty cache.
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# Pre-download external modules for every module in the workspace, mirroring what
# `inv deps` does per module (tasks/libs/common/go.py) at the same parallelism.
# manifests/ preserves the repo's directory layout so nested go.mod files (and their
# local `replace ../` targets) resolve. No GOFLAGS override: go.work puts this in
# workspace mode, where anything other than -mod=readonly is rejected outright. Only
# the zip cache is kept -- the manifests and extracted trees are removed in the same
# RUN, so the layer diff only ever contains the final state.
#
# The prune needs `chmod -R u+w` first: Go writes the module cache read-only (dirs
# 0555, files 0444) and `rm -rf` cannot descend into it as a non-root user.
# cache/vcs holds full git clones for any module fetched outside the proxy; it is not
# needed to rebuild a module from its zip, so it goes too.
COPY manifests/ /workspace/
RUN set -eux; \
git config --global --add safe.directory '*'; \
export PATH="${PATH}:/usr/local/go/bin"; \
cd /workspace; \
modules="$(find . -type f -name go.mod | wc -l)"; \
echo "workspace modules: ${modules}"; \
test "${modules}" -gt 0; \
find . -type f -name go.mod -print0 \
| xargs -0 -n1 -P8 bash -c \
'moddir="$(dirname "$1")"; echo "go mod download :: ${moddir}"; cd "${moddir}" && go mod download' _; \
cache="$(go env GOMODCACHE)"; \
echo "== populated module cache =="; \
du -sh "${cache}"; \
echo "files: $(find "${cache}" -type f | wc -l)"; \
echo "== zip cache =="; \
du -sh "${cache}/cache/download"; \
echo "files: $(find "${cache}/cache/download" -type f | wc -l)"; \
test "$(du -sm "${cache}/cache/download" | cut -f1)" -gt 100; \
find "${cache}" -mindepth 1 -maxdepth 1 ! -name cache -exec chmod -R u+w {} +; \
find "${cache}" -mindepth 1 -maxdepth 1 ! -name cache -exec rm -rf {} +; \
if [ -d "${cache}/cache/vcs" ]; then chmod -R u+w "${cache}/cache/vcs"; rm -rf "${cache}/cache/vcs"; fi; \
echo "== retained in image =="; \
du -sh "${cache}"; \
echo "files: $(find "${cache}" -type f | wc -l)"; \
rm -rf /workspace
53 changes: 53 additions & 0 deletions .github/scripts/agent-godeps-cache-metadata.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
# Computes the content-addressed tag for the stackstate-agent Go dependency cache
# image (STAC-25429) and the two image refs it is referenced by:
# * push ref -> quay.io/stackstate/stackstate-agent-godeps-cache (authoritative)
# * pull ref -> ${REGISTRY_HOST}/quay/... (registry.tooling proxy)
#
# The cache lives in its own quay repo rather than the shared sts-ci-images, which also
# holds the ARC runner images: stackstate-agent is PUBLIC, so whatever credential this
# workflow carries is reachable from any org member's same-repo branch. A dedicated repo
# keeps that reach down to a rebuildable cache.
#
# Mirrors StackVista/StackGraph .github/scripts/stackgraph-ci-metadata.sh: the tag is
# a hash of the dependency-defining inputs, so an unchanged module graph reuses the
# same image across every branch/PR/default build (the tag *is* the cache key), while
# any change to the graph rotates the tag and a stale cache is never reused.

agent_godeps_compute_metadata() {
: "${QUAY_REGISTRY:?QUAY_REGISTRY is required}" # quay.io
: "${REGISTRY_HOST:?REGISTRY_HOST is required}" # registry.tooling.stackstate.io (quay proxy)
: "${BASE_IMAGE_TAG:?BASE_IMAGE_TAG is required}" # datadog_build tag the cache derives FROM
: "${BASE_IMAGE_NAME:?BASE_IMAGE_NAME is required}" # datadog_build_linux_x64 | datadog_build_linux_arm64
: "${ARCH:?ARCH is required}" # amd64 | arm64

# Hash inputs: every module manifest (go.work + go.work.sum + modules.yml + all
# nested go.mod/go.sum) plus the base image and the two files that define the
# cache mechanism itself (Dockerfile + this script), so changing how the cache is
# built also rotates the tag.
#
# The base image NAME is hashed, not just its tag: the amd64 and arm64
# datadog_build images share tag 7af9194f, so hashing the tag alone would give
# both arches the same cache tag and let one arch's image satisfy the other's
# existence check. ARCH is in the tag as well, so the collision is impossible
# by construction and the arch is readable off the ref.
local godeps_hash
godeps_hash="$(
{
git ls-files -z -- \
'*go.mod' '*go.sum' 'go.work' 'go.work.sum' 'modules.yml' \
'.github/docker/godeps-cache/Dockerfile' \
'.github/scripts/agent-godeps-cache-metadata.sh' \
| LC_ALL=C sort -z \
| xargs -0 sha256sum
printf 'base:%s:%s\n' "${BASE_IMAGE_NAME}" "${BASE_IMAGE_TAG}"
} | sha256sum | cut -c1-16
)"

local image_repo="stackstate/stackstate-agent-godeps-cache"
local image_tag="godeps-${ARCH}-${godeps_hash}"
# shellcheck disable=SC2034 # consumed by the sourcing workflow step
ci_image_push="${QUAY_REGISTRY}/${image_repo}:${image_tag}"
# shellcheck disable=SC2034 # consumed by the sourcing workflow step
ci_image="${REGISTRY_HOST}/quay/${image_repo}:${image_tag}"
}
Loading
Loading