Skip to content

feat(roadmap): define launch success metrics baseline + 90-day targets - #412

Merged
mrbobbytables merged 1 commit into
mainfrom
strategist/launch-metrics-100
Sep 21, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
strategist/launch-metrics-100

Conversation

@mrbobbytables

Copy link
Copy Markdown
Member

Closes #100.

The site had no defined success metrics for the KubeCon NA 2026 launch: no baseline, no targets, so an announcement post going out is unfalsifiable — there's no way to know whether it worked.

What this adds

  • data/launch-metrics.json: 5 signals, each with a live-collected baseline and a target90Day: GitHub stars, GitHub watchers, GitHub forks, new unique human contributors since baseline, and good-first-issues claimed.
  • scripts/collect-launch-metrics.mjs: fetches live data from the GitHub API and writes the file above, following the same generated: true / never-hand-edited pattern as data/metrics.json. Wired as npm run collect:launch-metrics.
  • scripts/validate-launch-metrics.mjs: validates 3-5 signals, unique ids, required provenance (sourceUrl/collectedAt), and that baseline/target90Day are finite non-negative integers with target90Day >= baseline. Wired as npm run validate:launch-metrics, added to the read-only validator suite (tests/validators-smoke.test.mjs), and covered by tests/validate-launch-metrics.test.mjs (16 tests).
  • ROADMAP.md and LAUNCH.md now point at this file as the source of truth for the launch success-metrics bullet, replacing a hand-written, already-stale table.

Why the contributor signal is a delta, not a raw count

This repo's pre-launch history already has 45 human contributors (git log/GitHub contributors API), so a raw cumulative "unique contributors" count can never sit below a small post-launch target — that was true of the number already in LAUNCH.md before this PR (baseline 1, target 5), and it doesn't hold up against the real API data. Instead, the signal is "new unique human contributors since this baseline snapshot": the file records priorContributorLogins at capture time, baseline is 0 by definition, and the 90-day re-measurement counts logins present then but absent from that list.

Why the baseline is flagged as a rehearsal capture

A previous attempt at this issue (#143) was closed as stale because it froze a months-old (2026-08-08) baseline that conflicted with the launch plan's own W-6 pre-launch-checkpoint timing (2026-09-28, per LAUNCH.md). This PR avoids repeating that: the committed data/launch-metrics.json is live data captured today, but its checkpoint field and the ROADMAP.md/LAUNCH.md prose both say explicitly to re-run npm run collect:launch-metrics at (or after) 2026-09-28 before treating the numbers as the final pre-launch baseline for the 90-day comparison.

Verification

  • npm run test:unit — 501/501 passing (16 new tests in tests/validate-launch-metrics.test.mjs)
  • node scripts/validate-launch-metrics.mjs — passes
  • node scripts/collect-launch-metrics.mjs (with GH_TOKEN) — regenerates the committed file byte-for-byte modulo timestamps
  • npm run build — succeeds
  • npx prettier --check on all touched non-markdown files — passes (the touched .md files have pre-existing prettier warnings unrelated to this change)

— hive: backend=copilot model=claude-sonnet-5

🐝 Hive Agent: contributor | SHA: 0d81477

Issue #100: the site had no defined success metrics for the KubeCon NA 2026
launch. Add data/launch-metrics.json (issue #100), a machine-collected and
validated baseline for 5 signals: GitHub stars, watchers, forks, new unique
human contributors since baseline, and good-first-issues claimed, each with
a live-collected baseline and a 90-day post-launch target.

- scripts/collect-launch-metrics.mjs fetches live data from the GitHub API
  and writes data/launch-metrics.json, following the same
  generated/never-hand-edited pattern as data/metrics.json.
- scripts/validate-launch-metrics.mjs validates signal count, provenance,
  and that baseline/target values are sane non-negative integers with
  target90Day >= baseline; wired into the read-only validator suite.
- The contributor signal is defined as a delta (new contributors since this
  baseline snapshot) rather than a raw cumulative count, since the repo's
  pre-launch history already has 45 human contributors -- a cumulative
  count can never sit below a small post-launch target.
- ROADMAP.md and LAUNCH.md now point at this file as the source of truth
  instead of a hand-written table, and flag the committed baseline as a
  rehearsal capture to be refreshed at (or after) the W-6 pre-launch
  checkpoint (2026-09-28) so the numbers reflect that checkpoint rather
  than an earlier snapshot Phase 1 work has since made stale.

Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>

@castrojo castrojo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substantive review:

  • Adds launch success metrics tracking (data/launch-metrics.json, scripts/collect-launch-metrics.mjs, scripts/validate-launch-metrics.mjs) for KubeCon NA 2026 launch.
  • Provides test coverage in tests/validate-launch-metrics.test.mjs.
  • CI is green across all checks.
  • Awaiting second/independent maintainer review before merge.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Sep 21, 2026
Merged via the queue into main with commit ec68a02 Sep 21, 2026
2 checks passed
@mrbobbytables
mrbobbytables deleted the strategist/launch-metrics-100 branch September 25, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[strategist] No launch success metrics: define baseline + targets before Phase 1 closes

2 participants