feat(roadmap): define launch success metrics baseline + 90-day targets

CONTRIBUTING.md

@@ -46,6 +46,10 @@ Rules:
   `GH_TOKEN` to a GitHub personal access token with public-repo read access
   before running it (and `npm run fetch:community-people`) — these scripts
   make enough GitHub API calls to exceed the unauthenticated rate limit.
+- **Never edit `data/launch-metrics.json` by hand.** It is the machine-checked
+  launch success-metrics baseline (issue #100, `LAUNCH.md`). Refresh it with
+  `npm run collect:launch-metrics` (needs `GH_TOKEN`) and validate with
+  `npm run validate:launch-metrics`.
 - Verify award entries against the linked cncf.io announcement before adding.
 - Reference architectures are imported from
   [cncf/architecture](https://github.com/cncf/architecture) — fix content

LAUNCH.md

@@ -45,18 +45,26 @@ week-by-week status — it uses explicit dates and does not go stale.
 | W-1 | 2026-11-02 | Final content refresh; all Phase 1 issues closed or explicitly deferred |
 | W-0 | 2026-11-09 | **Launch during KubeCon NA week**: publish launch post, amplify on agreed channels, open feedback issue template |
 
-## Success metrics (baseline + 30-day post-launch targets)
-
-Baselines are as of 2026-08-08; capture the W-6 baseline snapshot in #100.
-
-| Signal | Baseline (2026-08-08) | 30-day post-launch target |
-| -------- | ---------------------- | --------------------------- |
-| GitHub stars | 0 | 50 |
-| Watchers / forks | 0 / 1 | 10 / 5 |
-| Unique contributors (humans, non-bot) | 1 | 5 |
-| Open good-first-issues claimed | 0 labeled | 3 claimed |
-| Blog posts since launch | 0 | 2 (launch post + 1 community voice) |
-| Referral from cncf.io properties | none | cross-link merged on at least one CNCF property (Phase 3) |
+## Success metrics (baseline + post-launch targets)
+
+Five of the signals below are machine-collected in `data/launch-metrics.json`
+(issue #100) rather than hand-tracked here, so they can't go stale the way a
+one-time table does: refresh with `npm run collect:launch-metrics`, validate
+with `npm run validate:launch-metrics`. Re-run the collector at (or after) the
+W-6 checkpoint (2026-09-28) so the committed baseline reflects that
+checkpoint — see the file's own `checkpoint` field for the current status.
+The two signals without a GitHub API source (blog cadence, cncf.io referral)
+are tracked here as prose and checked manually.
+
+| Signal | Baseline | 90-day post-launch target | Source |
+| -------- | ---------------------- | --------------------------- | ------ |
+| GitHub stars | see `data/launch-metrics.json` | 50 | `data/launch-metrics.json` |
+| GitHub watchers | see `data/launch-metrics.json` | 10 | `data/launch-metrics.json` |
+| GitHub forks | see `data/launch-metrics.json` | 5 | `data/launch-metrics.json` |
+| New unique human contributors since baseline | 0 (by definition) | 5 | `data/launch-metrics.json` |
+| Good-first-issues claimed | see `data/launch-metrics.json` | 3 | `data/launch-metrics.json` |
+| Blog posts since launch | 0 | 2 (launch post + 1 community voice) | manual (blog/) |
+| Referral from cncf.io properties | none | cross-link merged on at least one CNCF property (Phase 3) | manual |
 
 ## Launch-readiness gate: zero-red content pipelines (#136)
 

ROADMAP.md

@@ -61,16 +61,22 @@ shipped silently. Content completeness alone is not a launch plan.
 - [ ] **Announcement channels**: a cncf.io blog post, a mention at an End User
       TAB meeting, and a KubeCon + CloudNativeCon mention or session are the
       minimum bar; amplify further via the End User community Slack.
-- [ ] **Baseline metrics snapshot**: capture stars, watchers, forks, and unique
-      human contributors immediately before announcing, so launch impact is
-      measurable rather than assumed. As of 2026-08-08 the baseline is 0 stars,
-      0 watchers, 1 fork.
+- [ ] **Launch success metrics** (issue #100): `data/launch-metrics.json`
+      defines the falsifiable signals — GitHub stars, watchers, forks, new
+      unique human contributors since baseline, and good-first-issues claimed
+      — each with a live-collected baseline and a 90-day post-launch target.
+      Refresh with `npm run collect:launch-metrics`, validate with
+      `npm run validate:launch-metrics`. The committed file is a rehearsal
+      capture; re-run it at (or after) the W-6 pre-launch checkpoint
+      (2026-09-28, see `LAUNCH.md`) so the baseline reflects that checkpoint
+      rather than an earlier snapshot that Phase 1 work has since made stale.
 - [ ] **DNS cutover dependency**: cutover to `endusers.cncf.io` is gated on the
       Phase 2 ownership decision (issue #46, ADR 0001). If undecided by launch,
       the announcement links the current GitHub Pages URL instead of slipping
       the date.
-- [ ] **30-day check-in**: compare stars/watchers/forks/contributors against the
-      baseline snapshot and record whether the launch moved the needle.
+- [ ] **90-day check-in**: re-run `npm run collect:launch-metrics` 90 days
+      after launch and compare each signal against its `data/launch-metrics.json`
+      baseline to record whether the launch moved the needle.
 
 Related: #90 (this finding), #46 (ownership), #75 (events pillar), #76 (blog
 cadence).

data/launch-metrics.json

@@ -0,0 +1,100 @@
+{
+  "generated": true,
+  "generatedAt": "2026-09-21T18:59:10.436Z",
+  "capturedAt": "2026-09-21T18:59:10.436Z",
+  "checkpoint": {
+    "label": "W-6 pre-launch checkpoint (LAUNCH.md)",
+    "targetDate": "2026-09-28",
+    "note": "Re-run npm run collect:launch-metrics on or after targetDate and commit the refreshed file before this baseline is treated as final for the launch retrospective."
+  },
+  "source": "https://api.github.com/repos/cncf/endusers",
+  "signals": [
+    {
+      "id": "github-stars",
+      "label": "GitHub stars",
+      "baseline": 0,
+      "target90Day": 50,
+      "sourceUrl": "https://github.com/cncf/endusers/stargazers",
+      "collectedAt": "2026-09-21T18:59:10.436Z"
+    },
+    {
+      "id": "github-watchers",
+      "label": "GitHub watchers",
+      "baseline": 0,
+      "target90Day": 10,
+      "sourceUrl": "https://github.com/cncf/endusers/watchers",
+      "collectedAt": "2026-09-21T18:59:10.436Z"
+    },
+    {
+      "id": "github-forks",
+      "label": "GitHub forks",
+      "baseline": 2,
+      "target90Day": 5,
+      "sourceUrl": "https://github.com/cncf/endusers/forks",
+      "collectedAt": "2026-09-21T18:59:10.436Z"
+    },
+    {
+      "id": "new-contributors-since-baseline",
+      "label": "New unique human contributors since baseline",
+      "baseline": 0,
+      "target90Day": 5,
+      "sourceUrl": "https://github.com/cncf/endusers/graphs/contributors",
+      "collectedAt": "2026-09-21T18:59:10.436Z",
+      "note": "Baseline is 0 by definition (a delta metric). priorContributorLogins is the snapshot of existing human contributor logins at capture time; the 90-day re-measurement counts human contributor logins present then but absent from this list.",
+      "priorContributorLogins": [
+        "AmyJuneH",
+        "AudMonte01",
+        "CathPag",
+        "Joibel",
+        "Kavish-12345",
+        "MMerzinger",
+        "Mohiit70",
+        "Sriram27102003",
+        "TheFoxAtWork",
+        "aliok",
+        "carolynvs",
+        "castrojo",
+        "charlie0129",
+        "cjyabraham",
+        "denisborisov",
+        "devstein",
+        "esthernnolum",
+        "garethx",
+        "geekygirldawn",
+        "idvoretskyi",
+        "iiamabby",
+        "ivanfetch",
+        "jaytiaki",
+        "jberkus",
+        "jeefy",
+        "justaugustus",
+        "k8tgreenley",
+        "krol3",
+        "krook",
+        "mhall119",
+        "mrbobbytables",
+        "nate-double-u",
+        "ragashreeshekar",
+        "riaankleinhans",
+        "richochetclementine1315",
+        "sandeepkanabar",
+        "sestegra",
+        "sharlalane",
+        "staceypotter",
+        "swinslow",
+        "thedevelopnik",
+        "thetwopct",
+        "thisisobate",
+        "xmulligan"
+      ]
+    },
+    {
+      "id": "good-first-issues-claimed",
+      "label": "Good-first-issues claimed",
+      "baseline": 0,
+      "target90Day": 3,
+      "sourceUrl": "https://github.com/cncf/endusers/issues?q=is%3Aissue+label%3A%22good+first+issue%22",
+      "collectedAt": "2026-09-21T18:59:10.436Z"
+    }
+  ]
+}

package.json

@@ -52,6 +52,8 @@
     "update:pkgs": "npx npm-check-updates -u",
     "collect:metrics": "node scripts/collect-metrics.mjs",
     "validate:metrics": "node scripts/validate-metrics.mjs",
+    "collect:launch-metrics": "node scripts/collect-launch-metrics.mjs",
+    "validate:launch-metrics": "node scripts/validate-launch-metrics.mjs",
     "collect:case-studies": "node scripts/collect-case-studies.mjs",
     "validate:case-studies": "node scripts/validate-case-studies.mjs",
     "collect:radar-reports": "node scripts/collect-radar-reports.mjs",

scripts/collect-launch-metrics.mjs

@@ -0,0 +1,153 @@
+#!/usr/bin/env node
+// Collects the launch success-metrics baseline defined by issue #100: a small
+// set of signals (GitHub stars, watchers, forks, unique human contributors,
+// good-first-issue conversion) that turn "launch" into something falsifiable.
+//
+// The 90-day post-launch targets are policy decisions, not measured data, so
+// they live in TARGET_90_DAY below rather than being fetched. Everything else
+// is live GitHub API data, refreshed by re-running this script — the same
+// generated/never-hand-edited pattern as data/metrics.json.
+//
+// Per LAUNCH.md, the pre-launch checkpoint that should be treated as the
+// final launch baseline lands at W-6 (2026-09-28). Re-run this script at (or
+// after) that date and record capturedAt so the baseline reflects the actual
+// checkpoint rather than whatever day someone happened to run it.
+import { writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+import { githubFetch } from './lib/github.mjs';
+
+const root = new URL('..', import.meta.url).pathname;
+const REPO = 'cncf/endusers';
+const USER_AGENT = 'cncf-endusers-launch-metrics';
+
+const TARGET_90_DAY = {
+  'github-stars': 50,
+  'github-watchers': 10,
+  'github-forks': 5,
+  'new-contributors-since-baseline': 5,
+  'good-first-issues-claimed': 3,
+};
+
+async function main() {
+  const token = process.env.GH_TOKEN;
+  const [repo, contributors, goodFirstIssues] = await Promise.all([
+    githubFetch(`https://api.github.com/repos/${REPO}`, token, USER_AGENT),
+    githubAll(
+      `https://api.github.com/repos/${REPO}/contributors?per_page=100&anon=false`,
+      token,
+    ),
+    githubAll(
+      `https://api.github.com/repos/${REPO}/issues?state=all&labels=good+first+issue&per_page=100`,
+      token,
+    ),
+  ]);
+
+  const capturedAt = new Date().toISOString();
+  // The repository's commit history long predates this launch plan (200+
+  // merged PRs from dozens of human contributors already), so a raw
+  // cumulative contributor count is never a small number and can never sit
+  // below a "5 by launch+90d" target. The falsifiable signal is *new*
+  // contributors who show up after this baseline is captured, which is 0 by
+  // definition at capture time. priorContributorLogins is the snapshot a
+  // future run diffs against to count that delta.
+  const humanContributors = contributors.filter(
+    (c) => c.type !== 'Bot' && !/\[bot\]$|Bot$/i.test(c.login),
+  );
+  const claimedGoodFirstIssues = goodFirstIssues.filter(
+    (issue) => !issue.pull_request && (issue.assignees || []).length > 0,
+  );
+
+  const data = {
+    generated: true,
+    generatedAt: capturedAt,
+    capturedAt,
+    checkpoint: {
+      label: 'W-6 pre-launch checkpoint (LAUNCH.md)',
+      targetDate: '2026-09-28',
+      note: 'Re-run npm run collect:launch-metrics on or after targetDate and commit the refreshed file before this baseline is treated as final for the launch retrospective.',
+    },
+    source: `https://api.github.com/repos/${REPO}`,
+    signals: [
+      signal(
+        'github-stars',
+        'GitHub stars',
+        repo.stargazers_count,
+        `https://github.com/${REPO}/stargazers`,
+        capturedAt,
+      ),
+      signal(
+        'github-watchers',
+        'GitHub watchers',
+        repo.subscribers_count,
+        `https://github.com/${REPO}/watchers`,
+        capturedAt,
+      ),
+      signal(
+        'github-forks',
+        'GitHub forks',
+        repo.forks_count,
+        `https://github.com/${REPO}/forks`,
+        capturedAt,
+      ),
+      {
+        ...signal(
+          'new-contributors-since-baseline',
+          'New unique human contributors since baseline',
+          0,
+          `https://github.com/${REPO}/graphs/contributors`,
+          capturedAt,
+        ),
+        note: 'Baseline is 0 by definition (a delta metric). priorContributorLogins is the snapshot of existing human contributor logins at capture time; the 90-day re-measurement counts human contributor logins present then but absent from this list.',
+        priorContributorLogins: humanContributors.map((c) => c.login).sort(),
+      },
+      signal(
+        'good-first-issues-claimed',
+        'Good-first-issues claimed',
+        claimedGoodFirstIssues.length,
+        `https://github.com/${REPO}/issues?q=is%3Aissue+label%3A%22good+first+issue%22`,
+        capturedAt,
+      ),
+    ],
+  };
+
+  writeFileSync(
+    join(root, 'data/launch-metrics.json'),
+    JSON.stringify(data, null, 2) + '\n',
+  );
+  console.log(
+    `Collected ${data.signals.length} launch metrics at ${capturedAt}`,
+  );
+}
+
+function signal(id, label, baseline, sourceUrl, collectedAt) {
+  return {
+    id,
+    label,
+    baseline,
+    target90Day: TARGET_90_DAY[id],
+    sourceUrl,
+    collectedAt,
+  };
+}
+
+async function githubAll(url, token) {
+  const results = [];
+  let next = url;
+  while (next) {
+    const response = await fetch(next, {
+      headers: {
+        Accept: 'application/vnd.github+json',
+        'User-Agent': USER_AGENT,
+        ...(token ? { Authorization: `Bearer ${token}` } : {}),
+      },
+    });
+    if (!response.ok) throw new Error(`GitHub API ${response.status}: ${next}`);
+    results.push(...(await response.json()));
+    const link = response.headers.get('link') || '';
+    const match = link.match(/<([^>]+)>;\s*rel="next"/);
+    next = match ? match[1] : null;
+  }
+  return results;
+}
+
+main();