diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 73f65fd2..a6675cbe 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -46,6 +46,10 @@ Rules: `GH_TOKEN` to a GitHub personal access token with public-repo read access before running it (and `npm run fetch:community-people`) — these scripts make enough GitHub API calls to exceed the unauthenticated rate limit. +- **Never edit `data/launch-metrics.json` by hand.** It is the machine-checked + launch success-metrics baseline (issue #100, `LAUNCH.md`). Refresh it with + `npm run collect:launch-metrics` (needs `GH_TOKEN`) and validate with + `npm run validate:launch-metrics`. - Verify award entries against the linked cncf.io announcement before adding. - Reference architectures are imported from [cncf/architecture](https://github.com/cncf/architecture) — fix content diff --git a/LAUNCH.md b/LAUNCH.md index 7c6292a3..213c3135 100644 --- a/LAUNCH.md +++ b/LAUNCH.md @@ -45,18 +45,26 @@ week-by-week status — it uses explicit dates and does not go stale. | W-1 | 2026-11-02 | Final content refresh; all Phase 1 issues closed or explicitly deferred | | W-0 | 2026-11-09 | **Launch during KubeCon NA week**: publish launch post, amplify on agreed channels, open feedback issue template | -## Success metrics (baseline + 30-day post-launch targets) - -Baselines are as of 2026-08-08; capture the W-6 baseline snapshot in #100. - -| Signal | Baseline (2026-08-08) | 30-day post-launch target | -| -------- | ---------------------- | --------------------------- | -| GitHub stars | 0 | 50 | -| Watchers / forks | 0 / 1 | 10 / 5 | -| Unique contributors (humans, non-bot) | 1 | 5 | -| Open good-first-issues claimed | 0 labeled | 3 claimed | -| Blog posts since launch | 0 | 2 (launch post + 1 community voice) | -| Referral from cncf.io properties | none | cross-link merged on at least one CNCF property (Phase 3) | +## Success metrics (baseline + post-launch targets) + +Five of the signals below are machine-collected in `data/launch-metrics.json` +(issue #100) rather than hand-tracked here, so they can't go stale the way a +one-time table does: refresh with `npm run collect:launch-metrics`, validate +with `npm run validate:launch-metrics`. Re-run the collector at (or after) the +W-6 checkpoint (2026-09-28) so the committed baseline reflects that +checkpoint — see the file's own `checkpoint` field for the current status. +The two signals without a GitHub API source (blog cadence, cncf.io referral) +are tracked here as prose and checked manually. + +| Signal | Baseline | 90-day post-launch target | Source | +| -------- | ---------------------- | --------------------------- | ------ | +| GitHub stars | see `data/launch-metrics.json` | 50 | `data/launch-metrics.json` | +| GitHub watchers | see `data/launch-metrics.json` | 10 | `data/launch-metrics.json` | +| GitHub forks | see `data/launch-metrics.json` | 5 | `data/launch-metrics.json` | +| New unique human contributors since baseline | 0 (by definition) | 5 | `data/launch-metrics.json` | +| Good-first-issues claimed | see `data/launch-metrics.json` | 3 | `data/launch-metrics.json` | +| Blog posts since launch | 0 | 2 (launch post + 1 community voice) | manual (blog/) | +| Referral from cncf.io properties | none | cross-link merged on at least one CNCF property (Phase 3) | manual | ## Launch-readiness gate: zero-red content pipelines (#136) diff --git a/ROADMAP.md b/ROADMAP.md index 4791dd59..461cec62 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -61,16 +61,22 @@ shipped silently. Content completeness alone is not a launch plan. - [ ] **Announcement channels**: a cncf.io blog post, a mention at an End User TAB meeting, and a KubeCon + CloudNativeCon mention or session are the minimum bar; amplify further via the End User community Slack. -- [ ] **Baseline metrics snapshot**: capture stars, watchers, forks, and unique - human contributors immediately before announcing, so launch impact is - measurable rather than assumed. As of 2026-08-08 the baseline is 0 stars, - 0 watchers, 1 fork. +- [ ] **Launch success metrics** (issue #100): `data/launch-metrics.json` + defines the falsifiable signals — GitHub stars, watchers, forks, new + unique human contributors since baseline, and good-first-issues claimed + — each with a live-collected baseline and a 90-day post-launch target. + Refresh with `npm run collect:launch-metrics`, validate with + `npm run validate:launch-metrics`. The committed file is a rehearsal + capture; re-run it at (or after) the W-6 pre-launch checkpoint + (2026-09-28, see `LAUNCH.md`) so the baseline reflects that checkpoint + rather than an earlier snapshot that Phase 1 work has since made stale. - [ ] **DNS cutover dependency**: cutover to `endusers.cncf.io` is gated on the Phase 2 ownership decision (issue #46, ADR 0001). If undecided by launch, the announcement links the current GitHub Pages URL instead of slipping the date. -- [ ] **30-day check-in**: compare stars/watchers/forks/contributors against the - baseline snapshot and record whether the launch moved the needle. +- [ ] **90-day check-in**: re-run `npm run collect:launch-metrics` 90 days + after launch and compare each signal against its `data/launch-metrics.json` + baseline to record whether the launch moved the needle. Related: #90 (this finding), #46 (ownership), #75 (events pillar), #76 (blog cadence). diff --git a/data/launch-metrics.json b/data/launch-metrics.json new file mode 100644 index 00000000..7873aae1 --- /dev/null +++ b/data/launch-metrics.json @@ -0,0 +1,100 @@ +{ + "generated": true, + "generatedAt": "2026-09-21T18:59:10.436Z", + "capturedAt": "2026-09-21T18:59:10.436Z", + "checkpoint": { + "label": "W-6 pre-launch checkpoint (LAUNCH.md)", + "targetDate": "2026-09-28", + "note": "Re-run npm run collect:launch-metrics on or after targetDate and commit the refreshed file before this baseline is treated as final for the launch retrospective." + }, + "source": "https://api.github.com/repos/cncf/endusers", + "signals": [ + { + "id": "github-stars", + "label": "GitHub stars", + "baseline": 0, + "target90Day": 50, + "sourceUrl": "https://github.com/cncf/endusers/stargazers", + "collectedAt": "2026-09-21T18:59:10.436Z" + }, + { + "id": "github-watchers", + "label": "GitHub watchers", + "baseline": 0, + "target90Day": 10, + "sourceUrl": "https://github.com/cncf/endusers/watchers", + "collectedAt": "2026-09-21T18:59:10.436Z" + }, + { + "id": "github-forks", + "label": "GitHub forks", + "baseline": 2, + "target90Day": 5, + "sourceUrl": "https://github.com/cncf/endusers/forks", + "collectedAt": "2026-09-21T18:59:10.436Z" + }, + { + "id": "new-contributors-since-baseline", + "label": "New unique human contributors since baseline", + "baseline": 0, + "target90Day": 5, + "sourceUrl": "https://github.com/cncf/endusers/graphs/contributors", + "collectedAt": "2026-09-21T18:59:10.436Z", + "note": "Baseline is 0 by definition (a delta metric). priorContributorLogins is the snapshot of existing human contributor logins at capture time; the 90-day re-measurement counts human contributor logins present then but absent from this list.", + "priorContributorLogins": [ + "AmyJuneH", + "AudMonte01", + "CathPag", + "Joibel", + "Kavish-12345", + "MMerzinger", + "Mohiit70", + "Sriram27102003", + "TheFoxAtWork", + "aliok", + "carolynvs", + "castrojo", + "charlie0129", + "cjyabraham", + "denisborisov", + "devstein", + "esthernnolum", + "garethx", + "geekygirldawn", + "idvoretskyi", + "iiamabby", + "ivanfetch", + "jaytiaki", + "jberkus", + "jeefy", + "justaugustus", + "k8tgreenley", + "krol3", + "krook", + "mhall119", + "mrbobbytables", + "nate-double-u", + "ragashreeshekar", + "riaankleinhans", + "richochetclementine1315", + "sandeepkanabar", + "sestegra", + "sharlalane", + "staceypotter", + "swinslow", + "thedevelopnik", + "thetwopct", + "thisisobate", + "xmulligan" + ] + }, + { + "id": "good-first-issues-claimed", + "label": "Good-first-issues claimed", + "baseline": 0, + "target90Day": 3, + "sourceUrl": "https://github.com/cncf/endusers/issues?q=is%3Aissue+label%3A%22good+first+issue%22", + "collectedAt": "2026-09-21T18:59:10.436Z" + } + ] +} diff --git a/package.json b/package.json index 1c1f6819..02e9d12d 100644 --- a/package.json +++ b/package.json @@ -52,6 +52,8 @@ "update:pkgs": "npx npm-check-updates -u", "collect:metrics": "node scripts/collect-metrics.mjs", "validate:metrics": "node scripts/validate-metrics.mjs", + "collect:launch-metrics": "node scripts/collect-launch-metrics.mjs", + "validate:launch-metrics": "node scripts/validate-launch-metrics.mjs", "collect:case-studies": "node scripts/collect-case-studies.mjs", "validate:case-studies": "node scripts/validate-case-studies.mjs", "collect:radar-reports": "node scripts/collect-radar-reports.mjs", diff --git a/scripts/collect-launch-metrics.mjs b/scripts/collect-launch-metrics.mjs new file mode 100644 index 00000000..e5627096 --- /dev/null +++ b/scripts/collect-launch-metrics.mjs @@ -0,0 +1,153 @@ +#!/usr/bin/env node +// Collects the launch success-metrics baseline defined by issue #100: a small +// set of signals (GitHub stars, watchers, forks, unique human contributors, +// good-first-issue conversion) that turn "launch" into something falsifiable. +// +// The 90-day post-launch targets are policy decisions, not measured data, so +// they live in TARGET_90_DAY below rather than being fetched. Everything else +// is live GitHub API data, refreshed by re-running this script — the same +// generated/never-hand-edited pattern as data/metrics.json. +// +// Per LAUNCH.md, the pre-launch checkpoint that should be treated as the +// final launch baseline lands at W-6 (2026-09-28). Re-run this script at (or +// after) that date and record capturedAt so the baseline reflects the actual +// checkpoint rather than whatever day someone happened to run it. +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { githubFetch } from './lib/github.mjs'; + +const root = new URL('..', import.meta.url).pathname; +const REPO = 'cncf/endusers'; +const USER_AGENT = 'cncf-endusers-launch-metrics'; + +const TARGET_90_DAY = { + 'github-stars': 50, + 'github-watchers': 10, + 'github-forks': 5, + 'new-contributors-since-baseline': 5, + 'good-first-issues-claimed': 3, +}; + +async function main() { + const token = process.env.GH_TOKEN; + const [repo, contributors, goodFirstIssues] = await Promise.all([ + githubFetch(`https://api.github.com/repos/${REPO}`, token, USER_AGENT), + githubAll( + `https://api.github.com/repos/${REPO}/contributors?per_page=100&anon=false`, + token, + ), + githubAll( + `https://api.github.com/repos/${REPO}/issues?state=all&labels=good+first+issue&per_page=100`, + token, + ), + ]); + + const capturedAt = new Date().toISOString(); + // The repository's commit history long predates this launch plan (200+ + // merged PRs from dozens of human contributors already), so a raw + // cumulative contributor count is never a small number and can never sit + // below a "5 by launch+90d" target. The falsifiable signal is *new* + // contributors who show up after this baseline is captured, which is 0 by + // definition at capture time. priorContributorLogins is the snapshot a + // future run diffs against to count that delta. + const humanContributors = contributors.filter( + (c) => c.type !== 'Bot' && !/\[bot\]$|Bot$/i.test(c.login), + ); + const claimedGoodFirstIssues = goodFirstIssues.filter( + (issue) => !issue.pull_request && (issue.assignees || []).length > 0, + ); + + const data = { + generated: true, + generatedAt: capturedAt, + capturedAt, + checkpoint: { + label: 'W-6 pre-launch checkpoint (LAUNCH.md)', + targetDate: '2026-09-28', + note: 'Re-run npm run collect:launch-metrics on or after targetDate and commit the refreshed file before this baseline is treated as final for the launch retrospective.', + }, + source: `https://api.github.com/repos/${REPO}`, + signals: [ + signal( + 'github-stars', + 'GitHub stars', + repo.stargazers_count, + `https://github.com/${REPO}/stargazers`, + capturedAt, + ), + signal( + 'github-watchers', + 'GitHub watchers', + repo.subscribers_count, + `https://github.com/${REPO}/watchers`, + capturedAt, + ), + signal( + 'github-forks', + 'GitHub forks', + repo.forks_count, + `https://github.com/${REPO}/forks`, + capturedAt, + ), + { + ...signal( + 'new-contributors-since-baseline', + 'New unique human contributors since baseline', + 0, + `https://github.com/${REPO}/graphs/contributors`, + capturedAt, + ), + note: 'Baseline is 0 by definition (a delta metric). priorContributorLogins is the snapshot of existing human contributor logins at capture time; the 90-day re-measurement counts human contributor logins present then but absent from this list.', + priorContributorLogins: humanContributors.map((c) => c.login).sort(), + }, + signal( + 'good-first-issues-claimed', + 'Good-first-issues claimed', + claimedGoodFirstIssues.length, + `https://github.com/${REPO}/issues?q=is%3Aissue+label%3A%22good+first+issue%22`, + capturedAt, + ), + ], + }; + + writeFileSync( + join(root, 'data/launch-metrics.json'), + JSON.stringify(data, null, 2) + '\n', + ); + console.log( + `Collected ${data.signals.length} launch metrics at ${capturedAt}`, + ); +} + +function signal(id, label, baseline, sourceUrl, collectedAt) { + return { + id, + label, + baseline, + target90Day: TARGET_90_DAY[id], + sourceUrl, + collectedAt, + }; +} + +async function githubAll(url, token) { + const results = []; + let next = url; + while (next) { + const response = await fetch(next, { + headers: { + Accept: 'application/vnd.github+json', + 'User-Agent': USER_AGENT, + ...(token ? { Authorization: `Bearer ${token}` } : {}), + }, + }); + if (!response.ok) throw new Error(`GitHub API ${response.status}: ${next}`); + results.push(...(await response.json())); + const link = response.headers.get('link') || ''; + const match = link.match(/<([^>]+)>;\s*rel="next"/); + next = match ? match[1] : null; + } + return results; +} + +main(); diff --git a/scripts/validate-launch-metrics.mjs b/scripts/validate-launch-metrics.mjs new file mode 100644 index 00000000..03812f2a --- /dev/null +++ b/scripts/validate-launch-metrics.mjs @@ -0,0 +1,123 @@ +#!/usr/bin/env node +import { readFileSync } from 'node:fs'; +import { reportAndExit } from './lib/validate-utils.mjs'; +const data = JSON.parse( + readFileSync(new URL('../data/launch-metrics.json', import.meta.url)), +); +const errors = []; + +function checkUrl(path, field, value) { + if (value === undefined || value === null || value === '') return; + let parsed; + try { + parsed = new URL(value); + } catch { + errors.push({ + path, + severity: 'error', + message: `${field} must be an absolute URL`, + }); + return; + } + if (parsed.protocol !== 'https:') + errors.push({ + path, + severity: 'error', + message: `${field} must use https, got ${parsed.protocol}`, + }); +} + +function checkNonNegativeInteger(path, field, value) { + if (!Number.isInteger(value) || value < 0) + errors.push({ + path, + severity: 'error', + message: `${field} must be a finite, non-negative integer`, + }); +} + +if (!data.generated) + errors.push({ + path: 'launch-metrics.json', + severity: 'error', + message: 'generated must be true', + }); +if (Number.isNaN(Date.parse(data.generatedAt))) + errors.push({ + path: 'launch-metrics.json', + severity: 'error', + message: 'generatedAt must be ISO 8601', + }); +if (Number.isNaN(Date.parse(data.capturedAt))) + errors.push({ + path: 'launch-metrics.json', + severity: 'error', + message: 'capturedAt must be ISO 8601', + }); +checkUrl('launch-metrics.json', 'source', data.source); +if (!data.checkpoint?.label || !data.checkpoint?.targetDate) + errors.push({ + path: 'checkpoint', + severity: 'error', + message: 'checkpoint requires a label and a targetDate', + }); +if ( + data.checkpoint?.targetDate && + Number.isNaN(Date.parse(data.checkpoint.targetDate)) +) + errors.push({ + path: 'checkpoint', + severity: 'error', + message: 'checkpoint.targetDate must be a valid date', + }); + +const signals = data.signals || []; +if (signals.length < 3 || signals.length > 5) + errors.push({ + path: 'launch-metrics.json', + severity: 'error', + message: `signals must define 3-5 signals, got ${signals.length}`, + }); + +const ids = new Set(); +for (const signal of signals) { + const path = signal.id || '(missing id)'; + if (!signal.id || ids.has(signal.id)) + errors.push({ + path, + severity: 'error', + message: 'duplicate or missing signal id', + }); + ids.add(signal.id); + if (!signal.label) + errors.push({ path, severity: 'error', message: 'missing label' }); + if (!signal.sourceUrl || !signal.collectedAt) + errors.push({ + path, + severity: 'error', + message: 'missing provenance (sourceUrl/collectedAt)', + }); + if (Number.isNaN(Date.parse(signal.collectedAt))) + errors.push({ + path, + severity: 'error', + message: 'collectedAt must be ISO 8601', + }); + checkUrl(path, 'sourceUrl', signal.sourceUrl); + checkNonNegativeInteger(path, 'baseline', signal.baseline); + checkNonNegativeInteger(path, 'target90Day', signal.target90Day); + if ( + Number.isInteger(signal.baseline) && + Number.isInteger(signal.target90Day) && + signal.baseline > signal.target90Day + ) { + errors.push({ + path, + severity: 'error', + message: `target90Day (${signal.target90Day}) must be >= baseline (${signal.baseline})`, + }); + } +} + +reportAndExit(errors, 'launch metrics'); +console.log(`Validated ${signals.length} launch metrics`); diff --git a/tests/validate-launch-metrics.test.mjs b/tests/validate-launch-metrics.test.mjs new file mode 100644 index 00000000..df9b9995 --- /dev/null +++ b/tests/validate-launch-metrics.test.mjs @@ -0,0 +1,232 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { runScriptWithFixtures } from './helpers.mjs'; + +const SCRIPT = 'validate-launch-metrics.mjs'; + +const validSignal = { + id: 'github-stars', + label: 'GitHub stars', + baseline: 0, + target90Day: 50, + sourceUrl: 'https://github.com/cncf/endusers/stargazers', + collectedAt: '2026-09-21T00:00:00.000Z', +}; + +const validData = { + generated: true, + generatedAt: '2026-09-21T00:00:00.000Z', + capturedAt: '2026-09-21T00:00:00.000Z', + checkpoint: { + label: 'W-6 pre-launch checkpoint (LAUNCH.md)', + targetDate: '2026-09-28', + }, + source: 'https://api.github.com/repos/cncf/endusers', + signals: [ + validSignal, + { + ...validSignal, + id: 'github-watchers', + label: 'GitHub watchers', + target90Day: 10, + }, + { + ...validSignal, + id: 'github-forks', + label: 'GitHub forks', + baseline: 2, + target90Day: 5, + }, + ], +}; + +function fixture(data) { + return { 'data/launch-metrics.json': JSON.stringify(data) }; +} + +test('accepts a minimal valid launch-metrics file', () => { + const result = runScriptWithFixtures(SCRIPT, fixture(validData)); + assert.equal(result.status, 0, result.stderr); + assert.match(result.stdout, /Validated 3 launch metrics/); +}); + +test('rejects generated: false', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, generated: false }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /generated must be true/); +}); + +test('rejects a non-ISO generatedAt', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, generatedAt: 'not a date' }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /generatedAt must be ISO 8601/); +}); + +test('rejects a non-ISO capturedAt', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, capturedAt: 'not a date' }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /capturedAt must be ISO 8601/); +}); + +test('rejects a missing checkpoint', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, checkpoint: {} }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /checkpoint requires a label and a targetDate/); +}); + +test('rejects an invalid checkpoint targetDate', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + checkpoint: { label: 'x', targetDate: 'not a date' }, + }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /checkpoint.targetDate must be a valid date/); +}); + +test('rejects a non-https source', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + source: 'http://api.github.com/repos/cncf/endusers', + }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /must use https/); +}); + +test('rejects fewer than 3 signals', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, signals: [validSignal] }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /signals must define 3-5 signals/); +}); + +test('rejects more than 5 signals', () => { + const signals = Array.from({ length: 6 }, (_, i) => ({ + ...validSignal, + id: `signal-${i}`, + })); + const result = runScriptWithFixtures( + SCRIPT, + fixture({ ...validData, signals }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /signals must define 3-5 signals/); +}); + +test('rejects duplicate signal ids', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [validSignal, validSignal, { ...validSignal, id: 'x' }], + }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /duplicate or missing signal id/); +}); + +test('rejects a signal missing provenance', () => { + const { sourceUrl, ...rest } = validSignal; + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [rest, validData.signals[1], validData.signals[2]], + }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /missing provenance/); +}); + +test('rejects a non-integer baseline', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [ + { ...validSignal, baseline: 1.5 }, + validData.signals[1], + validData.signals[2], + ], + }), + ); + assert.equal(result.status, 1); + assert.match( + result.stderr, + /baseline must be a finite, non-negative integer/, + ); +}); + +test('rejects a negative target90Day', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [ + { ...validSignal, target90Day: -1 }, + validData.signals[1], + validData.signals[2], + ], + }), + ); + assert.equal(result.status, 1); + assert.match( + result.stderr, + /target90Day must be a finite, non-negative integer/, + ); +}); + +test('rejects a target90Day below baseline', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [ + { ...validSignal, baseline: 10, target90Day: 1 }, + validData.signals[1], + validData.signals[2], + ], + }), + ); + assert.equal(result.status, 1); + assert.match(result.stderr, /must be >= baseline/); +}); + +test('accepts a target90Day equal to baseline (delta metrics start at zero)', () => { + const result = runScriptWithFixtures( + SCRIPT, + fixture({ + ...validData, + signals: [ + { + ...validSignal, + id: 'new-contributors-since-baseline', + baseline: 0, + target90Day: 5, + }, + validData.signals[1], + validData.signals[2], + ], + }), + ); + assert.equal(result.status, 0, result.stderr); +}); diff --git a/tests/validators-smoke.test.mjs b/tests/validators-smoke.test.mjs index 3ad81d85..fd93f31a 100644 --- a/tests/validators-smoke.test.mjs +++ b/tests/validators-smoke.test.mjs @@ -12,6 +12,7 @@ const repoRoot = resolveRepoRoot(import.meta.url); // intentionally excluded. const READ_ONLY_VALIDATORS = [ 'validate-metrics.mjs', + 'validate-launch-metrics.mjs', 'validate-awards.mjs', 'validate-architectures.mjs', 'validate-architecture-assets.mjs',