Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions .github/workflows/compatibility.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
name: compatibility matrix

on:
pull_request:
paths:
- ".github/workflows/compatibility.yml"
- "docs/support-matrix.yaml"
- "docs/SUPPORT-MATRIX.md"
- "scripts/check-support-matrix.py"
- "scripts/check-support-matrix.sh"
- "libs/baggage-python/**"
- "libs/baggage-node/**"
- "libs/baggage-php/**"
- "libs/baggage-spring-boot-starter/**"
- "libs/baggage-servlet-filter/**"
- "libs/tekton-dag-common/**"
workflow_dispatch:
schedule:
# Weekly language-version evidence on the default branch.
- cron: "27 8 * * 5"

permissions:
contents: read

concurrency:
group: compatibility-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
contract:
name: Support matrix contract
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
with:
python-version: "3.12"

- name: Install checker
run: python -m pip install --requirement requirements-quality.txt

- name: Check support matrix
run: bash scripts/check-support-matrix.sh

python:
name: Python ${{ matrix.python }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python: ["3.11", "3.12"]
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
with:
python-version: ${{ matrix.python }}

- name: Install and test
run: |
python -m pip install --upgrade pip
python -m pip install --editable 'libs/tekton-dag-common[test]' --editable 'libs/baggage-python[test]'
python -m pytest libs/baggage-python/tests libs/tekton-dag-common/tests -q --tb=short

node:
name: Node ${{ matrix.node }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node: ["20", "22"]
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up Node
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: ${{ matrix.node }}
cache: npm
cache-dependency-path: libs/baggage-node/package-lock.json

- name: Install and test
working-directory: libs/baggage-node
run: |
npm ci
npm test

java:
name: Java ${{ matrix.java }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
java: ["21"]
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up Java
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5
with:
distribution: temurin
java-version: ${{ matrix.java }}
cache: maven
cache-dependency-path: |
libs/baggage-spring-boot-starter/pom.xml
libs/baggage-servlet-filter/pom.xml

- name: Maven tests
run: |
(cd libs/baggage-spring-boot-starter && mvn -B -q test)
(cd libs/baggage-servlet-filter && mvn -B -q test)

php:
name: PHP ${{ matrix.php }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ["8.3"]
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up PHP
uses: shivammathur/setup-php@bf6b4fbd49ca58e4608c9c89fba0b8d90bd2a39f # 2.35.5
with:
php-version: ${{ matrix.php }}
extensions: dom
tools: composer
coverage: none

- name: PHPUnit
working-directory: libs/baggage-php
run: |
composer install --no-interaction --quiet
./vendor/bin/phpunit
3 changes: 3 additions & 0 deletions .github/workflows/static-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,6 @@ jobs:

- name: Check representation sync
run: bash scripts/check-representation-sync.sh

- name: Check support matrix
run: bash scripts/check-support-matrix.sh
3 changes: 2 additions & 1 deletion docs/REGRESSION.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Do **not** confuse these:
| Scope | What runs | Typical trigger |
|-------|-----------|-----------------|
| **Application PR** (`stack-pr-test` on an **app** repo) | Stack-defined tests only — e.g. that app’s Newman/Playwright/Artillery as declared in `stacks/*.yaml`, against the intercept build. | Every PR on the **application** repository (when webhooks/Tekton are wired). |
| **Platform regression** (`scripts/run-regression*.sh` on **this** repo) | **System / integration** tiers: Phase 1 + orchestrator + shared libs + GUI pytest, Playwright for **management-gui**, real **`stack-dag-verify`** PipelineRun, Newman against **orchestrator** API, optional Tekton Results, optional Kind E2E. | **PRs / `main`:** [`.github/workflows/local-regression.yml`](../.github/workflows/local-regression.yml) runs **`--local-only --require-lang-tests`**. **Nightly / dispatch / `v*` tags:** [`.github/workflows/cluster-regression.yml`](../.github/workflows/cluster-regression.yml) runs Playwright + Kind. **Weekly / dispatch:** [`intercept-e2e.yml`](../.github/workflows/intercept-e2e.yml) runs both intercept backends and [`results-regression.yml`](../.github/workflows/results-regression.yml) runs strict Results/Postgres verification. |
| **Platform regression** (`scripts/run-regression*.sh` on **this** repo) | **System / integration** tiers: Phase 1 + orchestrator + shared libs + GUI pytest, Playwright for **management-gui**, real **`stack-dag-verify`** PipelineRun, Newman against **orchestrator** API, optional Tekton Results, optional Kind E2E. | **PRs / `main`:** [`.github/workflows/local-regression.yml`](../.github/workflows/local-regression.yml) runs **`--local-only --require-lang-tests`**. **Nightly / dispatch / `v*` tags:** [`.github/workflows/cluster-regression.yml`](../.github/workflows/cluster-regression.yml) runs Playwright + Kind. **Weekly / dispatch:** [`intercept-e2e.yml`](../.github/workflows/intercept-e2e.yml) runs both intercept backends, [`results-regression.yml`](../.github/workflows/results-regression.yml) runs strict Results/Postgres verification, and [`compatibility.yml`](../.github/workflows/compatibility.yml) runs the [support matrix](SUPPORT-MATRIX.md). |

So: **not all tests run on every PR.** `--local-only` (including Java/PHP/operator) is PR-gated. Playwright, Newman, Phase 2, and Kind isolation measurements run on **cluster-regression** (nightly / `workflow_dispatch` / version tags), not on pull requests. The slower Telepresence and mirrord product paths run weekly and on dispatch. App PRs run a narrower, stack-scoped test stage.

Expand Down Expand Up @@ -49,6 +49,7 @@ traffic artifact as verification.
| **F — GUI Postman** | [management-gui-tests.json](../tests/postman/management-gui-tests.json) vs a live Flask backend | `--gui-newman` or `scripts/run-gui-newman.sh`; weekly/dispatch in `graph-gui-newman.yml` |
| **F — Graph Postman** | [graph-tests.json](../tests/postman/graph-tests.json) vs orchestrator + Neo4j | `run-cluster-ci.sh --with-graph`; weekly/dispatch in `graph-gui-newman.yml` |
| **F — Representation sync** | Helm CRD copies, Stack/Team conversion, and PipelineRun params | `scripts/check-representation-sync.sh` in `static-quality.yml` |
| **F — Compatibility matrix** | Python, Node, Java, PHP, Kind, and Tekton versions from [SUPPORT-MATRIX.md](SUPPORT-MATRIX.md) | Weekly/dispatch/path in `compatibility.yml`; contract check in `static-quality.yml` |
| **G — Full Kind E2E** | [run-all-setup-and-test.sh](../scripts/run-all-setup-and-test.sh) | `--kind-e2e` |
| **H — Intercept product E2E** | [run-product-intercept-e2e.sh](../scripts/run-product-intercept-e2e.sh) via authenticated orchestrator API | Weekly/dispatch matrix in `intercept-e2e.yml`; requires repository secret `E2E_GIT_SSH_PRIVATE_KEY` with read access to application repos |

Expand Down
37 changes: 37 additions & 0 deletions docs/SUPPORT-MATRIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Support matrix

Canonical machine-readable source: [`support-matrix.yaml`](support-matrix.yaml).
`scripts/check-support-matrix.sh` fails when GitHub Actions jobs drift from it.

Only versions listed here are claimed as tested. Compile-image variants in
`helm/tekton-dag/values.yaml` may include additional tool tags; those are not
supported until a job in this matrix runs them.

## Languages

| Runtime | Tested versions | Primary (every PR) | Compatibility job |
|---------|-----------------|--------------------|-------------------|
| Python | 3.11, 3.12 | 3.12 | baggage-python + tekton-dag-common pytest |
| Node.js | 20, 22 | 22 | baggage-node vitest |
| Java | 21 | 21 | Maven baggage modules |
| PHP | 8.3 | 8.3 | baggage-php PHPUnit |
| Go | 1.26.8 (module 1.26.0) | 1.26.8 | operator CI / local regression |

Python 3.11 is also the docgen/demo-validation interpreter. PHP and Java stay
on a single version because `libs/baggage-php` requires `>=8.3` and the Java
baggage modules compile as source/target 21.

## Kubernetes and Tekton

| Component | Tested version | Cadence |
|-----------|----------------|---------|
| Kind | v0.27.0 | Nightly `cluster-regression`, weekly intercept + Results |
| Tekton Pipelines | v1.6.0 | `scripts/install-tekton.sh` |
| Tekton Triggers | v0.34.0 | `scripts/install-tekton.sh` |
| Tekton Results | v0.20.0 | weekly `results-regression` |

## Cadence

- **Every PR:** [local-regression.yml](../.github/workflows/local-regression.yml) uses the primary versions.
- **Weekly / dispatch / matrix-path PRs:** [compatibility.yml](../.github/workflows/compatibility.yml) runs every language version above.
- **Contract:** `static-quality` and local regression run `check-support-matrix.sh`.
31 changes: 31 additions & 0 deletions docs/support-matrix.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Canonical tested support matrix. check-support-matrix.py fails when GitHub
# Actions jobs drift from these versions. Do not list a version here unless a
# job actually runs it.
primary:
python: "3.12"
node: "22"
java: "21"
php: "8.3"
go: "1.26.8"

languages:
python:
- "3.11"
- "3.12"
node:
- "20"
- "22"
java:
- "21"
php:
- "8.3"

cluster:
kind: "v0.27.0"
tekton_pipelines: "v1.6.0"
tekton_triggers: "v0.34.0"
tekton_results: "v0.20.0"

go_module: "1.26.0"
cadence: "weekly"
workflow: ".github/workflows/compatibility.yml"
31 changes: 31 additions & 0 deletions libs/tekton-dag-common/tests/test_m17_compatibility.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""Static acceptance checks for M17.14 compatibility coverage."""

from pathlib import Path

ROOT = Path(__file__).resolve().parents[3]


def test_compatibility_workflow_covers_documented_runtimes():
workflow = (ROOT / ".github/workflows/compatibility.yml").read_text()
matrix = (ROOT / "docs/support-matrix.yaml").read_text()

assert "schedule:" in workflow
assert "workflow_dispatch:" in workflow
assert "pull_request:" in workflow
assert 'python: ["3.11", "3.12"]' in workflow
assert 'node: ["20", "22"]' in workflow
assert 'java: ["21"]' in workflow
assert 'php: ["8.3"]' in workflow
assert "bash scripts/check-support-matrix.sh" in workflow
assert 'workflow: ".github/workflows/compatibility.yml"' in matrix


def test_support_matrix_document_lists_cluster_pins():
doc = (ROOT / "docs/SUPPORT-MATRIX.md").read_text()

assert "3.11" in doc and "3.12" in doc
assert "20" in doc and "22" in doc
assert "v0.27.0" in doc
assert "v1.6.0" in doc
assert "v0.34.0" in doc
assert "v0.20.0" in doc
1 change: 1 addition & 0 deletions libs/tekton-dag-common/tests/test_m17_static_quality.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ def test_static_quality_workflow_covers_required_domains():
assert "npm run build" in workflow
assert "bash scripts/check-helm-chart.sh" in workflow
assert "bash scripts/check-representation-sync.sh" in workflow
assert "bash scripts/check-support-matrix.sh" in workflow


def test_quality_tool_downloads_and_actions_are_immutable():
Expand Down
20 changes: 12 additions & 8 deletions milestones/milestone-17.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Milestone 17 — End-to-end quality and production-readiness closure

**Status:** In progress after M17.12; next is M17.13 stack test runners
**Status:** In progress after M17.13; next is M17.14 compatibility matrix

This milestone converts the September 2026 end-to-end audit into an executable
backlog. Work is ordered by production risk, not by subsystem. A checkbox is
Expand All @@ -24,21 +24,22 @@ regression criteria in `docs/AGENT-REGRESSION.md` are satisfied.

## Resume checkpoint — 2026-09-16

- Completed with recorded acceptance: M17.1–M17.12.
- Still open: M17.13–M17.21.
- Completed with recorded acceptance: M17.1–M17.13.
- Still open: M17.14–M17.21.
- M17.4 live Results run `35108434664` on [PR #63](https://github.com/jmjava/tekton-dag/pull/63) exited 0 after the Java 21 toolchain fix.
- M17.11 live graph + GUI Newman run `35109309784` on [PR #65](https://github.com/jmjava/tekton-dag/pull/65) reported zero failed assertions.
- M17.12 live static-quality run `35111800721` on [PR #67](https://github.com/jmjava/tekton-dag/pull/67) passed Ruff, Go lint/vet, ShellCheck, both frontend builds, Helm package/render, and representation sync.
- Continue in numeric order from M17.13. Do not skip directly to
maintainability work because M17.13–M17.14 establish the test evidence needed
- M17.13 live local-regression run `35112504007` on [PR #68](https://github.com/jmjava/tekton-dag/pull/68) passed Newman/Playwright/Artillery runner fixtures.
- Continue in numeric order from M17.14. Do not skip directly to
maintainability work because M17.14 establishes the version evidence needed
to refactor safely.
- Before starting a slice, fetch the latest `cursor/close-e2e-audit-gaps-fc5f` and
create a fresh `cursor/<slice>-fc5f` branch. Do not reuse merged slice
branches.

| Resume order | Work | Terminal condition |
|---|---|---|
| 1 | M17.13–M17.14 test depth | Runner branches and compatibility matrix enforced |
| 1 | M17.14 test depth | Compatibility matrix enforced |
| 2 | M17.15–M17.19 maintainability | Duplication, ownership, legacy surface, errors, and config contracts consolidated behind green tests |
| 3 | M17.20–M17.21 docs/release | Canonical docs, governance, and reproducible release automation complete |
| 4 | Final verification | Full prescribed regression satisfies `docs/AGENT-REGRESSION.md` |
Expand Down Expand Up @@ -142,10 +143,12 @@ regression criteria in `docs/AGENT-REGRESSION.md` are satisfied.
- Evidence: run `35111800721` passed representation sync plus the rest of
static-quality after the Ruff import-order hotfix.

- [ ] **M17.13 Test embedded Task shell and stack test runners**
- [x] **M17.13 Test embedded Task shell and stack test runners**
- Add shell-level fixtures for malformed input and exercise Newman,
Playwright, and Artillery branches of `run-stack-tests`.
- Acceptance: each supported runner has success and failure-path coverage.
- Evidence: run `35112504007` passed local regression including malformed
stack-json and Newman, Playwright, and Artillery success/failure fixtures.

- [ ] **M17.14 Expand compatibility coverage**
- Test supported Python, Node, Java, PHP, and Kubernetes/Tekton versions at an
Expand Down Expand Up @@ -212,5 +215,6 @@ regression criteria in `docs/AGENT-REGRESSION.md` are satisfied.
| 2026-09-16 | M17.4 Results live acceptance | Java 21 toolchain; Phase 2; Newman; Results DB | Run 35108434664 passed |
| 2026-09-16 | M17.11 graph and GUI Newman | Live Flask GUI collection; Kind Neo4j graph collection | Run 35109309784 passed, zero assertion failures |
| 2026-09-16 | M17.12 representation sync acceptance | CRD copy, Stack/Team conversion, PipelineRun param drift, and static-quality | Run 35111800721 passed |
| 2026-09-16 | M17.13 stack test runner fixtures | Extracted `run-stack-tests` Newman/Playwright/Artillery runners; malformed JSON plus success/failure fixtures | Automation added; first live local-regression run still required |
| 2026-09-16 | M17.13 stack test runner fixtures | Extracted `run-stack-tests` Newman/Playwright/Artillery runners; malformed JSON plus success/failure fixtures | Run 35112504007 passed |
| 2026-09-16 | M17.14 compatibility matrix automation | Documented language/Kind/Tekton matrix; weekly jobs; contract check | Automation added; first live compatibility run still required |

Loading
Loading