Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
261 changes: 212 additions & 49 deletions .github/actions/reprocase-scan/action.yml

Large diffs are not rendered by default.

36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,41 @@ All notable changes to ReproCase are documented here.
Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versioning follows [Semantic Versioning](https://semver.org/).

## [0.6.0] — 2026-11-08

Reliable CI and developer feedback release. Fixes the default ref normalisation defect from
v0.5.1 and adds a professional GitHub Actions Job Summary.

### Added

- **GITHUB_STEP_SUMMARY**: professional Markdown summary in the GitHub Actions UI showing
target, ReproCase version, findings by severity (verified/suspected), compare results
(new/resolved/persistent/unconfirmed), gate result with clear reasoning, report artifact
reference, observation timeouts, and coverage-shrink warnings. All scanned-site text is
HTML-escaped (no Markdown injection); long values truncated; generated even when the scan
fails (if: always()); no external dependencies.
- **Ref normalisation**: correctly handles `refs/tags/v0.5.1`, `refs/heads/main`, bare tags
(`v0.5.1`), bare branches (`main`), and full 40-char commit SHAs. Shell injection and path
traversal in the ref are rejected with explicit errors. The checked-out commit is verified
to match the requested SHA.
- **Caller github.ref isolation**: the action uses `github.action_ref` (not the caller's
`github.ref`) for the default reprocase-ref, preventing accidental installation of the
wrong code.
- **Action structure tests**: 21 tests covering ref normalisation, Job Summary content,
security guards, and artifact privacy.

### Fixed

- **Default ref normalisation** (P0): the v0.5.1 action stripped only `refs/` from the ref,
leaving `tags/` in the path for `git clone --branch`. Now all three GitHub ref formats are
correctly normalised.
- **Late-fault test timing**: observation-window budget test now uses an 8000 ms fault delay
(beyond page teardown) for reliable budget verification.

### Verification

181/181 tests (72 unit + 109 integration/e2e); npm audit 0; four-OS CI matrix.

## [0.5.1] — 2026-11-06

GitHub Actions composite-action reliability release. Three real defects found by the
Expand Down Expand Up @@ -502,6 +537,7 @@ First verified release.
- No accessibility, performance, or visual-regression coverage beyond horizontal overflow.
- Single-browser (Chromium) at this stage; Firefox/WebKit untested.

[0.6.0]: https://github.com/geyutu6755/ReproCase/releases/tag/v0.6.0
[0.5.1]: https://github.com/geyutu6755/ReproCase/releases/tag/v0.5.1
[0.5.0]: https://github.com/geyutu6755/ReproCase/releases/tag/v0.5.0
[0.4.0]: https://github.com/geyutu6755/ReproCase/releases/tag/v0.4.0
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ fault through the real Playwright runner.

## Status

v0.5.1 — GitHub Actions reliability release (fail-on-new semantics, exit-code ordering, SHA support, npm ci). See the [changelog](CHANGELOG.md).
v0.6.0 — reliable CI & developer feedback release (ref normalisation fix, Job Summary, injection guards). See the [changelog](CHANGELOG.md).
`npm i -g github:…#tag` now works and the quick start drops the manual build), a reconciled
Node baseline (**22+**, since Node 18/20 are End-of-Life; CI-tested on 22 & 24), the deprecated
Vitest `poolOptions` config fixed, and refreshed real screenshots. Detection itself is unchanged
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ npm run test:all # 单元 + 集成(真实浏览器)+ CLI 端到端

## 状态

v0.5.1 —— GitHub Actions 可靠性修复版本(fail-on-new 语义、退出码顺序、SHA 支持、npm ci)。详见 [变更日志](CHANGELOG.md)。
v0.6.0 —— 可靠 CI 与开发者反馈版本(引用规范化修复、Job Summary、注入防护)。详见 [变更日志](CHANGELOG.md)。
[变更日志](CHANGELOG.md)、[测试基线](docs/test-baselines.md) 与[兼容性矩阵](docs/compatibility.md)。

## 许可证
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "reprocase",
"version": "0.5.1",
"version": "0.6.0",
"description": "Automated website fault detection & reproduction. Scans a site in a real browser, verifies each fault independently, and emits reproducible Playwright regression tests plus human- and CI-friendly reports.",
"type": "module",
"license": "MIT",
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/observation-window.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ describe('adaptive async-observation window', () => {
const tmpSite = fs.mkdtempSync(path.join(os.tmpdir(), 'reprocase-late-'));
fs.copyFileSync(path.join(dir, 'index.html'), path.join(tmpSite, 'index.html'));
let html = fs.readFileSync(path.join(tmpSite, 'index.html'), 'utf8');
html = html.replace(', 1000);', ', 2500);');
html = html.replace(', 1000);', ', 8000);');
fs.writeFileSync(path.join(tmpSite, 'index.html'), html, 'utf8');

const { startStaticServer: start } = await import('../../src/dev-server.js');
Expand All @@ -81,7 +81,7 @@ describe('adaptive async-observation window', () => {
});
const { result } = await new Scanner(config, out).run();
const boom = result.findings.find((f) => f.title.includes('LateFault'));
expect(boom, 'fault at 2500 ms must be missed with a 400 ms budget (documented trade-off)').toBeUndefined();
expect(boom, 'fault at 8000 ms must be missed with a 400 ms budget (documented trade-off)').toBeUndefined();
} finally {
await shortServer.close();
fs.rmSync(tmpSite, { recursive: true, force: true });
Expand Down
121 changes: 121 additions & 0 deletions tests/unit/action-v060.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
import * as fs from 'node:fs';
import * as path from 'node:path';
import { describe, expect, it } from 'vitest';
import { PROJECT_ROOT } from '../helpers/demo-server.js';

const ACTION_YML = path.join(PROJECT_ROOT, '.github', 'actions', 'reprocase-scan', 'action.yml');
const yml = fs.readFileSync(ACTION_YML, 'utf8');

describe('v0.6.0 ref normalisation (P0 fix)', () => {
it('normalises refs/tags/ to bare tag', () => {
expect(yml).toContain("refs/tags/*) REF=\"${REF#refs/tags/}\"");
});

it('normalises refs/heads/ to bare branch', () => {
expect(yml).toContain("refs/heads/*) REF=\"${REF#refs/heads/}\"");
});

it('strips generic refs/ prefix as last resort', () => {
expect(yml).toContain("refs/*) REF=\"${REF#refs/}\"");
});

it('detects 40-char SHA and uses full clone (not --branch)', () => {
expect(yml).toContain("grep -qP '^[0-9a-f]{40}$'");
expect(yml).toContain('IS_SHA=true');
});

it('SHA mode uses full clone + git checkout (not --branch)', () => {
expect(yml).toContain('git clone https://github.com/geyutu6755/ReproCase.git');
expect(yml).toContain('git checkout "$REF"');
});

it('verifies SHA checkout matches the requested SHA', () => {
expect(yml).toContain("does not match requested SHA");
});

it('rejects shell injection characters in the ref', () => {
expect(yml).toContain('unsafe characters');
expect(yml).toContain('[;&|`$(){}!\\[\\]<>\\\\]');
});

it('rejects path traversal in the ref', () => {
expect(yml).toContain("path traversal");
expect(yml).toContain("'..'");
});

it('rejects empty ref with explicit error', () => {
expect(yml).toContain('reprocase-ref is empty');
});

it('does NOT use caller github.ref — only github.action_ref for defaults', () => {
// github.ref would be the CALLER's branch/tag, not the action's version.
// Using it would install the wrong code.
const installBlock = yml.slice(
yml.indexOf(' - name: Install ReproCase'),
yml.indexOf(' - name: Install Chromium')
);
expect(installBlock).not.toContain('github.ref');
expect(installBlock).toContain('github.action_ref');
});

it('resolves ref BEFORE clone (normalisation happens first)', () => {
const normalisePos = yml.indexOf("refs/tags/*)");
const clonePos = yml.indexOf('git clone --depth 1 --branch');
expect(normalisePos).toBeLessThan(clonePos);
});

it('exports reprocase-commit and reprocase-ref-resolved outputs', () => {
expect(yml).toContain('reprocase-commit=');
expect(yml).toContain('reprocase-ref-resolved=');
});
});

describe('v0.6.0 Job Summary (GITHUB_STEP_SUMMARY)', () => {
it('writes to GITHUB_STEP_SUMMARY', () => {
expect(yml).toContain('GITHUB_STEP_SUMMARY');
});

it('generates summary even when scan fails (if: always())', () => {
const summaryStep = yml.slice(
yml.indexOf(' - name: Generate Job Summary'),
yml.indexOf(' - name: Publish privacy-stripped report')
);
expect(summaryStep).toContain('if: ${{ always() }}');
});

it('sanitises scanned-site text in the summary (HTML entity escaping)', () => {
expect(yml).toContain('&amp;');
expect(yml).toContain('&lt;');
expect(yml).toContain('&gt;');
expect(yml).toContain('&quot;');
});

it('truncates long values in the summary', () => {
expect(yml).toContain("s.slice(0, max)");
});

it('shows observation timeouts when present', () => {
expect(yml).toContain('observationTimeouts');
expect(yml).toContain('not fully observed');
});

it('shows compare results when baseline is provided', () => {
expect(yml).toContain('Changes vs baseline');
expect(yml).toContain('resolvedUnconfirmedCount');
expect(yml).toContain('coverageShrunk');
});

it('shows the gate result with clear reasoning', () => {
expect(yml).toContain('Gate result:');
expect(yml).toContain('Baseline gate FAILED');
expect(yml).toContain('Baseline gate passed');
});

it('references the report artifact by name', () => {
expect(yml).toContain('reprocase-report/report.html');
});

it('notes that traces are private', () => {
expect(yml).toContain('private evidence');
});
});
3 changes: 2 additions & 1 deletion tests/unit/action-yml.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ describe('composite action structure (v0.5.1 reliability)', () => {
it('install step supports SHA checkout with explicit error for missing refs', () => {
expect(yml).toContain('git checkout');
expect(yml).toContain('git cat-file');
expect(yml).toContain('not found as a tag, branch or commit SHA');
expect(yml).toContain('not found as a tag or branch');
expect(yml).toContain('not found in geyutu6755/ReproCase');
expect(yml).toContain('git rev-parse HEAD');
});

Expand Down
Loading