Skip to content

test: strengthen attack, loader, and journey regression coverage - #24

Merged
mohamed2ze merged 4 commits into
mainfrom
sprint5/attacks-dashboard-handoff
Nov 17, 2026
Merged

mohamed2ze merged 4 commits into
mainfrom
sprint5/attacks-dashboard-handoff

Conversation

@mohamed2ze

Copy link
Copy Markdown
Collaborator

What

  • Improve tests only, with 81 net additional collected cases and no production, dependency, configuration, documentation, or generated-artifact changes.
  • tests/test_attacks.py: share synthetic setup and verify canonical filenames, mandatory hashes, same-size corruption, candidate qualification, and operating-cohort identity.
  • tests/test_attack_constraints.py: independently assert policy validation, projection bounds, integer rounding budgets, immutable features, and input preservation.
  • tests/test_data_loader.py: test each missing scaler file independently, exact holdout transformations, unchanged artifacts, and malformed CICIDS shapes.
  • tests/test_journey.py: verify first-case selection, repaired errors, absent outcomes, calibrated scores, threshold equality, and inverse preprocessing.

The work is split into four logical commits on the existing Sprint 5 branch. The temporary chore branch has been deleted.

Why

Protect high-risk research-integrity and numerical contracts with assertions that detect regressions, rather than adding coverage-only checks. Preserve all existing tests and optional skips.

How tested

  • make lint and pre-commit hooks passed before each commit.
  • Full pytest suite with configured line/branch coverage (python -m pytest --cov=src --cov=api --cov=dashboard --cov=scripts --cov-report=term-missing). This runs the same suite as make test / make coverage.
  • Changed suites passed three consecutive runs, including reversed file order: 126 passed, 3 optional skips each time.
  • Relevant test file rerun successfully for each of the four commits.
  • All 24 in-memory simulated regression scenarios were detected; production source files were never edited.
  • Final split-commit tree verified byte-identical to the full-suite-tested version.
Metric Baseline After
Passed / failed / skipped 468 / 0 / 6 549 / 0 / 6
Full coverage runtime 68.80s 79.76s
Statement coverage 90.39% 90.72%
Branch coverage 80.35% 81.48%
Combined coverage 88.10% 88.62%

The six existing skips are three opt-in real-data checks and three unavailable optional CleverHans cases. No speedup or flaky-test repair is claimed. Security scans, Docker checks, and real-MNIST regeneration were not rerun for this test-only change.

Checklist

  • Conventional Commits title and four focused commits.
  • No existing tests deleted, skipped, or loosened; no new skips or dependencies.
  • No datasets, weights, .env, or MLflow outputs committed.
  • Synthetic test outputs isolated; dependency boundaries and generated-file rules preserved.
  • Only the four test files above changed; both worktrees are clean.

No new production bug was confirmed. Documentation/contract updates and a new production security assessment are not applicable to this test-only PR.

@mohamed2ze
mohamed2ze merged commit f8adb8c into main Nov 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant