Defensive ML research for developers and researchers building malware and network-intrusion detectors. The repository implements dataset preprocessing, baseline training, evaluation, offline feature-space attacks and a research dashboard. Measured exploratory EMBER validation results are available; the frozen-baseline benchmark still awaits M1 and reviewed policy. Defenses remain planned.
The MNIST demo in this repo is only an environment check that proves the attack tooling works — it is not the project's security result.
The primary UI is plain HTML, CSS and JavaScript served by FastAPI; no frontend build or Streamlit process is needed. A fresh checkout includes recorded comparison results, not datasets or trained models. Start with Quick start, or follow the team handoff to synchronize an existing checkout.
Works today:
- ART demo on MNIST with FGSM, PGD, and CW-L2 (
scripts/art_quickstart.py), including accuracy-vs-epsilon plots, a JSON report, and local MLflow logging. - Test suite (
pytest) and lint/format checks (ruff). - FastAPI service with liveness, read-only research artifacts, and gated, bearer-token-protected feature prediction. See the API reference.
- HTML dashboard v0.6.0 (Sprint 06): guided measured AI journey with an English / Egyptian Arabic language switch, attack comparisons and detailed model/data/methodology/tooling pages; baseline comparisons use recorded validation artifacts. See the journey guide. The earlier Streamlit implementation is retained as a legacy reference.
Dockerfileanddocker-compose.ymlfor api, dashboard, and mlflow services (the image is built in CI; Compose publishes ports on localhost by default and runs the API/dashboard with dropped capabilities).- Cleaned/versioned EMBER2018 v2 and CIC-IDS-2017 datasets with manifest-based
versioning, reusable loaders, EDA, data cards, and a reproducible
scripts/preprocess.pypipeline. - Security baseline: upgraded vulnerable dependencies, API host/header
hardening, defensive dashboard URL handling, manifest path containment,
loopback-only Compose defaults, and CI dependency/static checks. Evidence
and residual risks are in
docs/security/SECURITY_AUDIT.md.
Sprint 3 adds Logistic Regression, Random Forest, Linear SVM and PyTorch MLP
training with reproducible validation reports and MLflow tracking. Follow
docs/sprint3.md for local data setup and commands.
Sprint 4 adds CPU MLP/CNN tuning, calibration, gated freezing and authenticated
feature prediction. See docs/sprint4.md and the
Sprint 4 report for measured results and M1 status.
Sprint 5 adds an offline attack pipeline and HTML results explorer.
Sprint 6 adds offline surrogate transfer evaluation and a bilingual
threat/realism explorer. Real transfer measurements remain
pending colleague-provided bundles/data and authorization; no training or
downloads were run on this laptop. See Sprint 6 evidence.
The planned frozen benchmark requires M1 and technical feature-policy review.
Explicit offline candidate exploration
uses operating validation only and is labelled development—not frozen;
it does not enable prediction or complete M1. The explorer shows pending until
measured reports exist. Defenses remain later-sprint work.
Sprint-by-sprint status is tracked in
docs/progress.md; scope and schedule are in
docs/project_plan.md.
Sprint 5 has 21 completed torch-only CleverHans cross-checks and 57 measured
case summaries; this does not extend the historical MNIST demo evidence.
Docker is covered differently: the image builds
in CI on pull requests and pushes to main; the 2026-10 security audit rebuilt
the image, started the full three-service stack on loopback, and health-checked
the API, dashboard, and MLflow. Browser/MLflow artifact workflows and sustained load were not
exercised, and the official MLflow container runs as root. See the
security audit and the
troubleshooting section for details.
Prerequisites: Git and Python 3.11–3.12 on Linux or Windows. The Linux example
uses GNU Make and python3.11; use PY=python3.12 if that is your installed
supported interpreter. Docker is optional. The first install needs internet;
viewing recorded results does not download MNIST.
Native macOS setup is not supported by the current exact CPU torch pins: the configured index has no matching macOS wheels. Do not replace the pins silently. See the platform notes.
git clone https://github.com/MatsoDev/Adversarial-Resilient.git
cd Adversarial-Resilient
python3.11 --version # must report Python 3.11.x
make setup PY=python3.11 # .venv, pinned dependencies, attempts Git-hook installation
make lint # ruff check + format check (same as CI)
make test # pytest, synthetic fixtures only
make coverage # pytest with line/branch coverage
make security # dependency + static security checks
source .venv/bin/activate # Linux; Make uses .venv automatically, Python does not
python -m uvicorn api.main:app --host 127.0.0.1 --port 8000 --no-proxy-headersOpen http://localhost:8000/dashboard/. Viewing the recorded results needs no
datasets, trained models, token, database, or .env. Prediction remains disabled
without both a qualifying frozen bundle and a local token; implementation is
not evidence that M1 has passed. See current status.
Success: /health returns {"status":"ok"}, the dashboard shows recorded
results, lint is clean and tests pass. Three real-data tests skip without
RUN_REAL_DATA_TESTS=1; three CleverHans cases skip without its optional
installation. Four JavaScript component cases skip without Node.js, which is
test tooling, not a UI runtime dependency. See optional checks.
The existing Linux environment, checks and HTTP responses were verified on 2026-11-17; a fresh clone/install was not rerun. See the documentation audit for evidence and limits.
Windows and non-Make setups are documented in the
development guide. make setup PY=python3 is also valid
if python3 --version reports 3.11 or 3.12, not a newer unsupported version.
With the environment active, the equivalent lint/test commands without Make are:
python -m ruff check src scripts tests api dashboard
python -m ruff format --check src scripts tests api dashboard
python -m pytest| Command | What it does |
|---|---|
make demo |
Run the fast ART MNIST demo (downloads MNIST, takes a few minutes) |
make data |
Full EMBER + CICIDS preprocessing (needs local raw datasets) |
make data-smoke |
Deterministic 2000-row subsets into /tmp |
make eda |
Regenerate EDA JSON, figures, and report under docs/ |
make dashboard |
Run FastAPI + HTML dashboard at http://localhost:8501/dashboard/ |
make docker-build / make docker-up |
Build the image / start the compose stack (cp .env.example .env first) |
make docker-scan |
Scan the built image for fixed HIGH/CRITICAL vulnerabilities |
make format |
Rewrite files with ruff format |
make coverage |
Full test suite with line/branch coverage (CI uses this) |
make security |
Dependency audit (pip-audit) + static security scan (bandit) |
make check |
make lint + make test |
make clean |
Remove caches and mlruns/ (committed demo/EDA artifacts stay) |
The API and HTML dashboard share one process; choose the uvicorn command above
or make dashboard, not both. Full setup, legacy Streamlit instructions,
demo flags, service details, and troubleshooting are in
docs/development.md.
Start with the documentation index: docs/README.md.
Highlights:
docs/development.md— setup, verification, ART demo, API/dashboard, Docker, troubleshooting.docs/configuration.md— environment variables, dataset resolution, Compose ports, paths, and seeds.docs/testing.md— test layout, commands, fixtures, and optional real-data checks.docs/architecture.md— components, dependency rules, and data flows.docs/project-structure.md— directory responsibilities and where new code belongs.docs/repository_audit.md— maintenance findings, cleanup evidence, verification and intentionally retained files.docs/data/README.md— dataset locations, preprocessing/EDA commands, and loaders.api/README.md— HTTP endpoints, responses, and OpenAPI URLs.docs/progress.md— current sprint and milestone status (scope baseline:docs/project_plan.md).AGENTS.md— quick navigation map for AI coding agents.CONTRIBUTING.md— branching, commit style, PR rules.docs/DECISIONS.md— non-obvious technical decisions.docs/threat_model_v0.1.md— security scope and attacker model.docs/security/SECURITY_AUDIT.md— application security audit, fixes, verification, and remaining risks.
api/ FastAPI health/artifact routes, authenticated feature prediction
dashboard/ web/ HTML/CSS/JS (primary), app.py + client.py (legacy Streamlit)
data/ raw/, processed/, adversarial/, external/ runtime data (gitignored)
docs/ documentation (index at docs/README.md), data guides, reports
docs/data/ feature layout, data guide, generated EDA report, manifests
docs/figures/ demo + EDA plots (generated, committed on purpose)
docs/results/ demo + EDA JSON (generated, committed on purpose)
models/ local baseline checkpoints and Sprint 4 bundles (gitignored)
notebooks/ exploratory notebooks (later)
scripts/ environment setup, demo, preprocessing, EDA, training and tuning CLIs
src/ config.py paths/seeds, utils/, data/, models/, attacks/, defenses/, evaluation/
tests/ seed, config, API, data (synthetic), docs links, and smoke tests
Full reference: docs/project-structure.md.
- Branch names:
feat/<topic>,fix/<topic>,docs/<topic>,chore/<topic>(orsprint<N>/<topic>). - Commits use Conventional Commits:
feat:,fix:,refactor:,docs:,chore:,test:,ci:. - Verified direct pushes and autonomous agent integration are allowed when GitHub rules permit; required PRs, checks and approvals still apply. See the Git workflow.
- Never commit datasets, model weights,
.env, ormlruns/. Checkgit statusbefore every commit.
Semester 1:
- Foundations: environment, ART demo on MNIST, threat model draft.
- Data: EMBER and CIC-IDS-2017 download, EDA, preprocessing pipeline.
- Attacks: FGSM, PGD, and CW against the real detectors.
- Defenses: adversarial training, distillation, ensembles, sanitization, detector.
Semester 2:
- Backend: extend the existing gated feature prediction with attack/defense endpoints.
- Frontend: extend the research dashboard with robustness comparisons.
- Hardening: containers, CI checks, and the final evaluation.
Defensive research only. This repo works with public feature datasets and the MNIST demo images — no malware samples and no weaponized tooling are included or accepted here.