Skip to content

[T-014] Campaign export/import bundle (.tar.zst, JSON Lines + artefacts) with safe extraction #125

Closed

Description

@CodewithJha

Task T-014 · Milestone M6 — Analysis + risk · Size M · Priority P2 · Epic #6

Implements FR-STO-003 · PR-20 · NFR-SEC-003

Branch feat/t-014-campaign-export-import-bundle-json — one task = one branch = one PR (doc 32 §6)

Summary

Implement roboprobe export <campaign_id> -o bundle.tar.zst and roboprobe import <bundle>: DB rows as JSON Lines with schema versions, artefacts by hash, manifest with checksums, and hardened extraction (doc 31 TM-14).

Context / motivation

Sharing reproducible results with supervisors/peers (persona Dr. Rao) without sharing a database.

Scope

In scope

  • store/bundle.py: export_campaign(campaign_id, uow, store, out) and import_bundle(path, uow, store, *, on_conflict='skip'|'fail').
  • Bundle layout: bundle.json (format version, campaign id, row counts, sha256 of each file), rows/<table>.jsonl, blobs/<sha256>.
  • Safe tar: reject absolute paths, .., symlinks/hardlinks, device files; size limits; verify all hashes before DB writes.
  • Host names hashed in manifests (TM-15).
  • CLI cli/bundle.py.

Implementation guidance

  • Use zstandard streaming (add to deps if not already in T-001). Python 3.12 tarfile extraction filter 'data' plus explicit checks.
  • Standards: respect the layering contracts (doc 16 §2) and conventions in doc 34; no hard-coded tunables — thresholds/defaults come from typed settings (doc 27); no magic numbers (ruff PLR2004); public APIs typed (mypy strict) and documented.

Acceptance criteria

  • Round-trip: export → import into empty DB reproduces identical rows and artefact hashes.
  • Malicious bundles (traversal, symlink, wrong hash) are rejected with clear errors and leave the DB untouched.
  • Bundle format documented in doc 11.
  • Tests listed below added and green in CI; coverage of new code meets the gate in doc 18 §3.
  • Docs, config reference and CHANGELOG Unreleased updated where behaviour, keys or interfaces changed.
  • Definition of Done (doc 32 §3) satisfied.

Tests required

  • tests/integration/store/test_bundle_roundtrip.py, tests/unit/store/test_bundle_security.py.

Dependencies

Blocked by: #62 (T-011b), #66 (T-013a), #20 (T-008)

Blocks: #161 (T-109)

Parallelization notes

Files / paths owned by this task — avoid editing other paths; shared files (pyproject.toml entry points, cli/main.py, api/routers/__init__.py, router.tsx, docker/compose.yaml, ci.yml) are append-only:

  • backend/src/roboprobe/store/bundle.py
  • backend/src/roboprobe/cli/bundle.py
  • docs/11-data-model-and-database.md (bundle section)

Independent new modules.

Definition of Done

Per doc 32 §3 and AGENTS.md §5: lint, format, mypy strict and import-linter green; tests added and passing; docs updated; PR template completed with Closes #<this issue>; Conventional Commit messages referencing T-014 (doc 16 §6); commits authored only by the repository owner identity (AGENTS.md §4); CI green; reviewer checklist (doc 32 §5.1) satisfied.

References

Design docs: doc 11 §6 · doc 31 §3 · doc 21 §4 · doc 02 §3.10

Requirements: FR-STO-003 · PR-20 · NFR-SEC-003 (traceability: doc 28)

External:

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cliCommand-line interfacearea:securitySecurity hardening, supply chainarea:storagePostgres schema, repositories, artifactsmilestone:M6M6 — Analysis + riskpriority:P2Important, not blockingsize:M1-3 daystaskChild task of a milestone epictype:featureNew functionality

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions