Skip to content

[scripts] Show live SHA-256 progress during duplicate check scanning - #105

Merged
Chain-Frost merged 8 commits into
mainfrom
feat/check-file-hash-progress-dashboard
Oct 13, 2026
Merged

Chain-Frost merged 8 commits into
mainfrom
feat/check-file-hash-progress-dashboard

Conversation

@Chain-Frost

@Chain-Frost Chain-Frost commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Scope

Follow-up to merged #104. Add a live Rich terminal progress display to the SHA-256 comparison phase in ryan-scripts/TUFLOW-python/model_management/delete_duplicate_check_files.py.

  • Show byte-weighted percentage, processed/total bytes, transfer speed, ETA, current file, and completed candidate-file count, with redraws limited to 2/sec. Progress advances during each read chunk, including within very large rasters; a simple files-finished counter would not suffice.
  • Calculate total work from size/suffix match candidates only, before hashing; never hash unique-sized files merely to feed the dashboard.
  • Keep SHA-256 unchanged and preserve one content hash per candidate. The progress callback measures bytes from the original read; deletion still uses metadata checks and mandatory DELETE confirmation.
  • Use Rich's purpose-built Progress (the terminal dashboard technology already used in this repository), rather than creating an unrelated graphical window or expanding the shared task-oriented LiveWorkflowDashboard.
  • Editable SHOW_HASH_PROGRESS flag and low-cost refresh-rate option; redirected/headless sessions receive a summary rather than live terminal redraw.
  • Regression coverage for chunk-level progress, candidate selection, opt-out, empty files and original single-pass hashing; update the TUFLOW script README.

Validation

  • Current head: f99a1390ce868d35b2d57b738012fb9ffb5fba27.
  • GitHub Actions run 38389564545: Policy, Hosted tests and Package all passed on this head.
  • Local CI-equivalent checks passed: Ruff lint/format and strict Pyright on both PR Python files; repository documentation/logging policy; hosted test boundary (957 passed, 5 deselected, zero skips); retained-wheel verification; no-bump rebuild/verification; isolated installed-wheel smoke test; and git diff --check.
  • Rebuilt and published retained wheel 26.10.13.1 to fix the missing velocity_vectors.qml resource. SHA-256: 28e3ca26a75a6de6936082191d6de55ad6ddb2e8d5ec90b0124312d2c24019da.
  • Large-file Windows/network-drive visual smoke check remains recommended; synthetic chunk tests verify the byte counter without a 100 GB fixture.

Operational notes

Rich is already part of the repository's Python dependencies. If this standalone script is copied outside its installed environment, install rich to enable the terminal progress dashboard.

No change to deletion behaviour. The new display ends before the existing deletion confirmation prompt.

@Chain-Frost
Chain-Frost marked this pull request as ready for review October 13, 2026 10:02
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T10:05:47.315580Z 79e63f2 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Chain-Frost Chain-Frost self-assigned this Oct 13, 2026
@Chain-Frost
Chain-Frost merged commit 29aac5e into main Oct 13, 2026
3 checks passed
@Chain-Frost
Chain-Frost deleted the feat/check-file-hash-progress-dashboard branch October 13, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant