Skip to content

[scripts] Verify legacy FLT/ASC rasters against newer GeoTIFFs - #107

Merged
phantom-branch merged 16 commits into
mainfrom
feat/verify-converted-tuflow-rasters
Oct 14, 2026
Merged

phantom-branch merged 16 commits into
mainfrom
feat/verify-converted-tuflow-rasters

Conversation

@Chain-Frost

Copy link
Copy Markdown
Owner

Summary

  • Add a standalone, recursive legacy TUFLOW raster cleanup tool under model_management without changing the existing check-file deduplicator.
  • Match FLT/ASC files to same-stem TIF/TIFF files, in-place or in a mirrored folder tree.
  • Compare band count, dimensions, CRS, grid transform, valid/NoData masks, and every valid cell exactly, using windowed Rasterio reads.
  • Require newer TIFF timestamps, including newer than an FLT header if that header is scheduled for deletion.
  • Write non-overwriting deletion-candidate/error CSVs. No placeholders; .prj and other auxiliary metadata retained.
  • Default to validation-only. Optional deletion requires both --delete (or explicit setting) and typing DELETE. Recheck file metadata immediately before deletion.
  • Document usage in ryan-scripts/TUFLOW-python/README.md.

Validation

  • 15 focused synthetic Rasterio tests pass locally (pytest test_verify_converted_tuflow_rasters.py -q), covering match, mismatch, missing/ambiguous/corrupt TIFF, spatial metadata, NoData, timestamps, deletion confirmation and change detection.
  • Python syntax compilation passed.
  • Hosted CI pending. No representative legacy project rasters were available for testing.

Scope and cautions

  • Requires exact matching grid/CRS metadata and values. No resampling or numeric tolerance.
  • Old-model file references are not rewritten and may still point to removed FLT/ASC paths.
  • Metadata snapshots do not lock files. Run only on inactive result folders and review candidate reports/backup first.
  • Avoids deleting .prj and auxiliary files; FLT .hdr is included only when DELETE_FLT_HEADERS is enabled (default True).

@phantom-branch
phantom-branch marked this pull request as ready for review October 14, 2026 08:50
@phantom-branch

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

@Chain-Frost

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T09:27:54.972857Z 569d2fa Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 569d2fa955

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ryan-scripts/TUFLOW-python/model_management/verify_converted_tuflow_rasters.py Outdated
@phantom-branch
phantom-branch merged commit 31560a5 into main Oct 14, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants