new features#2
Draft
softlukas wants to merge 7 commits into
Draft
Conversation
softlukas
marked this pull request as draft
July 23, 2026 13:25
added 2 commits
July 23, 2026 18:12
…tion - Add idempotent install/migrate_db.py for Stage 1 schema - Add common/content_storage.py for SHA-256 based content deduplication - Extend etc/config.yaml with content_storage_path, max_storage_size_gb, sandbox_providers - Create /data/url_evaluator/content in prepare_environment.sh - Run migrate_db.py after create_db.sql in configure_sqlite.sh
- Implement SHA-256 based storage path layout. - Add save_content, load_content, content_exists helpers. - Add delete_content helper with empty parent directory cleanup. - Normalise and validate SHA-256 hashes in storage_path.
added 4 commits
July 24, 2026 10:56
- Add common/db_helpers.py with persist_content_snapshot(), set_url_latest_content() and record_url_history(). - Extend bin/evaluator.analyze_content() to save downloaded content, metadata and update content_snapshot/url_content/urls/url_history. - Keep existing VT/MB classification logic unchanged.
- Add record_url_source() and record_discovered_url() helpers to common/utils.py - Update process_new_session() to populate source_detail, origin_url, observed_at, idea_id and session_hash in url_source and discovered_at in discovered_urls - Update bin/honeynetasia2evaluator.py to record observed_at via record_url_source() - Update bin/tpot2evaluator.py to pass sensor/host as source_detail - Update bin/warden2evaluator.py to pass event metadata through process_new_session() - Ensure migrate_db.py creates unique index on url_source(url, source) for upserts
- Add global search box and bulk URL textarea to list page - Show observations, content history, HTTP headers, change history on detail page - Add download blob and request-sandbox stub routes - Make MISP optional for dev/local runs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.