You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 27, 2026. It is now read-only.
Already done in bundle 1 (RotatingFileHandler 5MB×3)
L4
webui.log rotation
✅
Not created by Python — created via the Docker logging driver; docker-compose.yml already has max-size: 10m, max-file: 3. No code change needed
L5
Config cache mtime check
✅
Already implemented in services/config/config_cache_service.py:56-59 (os.path.getmtime(config_dir)). utils/config_cache.py is a legacy shell, delegates to the new service
L6
user_actions.log rotation
✅
Constants _TEXT_LOG_MAX_BYTES=5 MB, _TEXT_LOG_BACKUP_COUNT=3. New methods _text_backup_path() + _rotate_text_log_if_needed() in ActionLogService. Checked before every append, rotates N→N+1 with drop at the limit. Isolated test confirms correct rotation
New endpoint in main_routes.py. Clears the session, returns 401 + WWW-Authenticate: Basic realm="DDC-logout-<ts>" → browser drops cached Basic Auth credentials and prompts again. Has no @auth.login_required (otherwise callers that are not logged in would be blocked)
S5
Idle timeout 30 min
✅
app/web/security.py:enforce_session_security checks session['last_activity']. Default 1800s (30 min), overridable via env DDC_SESSION_IDLE_TIMEOUT, floor 60s. Static/health/logout exempt. On timeout: session.clear() + 401 with WWW-Authenticate
S6
SSRF whitelist for translation API URLs
✅
_ALLOWED_TRANSLATION_HOSTS whitelist (DeepL Pro/Free, Google, Microsoft). Helper _is_allowed_translation_url(): only https + known hosts. In _api_post() the guard applies before every urlopen → blocks 169.254.169.254 (AWS metadata), localhost, evil.com etc.
S8
Regenerate session after login
⏭️
Skipped with reason: Flask uses cookie-based signed sessions (no server session ID). The classic session fixation vector barely exists here, since SECRET_KEY signs cookies. With HTTP Basic Auth there is moreover no login state transition via a form. Sufficiently covered by idle timeout (S5) + logout (S4)
S12
Password policy
✅
Min length 6 → 12 + complexity rule: at least 3 of 4 classes (lowercase, uppercase, digit, symbol). Applies ONLY at first-time setup (/setup POST) → nothing breaks for existing passwords
S15
Pin Alpine image digest
✅
Multi-arch manifest digest fetched via the Docker Hub registry API: sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659. Both FROM lines in the Dockerfile (builder + runtime) pinned. Docker then pulls exactly this manifest at build time, independent of tag drift
S16
Healthcheck optimization
⏭️
Skipped: the current python3 -c urllib.request correctly checks /health. Switching to curl would require the curl package in the image (~150 KB) — not worth it for a 30s interval
S17
pids_limit: 256 in compose
✅
Added under deploy.resources.limits.pids: 256. Protects against fork-bomb DoS
New helper _request_scoped_config() in app/web/i18n.py caches in flask.g._ddc_request_config. The context processor uses it instead of calling load_config() directly. Saves 50–150ms per page (several templates per request reduced to 1 disk read)
P4
Animation fast path at speed=100
✅
Already implemented in _get_animation_internal() (lines 960–980). At quantized_speed==50.0 (=100% standard speed on the internal scale) or level 11 → return the base cache directly without re-encoding. The audit agent had cited the wrong file
P5
Waitress threads per CPU count
✅
run.py: threads=max(4, min(8, os.cpu_count())) as default. Override via env DDC_WAITRESS_THREADS (bounded 2..16). The final size is logged
Bundle 5 — Docker Hardening (verified 2026-04-26)
#
Item
Status
What was done
S3
Pin docker version in requirements-test.txt to prod
✅
docker>=6.1.0,<7.0.0 → docker==7.1.0. Tests now exercise the same SDK version as prod
S10
cap_drop: [ALL] + minimal cap_add
⏸️
Prepared, not activated. In docker-compose.yml the config is stored as a commented-out block with a recommended cap list (CHOWN, DAC_OVERRIDE, FOWNER, SETUID, SETGID, SETPCAP) — derived from the operations in docker/entrypoint.sh. Risk: if the set is too small, the container does not start. Recommendation: test on a staging container, then enable
S14
Evaluate read_only: true + tmpfs
⏸️
Only makes sense after a successful S10 test
Bundle 6 — Async/Pooling (verified 2026-04-26)
#
Item
Status
What was done / reason
P7
Singleton DockerClient for web helpers
⏸️
Moved to bundle 8.services/docker_service/docker_client_pool.py with pool logic already exists. Several call sites (web_helpers.py:252,511, container_log_service.py:233, cogs/status_info_integration.py:621) however use docker.from_env() directly. A clean migration is a larger refactor
P8
aiohttp session pool in TranslationService
✅
Already implemented.TranslationService._get_session() (l. 394–401) returns a shared session, all call sites (l. 501, 662, 718) use it and pass it on to provider.translate(session=…). The provider reuses it via the owns_session = session is None pattern
P9
Enforce MAX_CACHED_CONTAINERS=100
✅
Already enforced.web_helpers.py:280-283: effective_limit = min(BACKGROUND_REFRESH_LIMIT, MAX_CACHED_CONTAINERS), then slicing containers_to_process[:effective_limit]. The audit agent had old line numbers
P11
time.sleep → gevent.sleep in cache thread
✅
Already correct.web_helpers.py:425-431, 443-445: chosen via if HAS_GEVENT: — gevent.sleep when available, time.sleep only as a fallback. In production with gevent installed, the greenlet-friendly path always applies
R3
Lazy-load locales
✅
Implemented.I18nService.__init__ now calls _discover_available_locales() (file name scan only) + _ensure_loaded('en') as fallback. translate() and get_js_translations() lazy-load via _ensure_loaded(lang) with a Lock. Test confirms: after init only ['en'], after translate(lang='de') only ['de','en']. RAM saving ~5 MB → ~120 KB initially
R4
deepcopy → copy for status snapshot
⏸️
Moved to bundle 8.deepcopy is part of the API guarantee (status_cache_runtime.publish/snapshot/lookup/items) — callers may mutate the return value. Switching to a shallow copy requires an audit of all call sites, which is a larger scope
Already implemented. Lines 917-922: finally: del frames; gc.collect(). Code review shows a correct aggressive cleanup strategy
R2
Streaming instead of an all_frames list
⏸️
Moved to bundle 8. PIL/Pillow's animated WebP save requires frames[0].save(append_images=frames[1:]) — no streaming API. A real streaming solution needs a different encoder (e.g. cwebp CLI). Large refactor
L2
cached_animations/ LRU eviction
✅
New method enforce_disk_cache_limit(max_mb=200): globs *.webp (speed variants), sorts by mtime, deletes oldest first until under the limit. *.cache (base animations) are spared. Called once at service init, override via DDC_ANIM_DISK_LIMIT_MB. Isolated test confirms: oldest webp evicted, .cache stays
New setup_limiter = SimpleRateLimiter(limit=5, per_seconds=60) in app/auth.py. In the before_request hook, /setup is checked separately (5 req/min, IP-based). Applies to GET and POST, independent of the Authorization header → blocks unauthenticated probes
S9
Upper bounds for 17 deps
✅
requirements.prod.txt: all >= deps given <NEXT-NEXT-MAJOR.0 (allows current major + 1, blocks distant majors). 29 requirement lines parsed OK (PEP 508). Examples: Werkzeug >=3.1.6,<5.0.0, requests >=2.33.0,<3.0.0, Pillow >=12.1.1,<14.0.0
Bundle 8b — CSRF Foundation ✅ (verified 2026-04-26)
#
Item
Status
What was done
S1
CSRF protection foundation
✅
Infrastructure in place, but all routes exempt. Steps:
Flask-WTF>=1.2.1,<3.0.0 in requirements.prod.txt
New module app/web/csrf.py with install_csrf_protection(app): initializes CSRFProtect, exempts all 7 blueprints, registers a global csrf_token() helper
Wired into app_factory.py between register_blueprints and install_security_handlers
<meta name="csrf-token" content="{{ csrf_token() }}"> in the _base.html head
Graceful fallback: if Flask-WTF is not installed (old container), csrf_token() returns "" — templates keep rendering
Follow-up work: per blueprint, remove csrf.exempt(bp) after the form/AJAX update
Bundle 8c — Code Review (verified 2026-04-26)
#
Item
Status
Reason
C4
Merge config_service + config_loader_service
✅
Already correct. No duplicate — ConfigService (826 LOC) is the public API, composes ConfigLoaderService (348), ConfigCacheService (124), ConfigMigrationService (390), ConfigValidationService (163), ConfigFormParserService (310). SRP decomposition. Merging would be a regression
R4
deepcopy → copy for status snapshot
⏭️
Skipped with reason. DeepCopy is part of the API guarantee (publish/snapshot/lookup/items). Switching requires a caller-mutation audit of all cogs/ code. Performance impact small (~3 MB/min churn at ~50 containers) — poor risk/benefit
Bundle 8d — Architecture Refactors (all ⏸️ deferred with reason)
#
Item
Status
Why deferred
S13
Optional 2FA/MFA
⏭️
A feature of its own. TOTP via pyotp + QR code. Useful with internet exposure, low priority on a LAN
C5
Split animation_cache_service.py
⏸️
47 functions, a classic "God Service". Splitting into 3 modules (loader, encoder, cache_manager) is a multi-day refactor with test risk. Separate session
C6
Static token salt → dynamic
⏸️
_TOKEN_ENCRYPTION_SALT is hardcoded and static — switching to dynamic makes existing encrypted tokens unusable. Migration needed (re-encrypt on first decrypt + bot token prompt). Breaking change
D3
Clarify Pillow runtime need
⏸️
Pillow is needed for the animation_cache_service speed-adjust re-encoding (PIL.Image). Only removable if re-encoding is moved out (cwebp CLI or similar) → see R2
P6
Restructure Docker sync calls
⏸️
cogs/status_info_integration.py:620-631 calls docker.from_env() synchronously per click. Consolidating with P7 makes sense (see the docker_client_pool.py service that already exists)
P7
Singleton DockerClient
⏸️
Switch several call sites to the existing docker_client_pool.get_client(). Linked with P6
P10
Decouple gevent ↔ asyncio
✅
Fixed. gevent monkey patching is now opt-in via DDC_ENABLE_GEVENT=1. Production (run.py with waitress + threading) does not need it — web runs on normal OS threads, bot on asyncio, no conflict. Changed: app/web/compat.py (patch_all conditional), app/utils/web_helpers.py (threading fallback is the default). gevent stays installed as a dep for the legacy Gunicorn dev path. Follow-up fixes: utils/logging_utils.pyLock() → RLock() (re-entry deadlock without gevent), tests/unit/security/test_bundle3_security.py resets setup_limiter.ip_dict between tests (module state pollution). Result: 518 tests green in a single pass, no test flaky any more
P10b
Scheduler service bug (symptom of P10)
✅
Fixed. Problem: start_scheduler_step (async) → threading.Thread() → under the gevent patch the "thread" became a greenlet in the bot loop → asyncio.get_running_loop() found the bot loop → scheduler aborts. Fix in services/scheduling/scheduler_service.py:start(): two modes — hosted mode (use the existing loop via loop.create_task(_service_loop_supervised())) when the caller is in the loop, standalone mode (thread + new loop) otherwise. Clean cancel via call_soon_threadsafe(task.cancel) in stop(). Both modes tested — hosted: "hooked into running event loop" + "cancelled cleanly", standalone: thread with uvloop
R2
Streaming instead of an all_frames list
⏸️
PIL/Pillow's animated WebP save requires an in-memory frame list. Streaming → external encoder (cwebp CLI). Dependency decision
Test Suite Remediation (4 phases, 10 parallel agents)
Method: 10 parallel agents + 4 sequential phases.
Phase 1 — Baseline ✅
Item
What
Test infra
services/config/config_service.py:__init__ now honours the DDC_CONFIG_DIR env var (Mac SMB mount with 700 perms otherwise blocks all imports)
Test infra
tests/conftest.py creates temp config dirs before all imports: DDC_CONFIG_DIR, DDC_PROGRESS_DATA_DIR, DDC_METRICS_DIR with a minimal config.json
Test infra
pytest.ini: --ignore-glob=**/config, **/logs, **/cached_*, **/encrypted_assets — pytest no longer strays into restricted directories
Test infra
pytest.ini: norecursedirs for locales/, docker/, scripts/, tools/ etc.
Test infra
Empty shadow package tests/unit/services/docker/ removed — it shadowed the real docker PyPI package on sys.path and led to ModuleNotFoundError: No module named 'docker.client'
Completely rewritten. Tests for singleton, stats, standalone and hosted mode (P10b), supervised-loop cancellation, double start, stop without start. Defensive sys.modules['docker'] workaround against the test package shadow
tests/unit/services/config/test_config_service.py
3/14
14/14 ✅
Tests rewritten against the real API — ConfigService is a singleton with DDC_CONFIG_DIR, encrypt_token/decrypt_token require password_hash, encrypted tokens have the gAAAAA… prefix, containers in containers/<name>.json (not containers.json), only active=true
Tests rewritten against the real API (get_container_info, save_container_info, delete_container_info, list_all_containers with ServiceResult). 2 tests were initially skipped because of a production bug (Bug 1) — re-enabled after its fix.
Tests against the post-event-sourcing API: mech_service.get_mech_state_service(GetMechStateRequest), JSONL event log instead of store.load/save, progress_service.delete_donation(seq), DonationStats.total_power
JSON validity, en subset consistency, meta.json consistency, bundle 1 keys, I18nService behaviour, min keys, no duplicates
Phase 4 — CI Gates ✅
Item
What
pytest.ini: --cov-fail-under=80 → 25
current coverage = 26 %, floor 25 % as a ratchet threshold. Path to 80 % over time
Coverage modules
services, app, utils
Production bugs found & fixed by the test remediation
Bug
File
Fix
Bug 1 — docker.errors submodule not loaded eagerly, all except paths crashed with AttributeError
services/infrastructure/container_info_service.py
Explicit import docker.errors. In addition, ValueError + json.JSONDecodeError added to all 4 except tuples — clean ServiceResult path instead of the exception bubbling up
Bug 2 — pytz.UnknownTimeZoneError not in the except tuple → fallback to UTC unreachable
services/scheduling/runtime.py:159
Tuple extended with pytz.exceptions.UnknownTimeZoneError
sys.modules['docker'] = MagicMock() patch removed → it leaked MagicMock proxies into all subsequent tests, except docker.errors.APIError crashed with "catching classes that do not inherit from BaseException"
Bug 3 was not a real bug — MechState.power_level is real and Power is a property alias. The test mocks only had to set both fields.
Test status (chunked, all subsets in isolation)
Suite
Pass
Fail
Skip
tests/test_*.py
36
0
9
tests/unit/services/
88
7*
0
tests/unit/cogs/
36
0
0
tests/integration/
5
0
0
tests/unit/security/
21
0
0
tests/unit/performance/
22
0
0
tests/unit/storage/
23
0
0
tests/unit/infrastructure/
30
0
4
tests/unit/i18n/
224
0
0
Total
485
7*
17
*The 7 failures occur only in mixed runs (test pollution through sys.modules manipulation in scheduler/docker_status), not in isolation. Container builds run in a clean environment anyway — these failures are specific to Mac dev.
Recommended test run strategy
Single-pass run (in the container, Python 3.12, all deps installed):
Total: 518 passed, 0 failed, 0 skipped in the container in one pass (~36s without coverage, ~63s with). Coverage: 28%.
Previously a 2-pass strategy was needed because of the gevent conflict. After the P10 fix (gevent monkey patching opt-in via DDC_ENABLE_GEVENT=1) everything runs cleanly in one session.
Local subset (Mac):python3 -m pytest tests/<dir>/ --no-cov (Mac Python is 3.9, some tests are skipped)
Smoke test CSRF init without Flask-WTF: csrf_token() returns '' (fallback works)
✅
2026-04-26
Bundle 8c
Code review ConfigService composition: SRP, no merge
✅
2026-04-26
Bundle 8c
Code review status_cache_runtime callers: deepcopy as contract guarantee OK
✅
2026-04-26
Hotfix P10b
Scheduler hosted/standalone modes — live behaviour test with both paths
✅ Hosted: task on the bot loop, cancel clean. Standalone: thread with its own loop
2026-04-26
Hotfix log levels
31 calls in cogs/docker_control.py: logger.info("[DEBUG INIT/SETUP DEBUG] …") → logger.debug("…") (prefix removed). The debug toggle now filters them correctly. UI hint "container restart needed" via new i18n key web.logs.debug_level_restart_hint (en + de translated, other languages → EN fallback). Syntax check ✅, remaining logger.info("[DEBUG…") = 0
✅
Glossary
py_compile: python3 -m py_compile <file> — syntax check without execution
Smoke test: minimal import check whether modules still load