[WIP] CAS draft (adopting to CI/CD, not for review / merge) by filimonov · Pull Request #2073 · Altinity/ClickHouse

filimonov · 2026-07-17T17:36:22Z

Changelog category (leave one):

Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

content addressable storage - draft PR

Documentation entry for user-facing changes

TBD.

Exclude tests:

Regression jobs to run:

github-actions · 2026-07-17T17:37:47Z

Workflow [PR], commit [ec1ee97]

…ent_addressed stateless lanes) Config Workflow check failed with 'Workflows are outdated' for master.yml, pull_request.yml, pull_request_community.yml, release_builds.yml. Regenerated via 'python3 -m praktika yaml'. The regeneration adds the two CAS stateless jobs to the generated workflows: 'Stateless tests (arm_binary, content_addressed storage, parallel)' and 'Stateless tests (arm_binary, content_addressed s3 storage, parallel)' (the rustfs-backed lane). CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=927ea142c9cb14759623861eb004261d0b4b1c8f&name_0=PR&name_1=Config+Workflow PR: #2073 Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

…teless lane The lane's start_rustfs expected a pre-extracted binary at ci/tmp/rustfs and failed on CI runners where nothing provisions it (the workflow wipes ci/tmp on every run). Download the static musl build for the runner architecture from the RustFS GitHub release (1.0.0-beta.9) when the binary is absent, mirroring how setup_minio.sh downloads minio/mc. Validated locally: the beta.9 binary passes the conditional-operation semantics the CA pool requires (second 'If-None-Match: *' PUT -> 412, wrong-etag conditional DELETE -> 412, right-etag DELETE succeeds), and download_rustfs provisions an executable binary end-to-end. PR: #2073 Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

Fast test fails at cmake generation: 'Target "dbms" links to ch_contrib::crc32c but the target was not found' — the fast-test job initializes a limited submodule list that does not include contrib/crc32c, so the unconditional add_contrib is skipped while the dbms link line still references the target. The dependency is dead: it was wired in for per-block CRC32C in the early CAS run-file format (5f1272c), which was later replaced by the text record-stream codecs; no source file includes the library today. Restore the pre-CAS state: crc32c is built only for google-cloud-cpp, and dbms does not link it. CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=835251f81cb5af73ad9eaa3a835f50f0c8b678db&name_0=PR&name_1=Fast+test PR: #2073 Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

Fast test builds without SSL and failed on the unconditional 'openssl/evp.h' include in CasBlobHashingWriteBuffer.cpp. Wrap the OpenSSL-backed Sha256 hashing write buffer and the one-shot digest in '#if USE_SSL'; on non-SSL builds selecting blob_hash = 'sha256' now fails closed with SUPPORT_IS_DISABLED. CityHash128 and XXH3-128 blob hashes are unaffected. PR: #2073 Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

… test regression) CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=aeb13b24394023fa8cd9d310d4cbcbc308380af1&name_0=PR&name_1=Fast+test PR: #2073 A CAS parser commit grouped `RELOAD_DICTIONARY`/`RELOAD_MODEL`/ `RELOAD_FUNCTION` with `CONTENT_ADDRESSED_GARBAGE_COLLECTION` into a format case that prints only the optional disk, dropping the reload targets: `SYSTEM RELOAD MODEL my_model` formatted as `SYSTEM RELOAD MODEL` (failed 04117_parser_system_query_variants and 04124_parser_system_query_extra in Fast test). Fold all four types back into the generic target-printing case (table / target_model / target_function / disk else-if chain) — for the CA GC command the disk branch produces the identical output. Both stateless tests verified locally via clickhouse-local against their references. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

…als (arm_tidy, T13 batch 1) CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=aeb13b24394023fa8cd9d310d4cbcbc308380af1&name_0=PR&name_1=Build+(arm_tidy) PR: #2073 Removes default arguments from all virtual/override methods flagged by `google-default-arguments` (147 sites: `CasBackend.h` interface, `IObjectStorage.h`/`S3ObjectStorage.h`, all backend implementers, test helpers/fixtures) and adds non-virtual convenience overloads on the base classes that forward the previous default values. Derived classes gain `using` declarations to unhide the base overloads. Qualified parent-implementation calls in test fault backends switched to the explicit 3-arg form — the 2-arg form would now route through the base forwarder and re-enter the derived override virtually (double fault injection; caught by the battery). Bulk edits produced by codex (gpt-5.6-luna) per the T13 brief; overload visibility and qualified-call fixes plus verification by Claude. Battery 919/919 green. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=aeb13b24394023fa8cd9d310d4cbcbc308380af1&name_0=PR&name_1=Build+(arm_tidy) PR: #2073 Semantics-preserving conformance for the remaining flagged classes: readability-container-contains, readability-isolate-declaration, google-runtime-int (AWS SDK retry-API overrides keep `long` with targeted NOLINT — the override contract owns the type), readability-duplicate-include, cppcoreguidelines-init-variables, cert-msc, modernize-raw-string-literal, modernize-use-starts-ends-with, bugprone-empty-catch (comments only — no new behavior), googletest naming, bugprone-argument-comment, bugprone-optional-value-conversion, bugprone-misplaced-widening-cast (CasTypes.h site audited: not a real precision bug — the value is range-validated to 0-5; cast made explicit without value change). CasRefCowMap's own `contains` keeps its `find` with NOLINT (self-recursion). Bulk edits by codex (gpt-5.6-luna) per the T13 brief (.superpowers/sdd/task-13-batch2-report.md); one over-removed include (PartFolderAccess.h) restored and verification by Claude. Battery 919/919. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

…n't abort the server (STID 0883) CI PR#2073 (content_addressed storage lanes) crashed with "Too large size (9223372036854775870) passed to allocator" (LOGICAL_ERROR, server abort) running the regression test 04070_no_crash_extreme_compress_block _size. Root cause: an extreme max_compress_block_size (2^63-1) flows into ContentAddressedTransaction::writeFile's buf_size and, unclamped, reaches the CaContentWriteBuffer base-class allocation (Memory::alloc), where checkSize (>= 0x8000000000000000) fires. The ordinary MergeTree writers clamp compress -block sizes to 256 MiB (MergeTreeWriterSettings::MAX_COMPRESS_BLOCK_SIZE) for exactly this reason; the CAS write path received the value unclamped. Fix: clamp buf_size and adaptive_write_buffer_initial_size to 256 MiB at the CAS allocation site (both CaContentWriteBuffer ctors), mirroring the ordinary clamp. New gtest CasContentWriteBuffer.ExtremeBufferSizeIsClampedNotPassedToAllocator reproduces the exact crash number without the clamp (verified RED) and passes with it. CA gtest gate 1057/1057. CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=latest&name_0=PR Related: #2073

…cle (UNMOUNT/FSCK) Design for STID 3982-3b48 (CI PR#2073). Five parts: (1) MountLeaseKeeper's vanished-backing-store case throws FILE_DOESNT_EXIST (not LOGICAL_ERROR) so a background renewal thread never aborts the server + new CasMountLeaseLost counter; (2) SYSTEM CONTENT ADDRESSED UNMOUNT <disk> (shutdown() the pool, auto-remount on next access) as a clean alternative to rm -rf under a live mount; (3) online SYSTEM CONTENT ADDRESSED FSCK <disk> via read-only runFsck(Pool&); (4) rewrite the no-leftovers test teardown to GC RUN -> FSCK -> UNMOUNT -> rm -rf; (5) rename the offline clickhouse-disks fsck applet to ca-fsck for consistency. Auto-teardown-on-DROP stays out of scope. Related: #2073

…ence Task 7 added the SYSTEM_CONTENT_ADDRESSED_FSCK AccessType but only updated the AccessType.h macro list, not the 01271_show_privileges reference — the test enumerates every privilege, so the new row (right after MOUNT, matching the AccessType declaration order) was missing and fasttest failed with a one-line diff. 01271 was not in the local gtest/FSCK-access test runs, so the gap surfaced only in CI. CI report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=d780fa7473807203916125e96cf73a9774c84464&name_0=PR&name_1=Fast%20test PR: #2073 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HKgdqVjZwkpWPxLyHzduPb

Add two DESIRABLE items under §3 (GC correctness/observability) surfaced by triaging the CAS-s3 stateless run (Altinity PR #2073, run 30019911967): - [RECOVERED-INDEGREE-ATTRIBUTION] the "delete_pending recovered in-degree — structurally impossible … investigate" warning is a false alarm (dedup-adopt vs condemn TOCTOU, spared, no data loss). Downgrade the GC log to a ProfileEvent + Debug and move the real adopt-without-resurrect detector to the writer's edge-commit. - [CONDEMN-GRACE-WINDOW] a cool-down before condemning a just-zeroed blob to kill hot-dedup churn (tiny system-log blobs) at the source; flagged higher-risk (condemn-timing/ack-floor, TLA-gated, protocol-veto). Docs-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…GABRT) CI-confirmed on Altinity PR #2073 (run 30019911967, asan_ubsan CAS-s3 stateless): a mount-lease renewal PUT that times out client-side (ambiguous — may have applied server-side) gets soft-retried with a stale token, mismatches against its own bumped body, and falls through MountLeaseKeeper's classifier (same uuid/epoch, unfenced — none of the 3 cases match) into the base class's generic LOGICAL_ERROR, aborting the server under ASan. A third variant of the STID 3982-3b48 family (parts 1a/1b covered vanished/absent-at-release). Also notes: amd_msan/amd_tsan CAS-s3 stateless jobs in the same run hit the 6h job timeout with zero artifacts — unknown if same crash-loop or a plain hang. Docs-only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The unsharded CA-s3 sanitizer lanes do not fit the 6h GitHub job timeout, and the kill lands after the test loop but before result upload, so the lane reports NO artifacts at all and praktika's aggregate stays RUNNING forever. In run 30203355812 (sha e2d04bf): tsan finished all 10990/10990 tests at 5h59m and was killed 16s later during teardown; msan was at 5824/10990 (53%, steady progress, no hang) when killed; asan_ubsan passed at 5.4h — too close. This is the full explanation of the "msan/tsan CA-s3 cancelled at exactly 6h with zero artifacts" pattern seen in three consecutive runs. Shard asan_ubsan and tsan 2 ways and msan (the slowest, ~11h projected) 3 ways, following the existing "amd_tsan, s3 storage, parallel, N/M" convention; ci/jobs/functional_tests.py already parses the N/M batch token generically. Workflow YAML regenerated with praktika yaml. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=e2d04bfe37eec5af58bde837e1cc89d11016ef6f&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

…irectory The refusal in prepareAdoptFromManifest throws LOGICAL_ERROR, which aborts the whole process in debug/sanitizer builds instead of behaving like a catchable exception, so the EXPECT_THROW form killed unit_tests_dbms on all three sanitizer CI lanes (asan_ubsan/tsan/msan, 0s failures with "Logical error: 'Relink target ... does not address a content-addressed part directory of a live table'"). Split it the same way as the CasWiringOpsDeathTest precedent in this file: EXPECT_THROW stays for plain release builds, and a CasWiringExchangeDeathTest EXPECT_DEATH variant proves the same refusals positively abort under DEBUG_OR_SANITIZER_BUILD. Verified: build_asan runs the death test (1 test, OK), build (release) runs the EXPECT_THROW variant (9 tests from 2 suites, all OK). Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=e2d04bfe37eec5af58bde837e1cc89d11016ef6f&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

The pre-scrape cleanup removed only filesystem_caches/*/status (one level), but dynamic cache disks created by tests nest their path — the binary CA-s3 lane died on filesystem_caches/disks/cache_03517/status with "Cannot lock file ... Another server instance in same directory is already running" (StatusFile.cpp flock, EWOULDBLOCK) raised from FileCache::initialize when clickhouse-local initialized the disk map. Make the glob recursive so any depth is covered, same mechanism as before: removing the file lets clickhouse-local create and lock a fresh inode even when the not-fully-stopped server still holds the old one. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=e2d04bfe37eec5af58bde837e1cc89d11016ef6f&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

…lanes Tag it no-content-addressed-storage: the coverage is the inline content-addressed disk the test creates itself, so it stays meaningful on every ordinary lane. On lanes whose DEFAULT MergeTree storage is content-addressed, system.remote_data_paths (no disk_name pushdown — the applyFilters TODO in StorageSystemRemoteDataPaths.cpp) also walks the huge shared default pool with the whole run's data, and on the S3 (RustFS) variant that walk exceeds the 600s test timeout (recurred in two consecutive runs, plus a "Some queries hung" ride-along). The BACKLOG pushdown item stays open as the real fix; also note the ConnectionGroup Disk-session pressure signal from the same run. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=e2d04bfe37eec5af58bde837e1cc89d11016ef6f&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

Same class as 5f292e0: PreparedPartWrite's one-shot-terminal rejection throws LOGICAL_ERROR, which aborts the whole process in debug/sanitizer builds (Exception.cpp's handle_error_code) instead of behaving like a catchable exception, so CasPartFolderAccess.PreparedPartWriteRejectsASecondTerminal killed unit_tests_dbms on all three sanitizer CI lanes (0s failures). The file's author knew the class (the NETWORK_ERROR/MEMORY_LIMIT_EXCEEDED choices further down cite it) — these two spots were the leftovers: - PreparedPartWriteRejectsASecondTerminal: split per the CasWiringOpsDeathTest precedent — expectThrowsCode stays for plain release builds, a CasPartFolderAccessDeathTest EXPECT_DEATH twin proves the same rejections positively abort under DEBUG_OR_SANITIZER_BUILD. - PreparedPartWriteMoveTransfersTheTerminalDuty: the single moved-from abort check is guarded in place (EXPECT_DEATH forks, so the rest of the test continues in the parent). A sweep of all CAS gtest files for unguarded LOGICAL_ERROR throw expectations found no other gaps (blob_upload_pool, ref_writer, ref_install_safety, upload_detached, upload_fanout all carry guards). Verified: build_asan 39/39 with both death tests OK; build (release) 30/30 with the EXPECT_THROW variants. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=8417137aeb842f463066fd68a75b60ffeae30d63&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

CasRefInstallSafetyDeathTest.DenyGuardStopsAnAllocation dispatched on DEBUG_OR_SANITIZER_BUILD, but DENY_ALLOCATIONS_IN_SCOPE itself is gated on MEMORY_TRACKER_DEBUG_CHECKS (MemoryTracker.h: defined only under !NDEBUG). Sanitizer builds define NDEBUG, so there the guard compiles to static_assert(true) and the death test "failed to die" on all three sanitizer CI lanes — visible verbatim in the CI error output. (These lanes only now reached this test at all: the earlier LOGICAL_ERROR aborts in gtest_ca_wiring / gtest_cas_part_folder_access killed the process before it, so each fix un-shadows the next tail failure.) Gate the death test on MEMORY_TRACKER_DEBUG_CHECKS instead, and drop the throw-only EXPECT_ANY_THROW branch as dead code: MEMORY_TRACKER_DEBUG_CHECKS implies !NDEBUG implies DEBUG_OR_SANITIZER_BUILD, so whenever the guard exists its LOGICAL_ERROR aborts — death is the only observable outcome (the old comment claimed the opposite implication). Verified: build_debug runs the death test (OK, guard fires); build_asan compiles it out, remaining CasRefInstallSafety* 21 tests x5 repeats green. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=07f8398acddff2c676aded5f804f122984845a4a&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

RCA of the fence cascades on the CA-s3 sanitizer lanes (run for 07f8398, msan 2/3 lane): clickhouse-server logged 351 bursts of 'Connection refused' (e.code() = 111) to rustfs at localhost:11121 spread over the whole run — the service stayed up between bursts, so this is fd/accept exhaustion, not a crash. rustfs was launched WITHOUT the open-files-limit raise that start_azurite performs for exactly this failure mode, while the server under parallel sanitizer load holds 10k+ active Disk-group S3 sessions. Most bursts were absorbed by the CAS request retries (max_attempts=16 / 90s deadline); the one at ~02:00-02:01 outlived (a) in-flight writes' 90s budgets (Code 210 UNCERTAIN) and (b) the mount-lease renewal confirm window (TTL 30s), so the mount fence tripped (fail-closed by design) and every writing test failed with Code 668/210 from 02:02:52 until remount completed ~02:05:00, with straggler failures to ~02:10 — ~40 of the run's 66 test failures. Also add rustfs.log to the uploaded artifacts: it was already written to ci/tmp/rustfs.log but never uploaded (azurite/kafka/minio logs are), which was exactly the missing evidence for this class of triage. Report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2073&sha=07f8398acddff2c676aded5f804f122984845a4a&name_0=PR PR: #2073 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKmSZa7T87WbRGKkNkSXky

ilejn · 2026-07-28T21:09:16Z

+
+```sql
+SYSTEM CONTENT ADDRESSED GC RUN [ON CLUSTER cluster_name] [disk_name]
+```


GC DISABLE ?

The S3-budget table claimed two full enumerations of `cas/refs/` per folding round, justified as keeping the cross-check probe non-vacuous. That is stale: `listRefPrefix` is the round's ONE full enumeration and its result is retained for the fold, while the store-quality detector ("probe A") is sampled on a deterministic cadence — `gc_probe_a_period`, default every 16th round — and only then pays for a second enumeration. The five reads of the adopted fold seal per round remain accurate; the code instruments that redundancy itself (`seal_reads` / `redundant_reads`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ons) All items from the T14 review, which found the spec work sound and the documentation wanting. Nothing re-run; this is citation and accuracy work. THREE FIGURES OF MINE WERE WRONG, and the review was right about each: - `CasGcClampSuppressedPasses` is 12, not 4 — I quoted a mid-run tick as if it were the final value. The maximum is now cited to `build/t14_soak2.log` with a rollup in `build/t14_gc_liveness/attempt2_clamp_values.txt`. - S30's `CasRootGet` is 10 -> 74 in the PASSING run; 18 -> 82 belongs to the earlier FAILING run. The reasoning was right and the attribution was not; both places now say which run each pair comes from. - S43 took five attempts, not four. EVIDENCE, where the doc previously pointed at nothing: - Row 12c, the gating row, cited no artifact. Its figures are LIVE-PASS readings taken against a cluster that was torn down undumped, so they are not reproducible from anything — `build/t14_gc_liveness/README.md` now ledgers them as such, per reading, with the times, and states plainly what IS greppable in `build/t14_soak.log` and what is not. Marking them unreproducible is the honest form; implying they could be grepped was not. - Row 14's quotes now resolve: `build/t14_w3_evidence/` carries the 668 refusal line, the key observations, and the run's `report.json`. THE TWO ACCEPTED DEVIATIONS were only in the worklog and are now in the document AND in row 12c's own text: attempt 2's end-capture was a mini-capture rather than the ordered full dump, and the W3 diagnostic cluster was the SECOND specimen lost to an undumped teardown. The consequence belongs to the gating row and is written there — T15's re-validation inherits a hypothesis whose specimen is gone and cannot re-read the original fold — together with the committed remedy (`scripts/run_soak.sh`, `5fefb188426`). ALSO: the truncation of `build/t14_soak2_final_state.log` is stated (606 bytes, ch1 only, ends mid-token) with the per-node counter claims softened to live-pass readings and the row-count oracle kept, since it is exact and unaffected; the third verdict value is explained rather than left contradicting the doc's own two-value rule; the probe-cadence observable gains its forward pointer to [PROBE-A-CADENCE-UNIT] and to T15's re-validation as the decision point; a stale "three of four scenarios" becomes all four; and the UNARMED/GATE-SKIPPED vocabulary is aligned, with the end-of-run banner softened to not-yet-observed — attempt 1 predates it and attempt 2 never reached that path. The `build/` artifacts are gitignored, as every artifact this campaign cites is; they exist on disk where the document points. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UransgqqWwqMX9BBMiWdaD

…wn = transient + precise unhappy-path test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Called the relink protocol a "three-trip handshake", which overstates its network cost: the offer and the confirm are the only requests to the sender. The publish and the promote are the receiver's own writes into the shared pool. That distinction matters when weighing the protocol's complexity against what it buys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…not the regrouping phase Correcting 84eca08, which renamed BOTH `fold_ref_list` sites to `fold_ref_group`. Only the first was right. `ff9f36a056f` did not just rename the phase, it SPLIT it: the old `fold_ref_list` covered the LIST, the grouping and probe A together, while `fold_ref_group` is the grouping alone and says so itself -- "No I/O: the keys are already in hand". A phase that performs no I/O has an empty ProfileEvents delta, so `max(length(ProfileEvents)) > 0` would have kept answering 0 and the test would have stayed red for a second reason. The `cas/refs/` LIST moved into `defer_decision`, which is where the assertion's stated intent now lives. `ref_list_probe`, the other half of the split, is the wrong target for an independent reason: it is sampled on `gc_probe_a_period` (default 16) while this test runs three rounds. The comment now says which phase this must NOT be pointed at and why, because the tempting choice is the one that shares the old phase's name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

DEBUG_OR_SANITIZER_BUILD gates the *DeathTest split added in 15de762; it resolved correctly via a transitive include in both files (confirmed by --gtest_list_tests showing the death arm under asan and the throw arm under release, both before and after this change), but a preprocessor gate this load-bearing should not depend on another header's include list -- add the explicit include per the documented gotcha (docs/superpowers/plans/2026-07-22-pr2073-sanitizer-aborts.md). Re-verified after this change: --gtest_list_tests unchanged on both binaries; --gtest_filter='CasNamespaceLifeId*:CasRefCatalog*' still 51/7 suites (asan) and 51/4 suites (release), 0 failed on each. Style check exit 0, 216 lines (unchanged).

c60911e introduced `UniversePolicy::kDefault == StageA_Suppressed`, which suppresses every destructive GC site until Stage B's Task 7b, and swept every gtest that asserted a physical delete -- but touched nothing under tests/queries/. `05008_ca_gc_snap_prune.sh` was already red (its 40-round poll for `objects_deleted > 0` never terminates under suppression), and `04290_content_addressed_no_leftovers.sh` / `04295_content_addressed_mutation_no_leftovers.sh` failed the same way once run: their `pending_*` drain-to-0 loop cannot terminate under suppression either (condemned entries are re-emitted as `still_retired` every round forever), so both exhausted their bounded retry and exited 1. Restate all three to assert the Stage-A contract instead of the Stage-B one: marking still condemns every dropped object (`entries_condemned > 0`), but nothing is physically deleted (`objects_deleted = 0`), and FSCK correctly recognizes the condemned garbage as `unreachable` (EXPECTED this stage, not an integrity failure) while `dangling` stays exactly 0 (Stage A never loses a reachable object, it only defers reclaiming garbage). 05008's drain-loop comment was also wrong prose: graduation is round-paced (`condemn_round < current_round` in `settleEntry`), not watermark-paced -- `renewWatermarkOnce` no longer gates it. Each restated assertion carries a `STAGE-A RETURN ITEM` marker matching the two c60911e already left in gtest_cas_gc_log.cpp and gtest_ca_wiring.cpp, so a grep finds all five. Also extends the `UniversePolicy::kDefault` comment in CasGc.h (comment-only) to name all five tests that must be restored when Task 7b flips that line -- a marker in a test file alone is how this sweep gap happened in the first place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

… review trigger; the 05008 blocker was never ours Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

pending_condemned is already candidates+retired per its own doc, so summing all three doubles it. Harmless as a >0 signal, but every figure it prints is twice the truth -- including the two this session's diagnosis quoted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…on gap on the production path Ruled while answering Task 3's scope question. The refusal cannot be enforced by a per-write catalog read -- that adds a protocol step to the write path -- so it rides on the universe moving to the catalog, which is Task 4. Recorded as a task step rather than a note, because a citation is not an executor and this campaign has already lost one deferral that way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…kened for Stage-A suppression Requested after the 05008 fix: the flip is not done until the tests that were weakened to survive suppression are strengthened again and observed green. The distinction the step insists on is that a test still passing with its WEAKENED assertion proves nothing about the flip, so each row names what it was weakened to and what it must be restored to. `04290`/`04295` draining to `PENDING = 0` is the end-to-end proof: they are the only tests that watch the whole pipeline reclaim. Also folds in the double-counted `PENDING` gauge, because Step 4 turns it from a `> 0` signal into a drain-to-zero comparison, where being twice the truth stops being harmless. And the exit condition is mechanical: after this task, a grep for the marker must return nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…hree-site recheck CasRefCatalog::createNamespace/completeCreation/reconcileStaleCreator implement spec §3's three-conditional-write sequence on top of Task 2's casUpdate/ casAdmitEntry: catalog CAS insert `{ns, Creating, fresh incarnation, creator}`, publishCkpt's `_ckpt` create, then a `Creating -> Live` CAS that re-presents the creator's admission generation (checked from inside `mutate`, the fence re-check point casUpdate's header doc now states explicitly) and value-CASes the observed entry (the ZombieGoLive guard). Stale-`Creating` reconciliation (reconcileStaleCreator) steals a stalled entry token-exactly, gated on a new cross-process isCreatorFenceTerminal predicate (CasServerRoot) built from writer_epoch plus the existing mount-terminality certificates (probeNonTerminalMountSlots/computeHeartbeatFloor's gc_fenced/clean-farewell), never from CreatorFence::fence_generation (in-process only, meaningless to a different actor). checkPublicationAdmittedOrThrow enforces "Creating forbids publication" at the catalog level; the plan (0cf1135) rules the production ref-write path's own enforcement is Task 4's, once discovery moves onto the catalog — adding a per-write catalog read now would add a protocol step the standing veto forbids. 19 new tests (gtest_cas_ns_creation_lifecycle.cpp + gtest_cas_mount.cpp's isCreatorFenceTerminal battery), red-first-verified for the ZombieGoLive token-exact check by temporarily disabling it and observing the token-stale test wrongly report Live. DeathTest split verified two-sided (--gtest_list_tests on release and ASan) for the three LOGICAL_ERROR preconditions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

…clusively rejected Closes BACKLOG {#ckpt-failed-birth-debris}. commitRefChunk publishes a birth chunk's `_ckpt` (INV-4's genesis-epoch record) before its ref-log PUT, per spec §3's ordering — but nothing deleted it if that PUT then failed, leaving permanent debris under a never-born namespace (namespace cleanup runs only for Removed namespaces) that later makes a drained server root refuse decommission (claimOwnerOrThrow -> CORRUPTED_DATA). Adds a best-effort, never-throws HEAD+deleteExact cleanup on the three failure branches that PROVE this attempt's bytes never landed and never can (the attempt-arm race, occupant-unreadable, SuccessorSeal, and genuine foreign interference) — deliberately NOT on the ambiguous-exception branch (Writing -> Wedged), where the write might still have landed and the `_ckpt` is the only record of the genesis epoch nothing else could recover. Red-first: gtest_cas_ref_wedge_every_attempt.cpp's new BirthCkptIsReclaimedWhenTheGenesisTransactionIsConclusivelyRejected test (genesis birth, successor-seal conflict at sequence 1) failed with the _ckpt still present when the cleanup call was temporarily removed; passes with it restored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

…the ckpt-cleanup argument at the new key shape Both carried from Task 3. The reconciler is tested but has no caller, and this campaign has already found one test hook that no test set. The ckpt cleanup's safety argument is scoped to today's sentinel keying, and this task moves the object the argument is about -- re-derive it, do not assume it carries, because _ckpt has no repair path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ve prose findings The review counted FOUR call sites where my Task 4 obligation said three, so the plan is corrected and told to re-derive rather than trust either figure -- fix round 1 removes one of the four, so the number moves again. D12 is the one worth reading: a comment justified ignoring the persisted fence generation because it 'never reaches the object store', which is false -- Task 2 serialises it. The true reason is that a process-local counter is not comparable across processes, which is a stronger argument than the false one it replaced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ow the coupling, exhaustive switch Review task-3-review.md, C1-C4 (0 critical / 4 important+minor code/test; the 5 prose findings go to the batched documentation pass, not here): C1 — cleanupOrphanedBirthCkptBestEffort() no longer runs from commitRefChunk's `!attempt_armed` arm. That arm makes NO lane transition (unlike the other three call sites, which drive Closed/Faulted), and getGreatestApplied() != candidate_base_id can mean some OTHER append for this table already advanced state -- including the birth itself, whose _ckpt a delete there could then erase. Only two chassert's guarded the other two sub-conditions; a chassert is not a release fail-close. The lambda's own comment and its "THREE branches" count are corrected to match (removing the site makes the count true again, rather than needing a separate restatement). C2 — commits the negative test from the prior (uncommitted) round (BirthCkptOfAnAlreadyLiveNamespaceSurvivesALaterConclusiveRejection) and adds the one still missing: BirthCkptSurvivesWhenTheGenesisTransactionIsAmbiguous, pinning that the ambiguous Writing -> Wedged branch (the one where the bytes MIGHT have landed) never deletes the _ckpt. Both red-first verified: temporarily adding the call to each site made the corresponding test fail with the _ckpt gone; removing it again restored green. The `!attempt_armed` arm (C1) is NOT independently pinned by a test -- see the reply to the reviewer for why: leader_active serializes commitRefChunk per table and nothing on the same leader thread mutates rt->state between the preparation snapshot and the arming check, so the sub-condition the delete's removal protects against appears unreachable via any current path or test hook. C3 — isCreatorFenceTerminal's classification is an exhaustive switch over a new FenceCertificate enum (None/GcFenced/CleanFarewell/SupersededEpoch), not a chain of ifs, mirroring CasPool.cpp's own exhaustive switch over MountPriorState -- with the attribution corrected (claimMount, in this file, only produces that classification; the switch consuming it lives in the caller). C4 — isCreatorFenceTerminal now takes (server_root_id, writer_epoch) as two scalars instead of the whole CreatorFence, so CasServerRoot.h no longer includes Formats/CasRefCatalogFormat.h -- removing a ref-catalog-format dependency from the mount/server-root layer that CasPool.h's 18 including files would otherwise all carry. Verification: release + ASan both green (21/21 new-suite tests, up from 19); full CA gate 1660 tests / 251 suites, 0 failures (was 1658/251, +2 tests matching this round's two committed test rows); style unchanged at 216 lines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

…t unpinned The ckpt-cleanup site count went stale twice in one afternoon -- three, then four by review, then three again once the fix removed the unsafe site -- and the second staleness was inside a plan step written to prevent staleness. That step now states no number and tells its reader to derive one. D11 is closed with a warning not to 'fix' a comment that is now true. Also records that the !attempt_armed arm's non-deletion has no test, why the implementer declined to add a hook for it, and why I endorse that: a hook whose purpose is to construct states the real code cannot reach is a design decision. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Layout::parseNamespaceFileKey's declaration doc (CasLayout.h) and the mirrored comment in CasLayout.cpp said the key "IS one of our namespace files" once it carries the reserved `_files` segment. That is necessary, not sufficient -- Layout::mountpointObjectKey never enforces the `_files` reservation, it only asserts that it holds. Deleted the overclaiming clause; the throw's actual trigger condition (incarnation segment missing/non-canonical/zero) speaks for itself without it. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D1.

…overs The comment justifying "record and continue" for an un-attributable key cited CasRefGc.UnIncarnatedRefKeyAbortsRefFoldingWithoutWedgingTheRound as proof that the GC round does not clear such a key. That test only builds a ref-family key (`cas/refs/<ns>/_log/...`); it says nothing about a `_files`- family key. Narrowed the citation to the ref family, and for `_files` keys stated the actually-verified reason instead: Cas::Gc's fold only LISTs casRefsPrefix(), never rootsPrefix(), so such a key never reaches a round at all. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D2.

NamespaceLifeIdAndRootNamespaceDoNotInterconvert's comment argued "the type declares no conversion operator and RootNamespace's own constructor is explicit, so nothing interconverts". explicit has no bearing on std::constructible_from<RootNamespace, NamespaceLifeId>; the true (and shorter) reason is that no RootNamespace constructor takes a NamespaceLifeId at all. Fixed the test comment, and also the Task 1c brief's own §Interfaces note in the plan doc, which is where the false reasoning originated -- fixing only the inherited copy would leave the next reader of the brief to inherit it again. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D3.

…comment The "AND IT REACHES NO PROSE" paragraph beside kFsckHardFindings' static assert said "two of them were wrong about the exit set on the day this assert was written" -- written before the same round found a third restatement in utils/ca-soak/soak/run.py, so the count had already gone stale once. Dropped the count entirely, mirroring the paragraph immediately above it (which had already dropped its own count for the same reason: nobody keeping a tally of restatements can promise the tally itself will not go stale). Also replaced "the soak harness's docstrings" with "comments and messages" -- two of the four known restatements are an inline comment and operator-facing WARNING strings, not docstrings. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D4.

…summary "a zero exit code from a summary scan is therefore not by itself proof of a clean pool" invited the reverse inference: that a --detail scan's zero exit IS proof. It is not -- CommandFsck::executeImpl never throws on stale_edge in either mode, so a --detail run that counts stale_edge=5 still exits 0. This is why utils/ca-soak/soak/run.py gates on exit_code AND stale_edge_verdict together. Rewrote the caveat to cover both modes and name stale_edge as the reason. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D6.

D7: worstCaseEntryFoldReservationBytes's doc said its cov+nsc sum covers "the two rows one namespace can simultaneously occupy". ns_cleanup_items is keyed per removal (CasFoldSealFormat.cpp), so a namespace removed and recreated more than once carries more than one nsc row -- the sum is an over-estimate per entry, not an exhaustive count of what a namespace can hold. Reworded to say so and stopped quantifying over the total. D9: foldSealFixedBytes's doc said "header + trailer, zero entries" and called the value fixed. encodeFoldSeal writes a meta line (g/pg) between header and trailer that the measured value already includes but the doc omitted; and the value is measured at generation=0/n=0, so it is a floor, not a constant -- larger decimal widths at real generation/entry counts add a few more bytes. Named the meta line and called it a floor. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D7, D9.

…sible" The cas_ref_catalog registry row's comment justified the tight 4 KiB line cap by saying "one entry's record is small and bounded by kMaxNamespaceBytes". The worst case (kMaxNamespaceBytes at 6-byte escaping, plus an escaped server_root_id) still exceeds 4 KiB -- what landed for this was a change of error code to LIMIT_EXCEEDED (CasRefCatalogFormat.cpp's checkLineBytes), not a change to the byte arithmetic. Reworded to describe the actual shipped behaviour: an over-cap entry is refused at encode time, not made impossible. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D8.

… it covers isCreatorFenceTerminal's doc cited probeNonTerminalMountSlots' own discipline as precedent for treating BOTH an absent mount slot and an undecodable body conservatively. Read probeNonTerminalMountSlots itself (CasServerRoot.cpp): an absent slot is treated as "nothing to hold" (not conservative -- it is skipped), while an undecodable body IS pushed as held (conservative). Moved the citation to sit only beside the undecodable-body case, the one it actually supports. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D13.

cleanupOrphanedBirthCkptBestEffort's safety comment argued deletion is safe because "the next successful birth attempt ... reaches publishCkpt's absent-object create path, not a merge onto this attempt's now-deleted body, so nothing is lost by removing it". That reasons about the NEXT birth, not about whether some recovery path still needs the object to ground the namespace -- a different question the sentence does not actually answer. Deleted the clause; the surrounding paragraph's safety argument (every call site drives the lane to a terminal state) does not depend on it. Batched documentation pass, docs/superpowers/cas/deferred-docs-fixes.md D14.

Batched prose-fix pass over docs/superpowers/cas/deferred-docs-fixes.md. D1-D9, D13, D14 fixed in code/docs (each in its own commit, this branch); D10 partially resolved (report text fixed, the test-name half is a code change and stays open, out of scope for this comment-only pass); D12 and D15 were found already fixed by an earlier round and are marked stale rather than re-edited. D11 was already RESOLVED before this pass and is left untouched per its own note. Full pass report: .superpowers/sdd/2026-07-28-cas-ref-chain-stage-b-catalog/prose-pass-report.md (untracked -- .superpowers/ is gitignored except for the one force-added task-5-report.md).

Cost a diagnosis during Task 10c: a positive config reported a missing file inside its own state pool, which reads as a model defect. Re-run on a clean metadir it completes green over 20.7M states. The metadir path carries no run identity, so two invocations of one config delete each other's state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…n-red instead 76ee70d made `05008_ca_gc_snap_prune.sh`, `04290_content_addressed_no_leftovers.sh` and `04295_content_addressed_mutation_no_leftovers.sh` pass under `UniversePolicy::kDefault == StageA_Suppressed` by weakening what they assert: `05008` stopped requiring `objects_deleted > 0`, and `04290`/`04295` stopped requiring the `pending_*` drain-to-0 loop to reach zero and `fsck_unreachable` to read back exactly 0. A test that passes on a weakened assertion is a false green, the same class of problem as a test whose fault never fires — it says nothing about whether Stage B's destructive path will work once enabled. Restore the strict assertions verbatim from `76ee70da4a7^` (the drain-to-zero loops, the poll-for-a-delete loop, and the `.reference` contents), so all three now assert the real Stage-B contract and FAIL for real under suppression. One correction from `76ee70da4a7` is kept rather than reverted: `05008`'s header claimed deletion "waits for the durable watermark floor (advanced by the background renewer)", which is false — graduation is round-paced (`condemn_round < current_round` in `settleEntry`), and `renewWatermarkOnce` does not gate it. Keep that corrected explanation; drop the paragraphs that argued for the weakened Stage-A contract. Register all three as known-red in `tests/broken_tests.yaml` instead, naming the cause (`UniversePolicy::kDefault == StageA_Suppressed` in `CasGc.h`) and its removal condition (Stage B Task 7b flipping that line). `05008` fails via a plain result diff, so its `message` cannot distinguish this cause from an unrelated regression in the same test — say so in the entry rather than implying a narrow match. `04290`/`04295` fail via the drain-loop-exhaustion `exit 1` with a distinctive stderr line, which is narrower but still not a guarantee: any bug that stalls pipeline convergence would print the same line. Verified all three now FAIL on their own merits and are classified `BROKEN` (known fail) by the harness, not silently passing. The `STAGE-A RETURN ITEM` markers stay in the three test files, reworded to point at their `broken_tests.yaml` entries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

…eless assertions The three stateless tests it referenced are no longer weakened (see the prior commit): they already assert the real Stage-B contract and carry known-red entries in `tests/broken_tests.yaml` instead. Rewrite Step 4's table so Task 7b closes them out by removing the three `broken_tests.yaml` entries and confirming the tests pass with no other change, rather than restoring assertions that are already restored. Leave the two gtest rows (`CasGcLog.EmitsStartFinishWithCounts` in `gtest_cas_gc_log.cpp` and the displacement test in `gtest_ca_wiring.cpp`) untouched — those still carry a weakened `EXPECT_GT` and are still Task 7b's to restore. Keep the point that `04290`/`04295` passing with their drain-to-`PENDING = 0` loop intact is the task's end-to-end proof, and add an explicit exit condition: `grep -n "STAGE-A RETURN ITEM"` returns nothing and none of the three `broken_tests.yaml` entries remain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MycaeDz4WDUZkqjEQzH7hH

strtgbb · 2026-07-31T18:19:58Z

            runs_on=RunnerLabels.ARM_SMALL,
            requires=[ArtifactNames.CH_ARM_BINARY_GH],
        ),
+        # Content-addressed disk over S3 (RustFS) as the default MergeTree storage — the real-S3


This comment is overly verbose. I suggest 2-3 lines max. The other comments in this file may not be necessary.

strtgbb · 2026-07-31T18:21:51Z

+        # incarnation pool needs enforced conditional deletes that MinIO OSS lacks; installs only
+        # content_addressed_s3_storage_policy_for_merge_tree_by_default.xml. Covered on amd across
+        # the plain binary and all three sanitizer builds, plus an arm binary run for cross-arch.
+        Job.ParamSet(


Please move the new ParamSets to altinity_jobs.py. This will make the new jobs easier to carry forward into new releases.

# altinity_jobs.py from ci.defs.job_configs import common_ft_job_config class AltinityJobConfigs: ... content_addressed_functional_tests_jobs = common_ft_job_config.parametrize( Job.ParamSet(parameter="amd_binary, content_addressed s3 storage, parallel", ...), # sanitizer shards... Job.ParamSet(parameter="amd_binary, content_addressed storage, parallel", ...), )

strtgbb · 2026-07-31T18:23:01Z

+        return True
+
+    def start_rustfs(self):
+        # RustFS backs the content-addressed-over-S3 pool (M-W D-W8): the incarnation pool needs


This file also has new and very long comments, please revise.

filimonov mentioned this pull request Jul 17, 2026

[WIP] CAS draft (adopting to CI/CD, not for review / merge) #2066

Closed

ilejn reviewed Jul 28, 2026

View reviewed changes

filimonov and others added 8 commits July 29, 2026 15:05

ca: user directives — GC full time accounting (track B); relink Unkno…

eedd55a

…wn = transient + precise unhappy-path test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ca: worklog — directives folded; T14 fix landed; re-review running

f7c048a

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ca: worklog — T14 closed; T15 review dispatched

3ec6735

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ca: worklog — T15 approved; merge sequence set

a6f96d1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ca: worklog — watchdog, woke a16 (2nd monitor miss)

84aee64

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

filimonov and others added 26 commits July 30, 2026 15:03

ca: worklog — prose batched out of fix rounds; the abort class gets a…

7b7a9a8

… review trigger; the 05008 blocker was never ours Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

svb-alt added antalya antalya-26.6 roadmap Key features and improvements for Antalya project labels Jul 30, 2026

strtgbb requested changes Jul 31, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] CAS draft (adopting to CI/CD, not for review / merge) - #2073

[WIP] CAS draft (adopting to CI/CD, not for review / merge)#2073
filimonov wants to merge 3563 commits into
antalya-26.6from
cas-gc-rebuild

filimonov commented Jul 17, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 17, 2026 •

edited

Loading

Uh oh!

ilejn Jul 28, 2026

Uh oh!

strtgbb Jul 31, 2026

Uh oh!

strtgbb Jul 31, 2026

Uh oh!

strtgbb Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

filimonov commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

Exclude tests:

Regression jobs to run:

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilejn Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

strtgbb Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

strtgbb Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

strtgbb Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

filimonov commented Jul 17, 2026 •

edited

Loading

github-actions Bot commented Jul 17, 2026 •

edited

Loading