feat(data-quality): add W65 bounded monitoring worker

[ammarheidari]

Authority

W65 / #214 Slice 4B durable lifecycle/evidence persistence is protected-main admitted via PR #253 at baseline 8d066a2dacdbf0c647d5be81dc61eea8c8680e34.

Slice 4C scope

• bounded monitoring cycle executor;
• ACTIVE-policy enumeration only;
• explicit per-policy/partition authorization port before any Kafka read;
• existing IKafkaRecordReadPort reuse with no consumer-group membership or offset commits;
• timestamp-anchored forward reads aligned to the policy evaluation window;
• policy-owned record/rate/byte/time ceilings intersected with server worker ceilings;
• bounded total clusters/policies/evaluations per cycle;
• aggregate/progress-only persistence through IDataQualityLifecycleStore;
• source/evaluation failures do not fabricate successful evidence;
• cancellation propagates; no unbounded retry loop.

Safety

• no authorization bypass;
• no direct Kafka client in worker;
• no raw payload durability;
• no hidden consumer group;
• no offset commits;
• no arbitrary predicate/runtime;
• no API/UI activation yet;
• no silent cluster truncation: cycle requests exceeding worker cluster ceiling are rejected.

Final corrective exact head: 40e447c4a503bacde36d1167f14271f5c33ac9f9.

Refs #214 #209.

Corrective closure

• source read budget outcomes now propagate into durable evaluation state/outcome instead of fabricating complete evidence;

• policy throughput allowance is shared across all targeted partitions in a cycle;

• records outside the captured time window are excluded conservatively and missing timestamps produce partial uncertainty rather than false completeness;

• cancelled Kafka read results propagate as cancellation rather than normal source failures.

• empty truncated reads now preserve a conservative lower/continuation-bound range instead of collapsing to [high, high);

• future/out-of-window timestamps are treated as uncertainty across the full returned batch rather than a monotonic end boundary.

• active-policy pagination now rotates from the last processed policy instead of permanently restarting from the first page;

• shared policy allowance is fairly divided and partition processing resumes round-robin across cycles;

• returned offsets beyond a stale captured high watermark are included in the admitted range;

• complete empty reads preserve the resolved forward anchor from the source batch.

• policy cursor no longer advances past a policy that performed no partition consideration because the global evaluation ceiling was already exhausted;

• cluster order now rotates across cycles and a cluster cursor advances only after real consideration (or a genuinely empty page), preventing later-cluster starvation under the global evaluation ceiling.

• exact worker-owned deadline_exceeded read failures now persist zero-counter Unknown/DurationLimit evidence instead of being misclassified as ordinary source failures; Kafka-native timeout codes remain source failures.

• rotating partitions now receive the full remaining policy byte capacity (bounded by the per-evaluation hard ceiling), avoiding permanent starvation of records larger than a proportional byte share.

• empty byte-limited reads now retain the current partition as the next rotation cursor and stop that policy cycle, so the next cycle retries it first with the full policy allowance instead of wrapping back to an earlier partition.

• empty byte-limited reads retain the current partition only when the attempt was temporarily underfunded below the full policy/server byte ceiling;

• a full-ceiling oversized record rotates onward so later partitions are still monitored;

• timestamp uncertainty no longer aborts inspection of the remaining already-bounded batch; later in-window records are evaluated while evidence remains partial.

• rebased onto protected-main cb2519bc853ed21da5b3e310530c6d062324ca20 after W67 Slice 3 admission;

• permanently oversized full-ceiling reads persist their returned continuation offset and resume beyond the blocked record on later cycles;

• stale resume offsets are discarded without fabricating source failures, returning later cycles to timestamp-window resolution.

• rebased onto protected-main 67edb7950d09020bbb03e5441df932ce7b0f0a0f after W66 webhook runtime admission;

• policy rate limits are now enforced by a rolling one-second ledger shared across partitions and immediate cycles;

• concurrent cycles on the same worker are serialized so they cannot double-spend the same rate allowance;

• resumed reads advance their continuation on truncated success and clear it on completion.

• regression expectations were aligned with the admitted rolling-rate semantics: immediate cycles do not receive fresh per-policy allowance; fairness is tested after the one-second rolling window releases capacity.

• rolling rate history is keyed by stable policy identity so a definition/version replacement cannot reset the per-second allowance;

• a retained resume that provably returns only timestamped records older than the current evaluation window is cleared and not persisted as a fresh window evaluation; subsequent cycles re-resolve from the timestamp window.

• byte-limited empty reads now conservatively charge one attempted record and the entire remaining policy byte allowance for the rolling window;

• truncated resumed reads advance from LastReturnedOffset + 1 when the adapter omits NextAnchor at the captured high watermark.

• every successfully persisted truncated read now installs/advances continuation from evaluated progress, including initial timestamp-anchored reads;

• continuation is bounded by source continuation and therefore never skips source-unread data;

• evaluator-returned cancellation is propagated before durable evidence persistence.

• rebased without semantic changes onto protected-main 51d9c109f2c9a6fb429e8638a5dd9ba321884ac3, whose only intervening changes were workflow action pin updates.

• rebased without semantic changes onto latest protected-main 5379d824661928770a00f6b3f7861ff18b6b0e14 after Dependabot PR chore(deps): Bump @tabler/core from 1.5.1 to 1.6.1 in /src/frontend #259 (@tabler/core 1.6.1); W65 files are unchanged semantically.

• timestamp-uncertain records advance continuation only across a safely inspected prefix before the first accepted record; a fully filtered bounded page therefore progresses, while accepted but unevaluated records are never skipped.

• worker-owned deadline_exceeded reads now conservatively charge the current read budget against the rolling policy allowance before persisting Unknown/DurationLimit evidence;

• source truncation no longer overwrites an evaluator outcome that already stopped earlier for Duration/Record/Byte limits.

• source truncation is now applied only after every admitted record was actually evaluated; if the evaluator stopped earlier, its limiting outcome is preserved.

[ammarheidari]

@codex review

Please review exact head b0583a4af88640d0f39b34a215ffeb11c7f4da0d only. Focus on authorization-before-read, bounded scheduling semantics, timestamp/window correctness, policy/server budget intersection, source-failure truthfulness, cancellation, and proving the worker introduces no Kafka consumer-group/offset-commit or raw-payload durability path.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-05T11:17:55.403655Z	40e447c	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0583a4af8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review corrective exact head 6c2546e9fc6d76564b3e37b222c03e2b8c15bc3c only. Verify all four prior findings are closed: truthful partial source outcomes with unread-range progress, policy-level throughput sharing across partitions, conservative window-end/missing-timestamp handling, and cancellation propagation from failed Kafka results.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c2546e9fc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final corrective exact head 67d4803cff6ff631a0e576754d600e00b43bcac7 only. Verify all prior findings are closed, including preserving the start of empty truncated reads and treating future/out-of-window producer timestamps as partial uncertainty without assuming timestamp monotonicity by offset.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67d4803cff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head 92a6062fc70d9269ae25c4062e0a561f2ef53226 only. Verify every prior finding is closed, especially fair rotation through truncated policy pages and partitions, stale-high-watermark range admission, complete-empty resolved-anchor preservation, source partial truth, and cancellation propagation.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 92a6062fc7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head 4ef1c7061c5ef74289cc237be72f95affbb63477 only. Verify all prior findings are closed, especially that an untouched policy never advances the pagination cursor and that clusters rotate fairly across cycles when the global evaluation ceiling is exhausted.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ef1c7061c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head 7dfbd6f9ef5d28027a1395c116048b5fabc800ef only. Verify every prior finding is closed, including policy/cluster fairness and that the worker-owned deadline_exceeded condition is represented truthfully as Unknown/DurationLimit evidence without hiding ordinary Kafka timeout failures.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7dfbd6f9ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head 6c693636be300a4ae24ed2f11f790749fdf9cf5f only. Verify every prior finding is closed, especially that rotating partitions receive enough of the remaining policy byte allowance to avoid permanent oversized-record starvation while total policy byte consumption stays bounded.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c693636be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review rebased final exact head 36838c018cdf555bb7c099f38451421e205d9e8e only. Semantic worker/test content is unchanged from previously clean head a87ca44fccbd4b6ec50cde5102b6d7b06fb4cbef; verify no regression from rebasing onto protected-main workflow-pin updates.

[chatgpt-codex-connector]

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Retry review for exact head 36838c018cdf555bb7c099f38451421e205d9e8e. The previous attempt failed with an internal Codex service error and produced no findings. Please review this exact head only; semantic worker/test content is unchanged from previously clean a87ca44fccbd4b6ec50cde5102b6d7b06fb4cbef.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 36838c018c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review latest exact head c205bbc1c0581896d2d36b72835ab024fa4681f0 only. Semantic worker/test content is unchanged from previously clean 36838c018cdf555bb7c099f38451421e205d9e8e; this head only rebases onto protected-main 5379d824661928770a00f6b3f7861ff18b6b0e14 after frontend dependency updates.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c205bbc1c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head 11da7ac360dc6f2897cbc4bd13cc850998732b0c only. Verify all prior findings are closed, especially safe advancement across fully inspected timestamp-uncertain prefixes without skipping accepted-but-unevaluated records, plus all existing rolling-rate, continuation, fairness, cancellation and evidence-truth invariants.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11da7ac360

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ammarheidari]

@codex review

Please review final exact head d9a60d4a3c0845bbba4eb72f1737686654c7cb76 only. Verify all prior findings are closed, especially conservative rolling-rate charge on worker-owned deadline_exceeded reads and preservation of an earlier evaluator limiting outcome when the source batch is also truncated.

[ammarheidari]

@codex review

Please review final exact head 40e447c4a503bacde36d1167f14271f5c33ac9f9 only. Verify all prior findings are closed, especially deadline-read rate charging and applying source truncation only after every admitted record was actually evaluated, while preserving all existing continuation, rolling-rate, fairness, cancellation and evidence-truth invariants.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 40e447c4a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".