Skip to content

Clarify restart-safe batches and scripted UI handler tests - #181

Open
Wenjie Fan (gggdttt) wants to merge 1 commit into
mainfrom
gggdttt-nikola-pr132-feedback
Open

Wenjie Fan (gggdttt) wants to merge 1 commit into
mainfrom
gggdttt-nikola-pr132-feedback

Conversation

@gggdttt

Copy link
Copy Markdown
Collaborator

Summary

Follow up on Nikola's review of #132 without reverting its false-positive guards. This changes two knowledge articles, their four AL samples, and the existing fixture selection/context. Article slugs and review skills remain unchanged.

Restart-safe batches

Separate commit cadence, restart safety, and bounded retrieval. Row-count or elapsed-time commits can be valid after complete business units. An atomic checkpoint/processed state is one retry strategy; demonstrably idempotent replay or durable deduplication is another. A missing watermark alone does not establish corruption, and committing does not itself require TOP-bounded retrieval.

Both samples now perform the same one-time credit-limit increase, select the same exact key buffer, and commit after at most 500 selected customers or an observed one-minute threshold. The helper is identical; only durable progress differs. The good sample initializes/reloads state and commits the last processed, not merely selected, key with the corresponding changes. Errors propagate. The time threshold is checked between complete units, not a fixed-duration guarantee. The examples assume one worker and a stable customer set.

The previous UpperCase operation demonstrated redundant replay, not a non-idempotent correctness failure. The replacement makes duplicate increases after a retry concrete; the old sample did contain periodic commits.

Addresses batch cadence, progress and time-based examples, safe replay versus corruption, and aligned samples.

Scripted UI interactions

Restore a paired Confirm/Message flow with the same Boolean outcome assertion. The good test clears LibraryVariableStorage, enqueues the question/reply/message, dequeues and verifies each interaction, and finishes with AssertEmpty. The bad test blindly accepts/dismisses the same interactions. Recommend reusing one handler per type where practical, not as a platform requirement.

Keep semantic reset/capture/assert testing, intentionally optional [SendNotificationHandler(true)], and the separate listed-but-unreached bad handler scenario. Neither missing queue storage nor a legitimate specialized handler is a finding by itself.

Addresses queue-driven expectations, restoring paired Confirm examples, and library guidance and handler reuse.

Deferred: IsHandled policy

The requests to stop introducing IsHandled and sync the event-review guidance with Marko are not addressed here. Scope, exceptions, and alternatives need Marko's confirmation. Existing publisher contracts and carry-over/reset false-positive guards remain intact. This PR does not resolve all nine comments or resolve the old threads.

Validation and limits

  • Frontmatter/structure validator: 0 errors, 0 warnings.
  • Knowledge-index and bounded-retrieval checks: 282 articles and 457 samples round-tripped.
  • Existing fixture preparation: 34 cases across 17 domains; 8 review-contract normalization cases passed.
  • Performance fixture selection now exercises this batch pair; testing retains its handler pair with an explicit interaction contract. Focused static checks confirmed all four current samples and target articles reach neutral per-case requests, batch helpers and Confirm flows match, and existing handler controls remain.

These are static/metadata and fixture-preparation checks, not AL compilation, Business Central runtime execution, or model evaluation. No live engine/model/judge calls or workflow dispatches were made. No Gold entries, engine pins, frozen candidates, or ongoing A/B runs are changed.

Address Nikola's batching and UI-handler feedback on PR #132. Preserve existing false-positive guards and defer the unconfirmed IsHandled policy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f841a18b-535a-498e-96f2-f279b5378da5

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two necessary contract/integration fixes remain before this is mergeable:

  1. Rebase onto current main. evaluation/review-fixtures.json now conflicts with #148's exact-tree coverage. Preserve its seven performance articles, add avoid-commit-inside-loops, and preserve this PR's testing context. The harness currently applies context to the whole domain, so do not attach the credit-limit scenario to every performance case; either rely on the now self-describing sample (the merged 48-case tree prepares cleanly without that context) or add per-article context support.
  2. Align microsoft/skills/review/al-performance-review.md with the revised article. Its targeted cue still only exempts a persisted watermark/state and says to match every full-tail FindSet with periodic commits. The new guidance correctly also exempts demonstrably idempotent/deduplicated replay and says bounded retrieval must not be required solely because a loop commits. Encode those same exclusions in the leaf contract so routing cannot reinstate the false positives this PR removes.

The changed AL samples otherwise match current BC behavior (Commit, TopNumberOfRows, Duration, handler signatures, Library - Variable Storage, and ExpectedConfirm/ExpectedMessage). Branch validators pass (34 fixtures); a conflict-resolved current-main exact-tree preparation passes 48 fixtures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants