Skip to content

refactor(eval)!: remove batch insights and online insights - #2576

Merged
jariy17 merged 2 commits into
refactorfrom
chore/remove-insights
Oct 12, 2026
Merged

jariy17 merged 2 commits into
refactorfrom
chore/remove-insights

Conversation

@jariy17

@jariy17 jariy17 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

What: Removes batch insights and online insights from the CLI.

  • agentcore eval batch-insights run|get|list|stop
  • agentcore eval online-insight create|get|list|update|pause|resume|delete
  • agentcore add online-insight / agentcore remove online-insight

Why: Standalone insights will be removed for now.
How: Removed all handlers, core clients and tuis that are part of those.

Link: https://github.com/aws/agentcore-l3-cdk-constructs/pull/420

jariy17 added 2 commits October 12, 2026 18:17
Remove the eval batch-insights and eval online-insight command groups, the
add/remove online-insight project resource, their TUI screens and pickers, and
the insights and clusteringConfig fields of online evaluation configs, which
now require at least one evaluator.
Evaluators became mandatory once insights were removed; mark the flag
required so a missing value fails at parse time with a clear message.
@github-actions github-actions Bot added the size/xl PR size: XL label Oct 12, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Oct 12, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 12, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Reviewed this as a mechanical removal of the batch-insights and online-insight surfaces on the unreleased refactor branch. The deletions are consistent across the stack:

  • Routes and screen imports in src/components/Root.tsx are removed in lockstep with the deleted screens.
  • createOnlineInsightHandler / createBatchInsightsHandler are removed from src/handlers/eval/index.tsx and the add/remove routers; corresponding entries in supportedTuiCommands, AddResourceInput, RemoveResourceInput, RemovableResourceType, and ChinaAddKind (via the union) all fall out cleanly.
  • EvalClient and TestEvalClient lose the insight methods, and the inputs/responses are removed from src/handlers/eval/types.tsx; FilteredPaginator is still used by src/core/gateway.tsx, so the import removal in src/core/eval.tsx is correct.
  • The schema change in src/projectSchemas/online-eval-config.ts (dropping insights/clusteringConfig, making evaluators required with min(1)) is matched by the test updates and by createAddOnlineEvalHandler marking --evaluators required. Existing spec files that only had insights will now fail validation, which is the stated intent for the unreleased branch.
  • No stale references to online-insight / batch-insights / clusteringConfig remain in src/ (the only hits are literal substrings inside unrelated batch-evaluation fixture names, and the surviving CloudWatch Logs Insights path called out in the PR body).

No telemetry concerns (feature removal) and no new mocking patterns introduced. Safe to merge.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Oct 12, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.52%. Comparing base (5d81751) to head (4a2a4aa).

Additional details and impacted files
@@             Coverage Diff              @@
##           refactor    #2576      +/-   ##
============================================
- Coverage     97.54%   97.52%   -0.02%     
============================================
  Files           657      634      -23     
  Lines         48779    47260    -1519     
============================================
- Hits          47580    46090    -1490     
+ Misses         1199     1170      -29     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Oct 12, 2026
@jariy17
jariy17 marked this pull request as ready for review October 12, 2026 19:10

@nborges-aws nborges-aws left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One nit, command.md needs to be updated as well to remove the insights commands and flip evaluators to be required

@jariy17
jariy17 merged commit e5a71d6 into refactor Oct 12, 2026
29 of 30 checks passed
@jariy17
jariy17 deleted the chore/remove-insights branch October 12, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants