Repository navigation
refactor(eval)!: remove batch insights and online insights - #2576
Conversation
Remove the eval batch-insights and eval online-insight command groups, the add/remove online-insight project resource, their TUI screens and pickers, and the insights and clusteringConfig fields of online evaluation configs, which now require at least one evaluator.
Evaluators became mandatory once insights were removed; mark the flag required so a missing value fails at parse time with a clear message.
|
Claude Security Review: no high-confidence findings. (run) |
There was a problem hiding this comment.
AgentCore Harness Review
Verdict: Looks good
Reviewed this as a mechanical removal of the batch-insights and online-insight surfaces on the unreleased refactor branch. The deletions are consistent across the stack:
- Routes and screen imports in
src/components/Root.tsxare removed in lockstep with the deleted screens. createOnlineInsightHandler/createBatchInsightsHandlerare removed fromsrc/handlers/eval/index.tsxand the add/remove routers; corresponding entries insupportedTuiCommands,AddResourceInput,RemoveResourceInput,RemovableResourceType, andChinaAddKind(via the union) all fall out cleanly.EvalClientandTestEvalClientlose the insight methods, and the inputs/responses are removed fromsrc/handlers/eval/types.tsx;FilteredPaginatoris still used bysrc/core/gateway.tsx, so the import removal insrc/core/eval.tsxis correct.- The schema change in
src/projectSchemas/online-eval-config.ts(droppinginsights/clusteringConfig, makingevaluatorsrequired withmin(1)) is matched by the test updates and bycreateAddOnlineEvalHandlermarking--evaluatorsrequired. Existing spec files that only hadinsightswill now fail validation, which is the stated intent for the unreleased branch. - No stale references to
online-insight/batch-insights/clusteringConfigremain insrc/(the only hits are literal substrings inside unrelated batch-evaluation fixture names, and the surviving CloudWatch Logs Insights path called out in the PR body).
No telemetry concerns (feature removal) and no new mocking patterns introduced. Safe to merge.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## refactor #2576 +/- ##
============================================
- Coverage 97.54% 97.52% -0.02%
============================================
Files 657 634 -23
Lines 48779 47260 -1519
============================================
- Hits 47580 46090 -1490
+ Misses 1199 1170 -29 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
nborges-aws
left a comment
There was a problem hiding this comment.
LGTM. One nit, command.md needs to be updated as well to remove the insights commands and flip evaluators to be required
Description
What: Removes batch insights and online insights from the CLI.
agentcore eval batch-insights run|get|list|stopagentcore eval online-insight create|get|list|update|pause|resume|deleteagentcore add online-insight/agentcore remove online-insightWhy: Standalone insights will be removed for now.
How: Removed all handlers, core clients and tuis that are part of those.
Link: https://github.com/aws/agentcore-l3-cdk-constructs/pull/420