Repository navigation
chore: backmerge main into develop after 1.24.0 - #1246
Merged
Merged
Conversation
* feat(documents): DocumentDigest built at upload and persisted on FileMetadata (offload PR-2)
Stacked on PR-1. The digest a later turn will carry instead of the document's
bytes (docs/specs/document-context-offload.md §4A), built once, off the model
path, when a document upload completes.
- apis/shared/files/document_digest.py: a deterministic outline (headings
with page/paragraph/line anchors, table and figure mentions, counts, a text
sample spread across the document) via pypdfium2 and the PR-1 DOCX
extractor, plus a 3-5 sentence abstract from Nova Micro
(DOCUMENT_DIGEST_MODEL_ID) that fails open. render_digest emits the
<document-digest> block under a hard 1,500-token budget (sections dropped
first, then the abstract; the handle always survives) and the estimate is
stored as digest.tokens.
- FileMetadata.digest + FileUploadRepository.update_file_digest;
complete_upload schedules the build as a strong-referenced background task
for document uploads only. DOCUMENT_DIGEST_ENABLED=false skips it.
- Content policy: digest.abstract / digest.sections denylisted; status,
format, count and tokens projected; the attachment profile reports digested
count and digest tokens. DocumentDigestGenerated/Tokens/Ms EMF in
AgentCoreStack/Compaction.
- Nothing in the chat path reads the digest yet (PR-3). Spec: PR-2 row
marked built; §8 decisions 12-15.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(documents): restore rehydrates stripped documents as digests (offload PR-3)
Stacked on PR-2. The correctness half of docs/specs/document-context-offload.md
§4E: on restore, an inline document block becomes its <document-digest …>
block (abstract, outline, upload_id handle for document_read) instead of the
contentless placeholder that lost a returning user's document.
- session/document_rehydration.py: match each block to the session's upload
rows on the sanitized filename PromptBuilder gave it (dot -> underscore,
_2/_3 duplicate suffix), byte size as tiebreak, newest first, each row
claimed once; render the row's digest. Rows without a digest get an
outline-only digest built from the bytes already in the restored message
(no S3, no model call on the synchronous restore path) and persisted.
Unmatched blocks keep the placeholder verbatim; every failure falls back
to it. DOCUMENT_REHYDRATE_ENABLED=false restores the old path.
- _strip_document_bytes delegates and records document_rehydrated and
document_stripped separately on the ledger.
- FileUploadRepository: list_session_files_sync / update_file_digest_sync
bodies (initialize() runs under the agent constructor and cannot await).
- Restore output is byte-stable across restores given the same rows (tested).
- Spec: PR-3 row marked built; §8 decisions 16-20.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(documents): offload unpinned documents to their digests when the re-write is free (offload PR-4)
Stacked on PR-3. The cost half of docs/specs/document-context-offload.md
(§4C / §4D): once a document is no longer the active subject, its inline
bytes leave the cacheable prefix and its digest (PR-2) plus the document_read
handle (PR-1) take their place.
- session/document_offload.py: pinning (attach turn + next; the incoming or
previous prompt names the document's stem; a recent document_read result),
the 5,000-token size floor, the crc32 rollout bucket
(DOCUMENT_OFFLOAD_ROLLOUT_PERCENT) and DOCUMENT_OFFLOAD_ENABLED kill
switch, in-place replacement, and document_read slice ageing.
- TurnBasedSessionManager.apply_document_offload runs head-of-turn right
after apply_pending_compaction (the compaction stack's §3.5 predicate is
this spec's cache-aware rule): only on cache_expired, prefix_changed or
over_ceiling; otherwise nothing moves. The replacement is PR-3's restore
transformation, so the live block equals a cold restore's. Records
document_offload on the ledger with cacheGapSeconds and emits
DocumentOffloaded/DocumentOffloadedTokens/DocumentSlicesAged EMF.
- Slices older than DOCUMENT_SLICE_MAX_TURNS are stubbed on the live path and
on restore, so a warm agent and a cold restore agree.
- CompactionEvent (py + TS) gains digestTokens / slices / sliceTokens.
- Spec: PR-4 row marked built; §8 decisions 21-26.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(compaction): record the truncation anchor's cache gap on the ledger (offload PR-5)
The spec's PR-5 asked for a cache-live guard on the truncation deferral
(July 2026: 34 of 47 truncation events fired inside the 5-minute TTL). That
defect was closed by the compaction stack before this spec's PRs began: the
anchor moves only with a cut (applied paid-when-free) or in
_maybe_advance_truncation_anchor behind _cache_window_expired, and the
stability suite pins that a warm cache never advances it. What was missing
was the proof from rows.
- truncation_anchor ledger event (anchorFrom, anchorTo, cacheGapSeconds
measured BEFORE the save re-stamps updated_at) and
TruncationAnchorAdvanced / TruncationAnchorCacheGapSeconds EMF.
- cacheGapSeconds on the restore-time `applied` event, measured at the top
of _apply_compaction, so a restore-time slice or truncation inside the TTL
is countable as a rebuild while the cache was live.
- No behavior change. Spec: PR-5 row rewritten; §8 decision 27.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(feedback): content-free thumbs on assistant messages joined to cost rows (offload PR-7)
A thumbs up/down on assistant messages, persisted as an F# row on the
sessions-metadata table keyed (sessionId, messageId) beside the C# cost row,
so the admin session profile joins the outcome signal to the call's turn
class in one key lookup. Value ±1, timestamp and an optional reason code
from a closed enum — never free text, by the request model and the storage
layer both.
- apis.shared.sessions.feedback: F#{sessionId}#{messageId} rows (documented
with the C#/D# schemas in metadata.py); a second click replaces; session
rollups thumbsUp/thumbsDown ADDed while COST_DIAGNOSTICS_ENABLED.
- GET /sessions/{id}/messages merges the thumb onto metadata.feedback so a
reload restores the pressed state.
- PUT/DELETE /sessions/{id}/messages/{messageId}/feedback via
get_current_user_from_session; MESSAGE_FEEDBACK_ENABLED kill switch (404).
- Admin profile: feedback {up, down, byTurnClass, unjoined} + dataCoverage.
feedback; byTurnClass is null ("not tracked") until the C# rows carry
hasDocuments / documentDigests / documentReads (#1137).
- content_policy: FEEDBACK_ROW_PROJECTION walked by the existing test;
session projection gains thumbsUp/thumbsDown; call projection gains the
three turn-class fields.
- SPA: thumbs pair with reason chips in the message actions, optimistic with
rollback; Feedback tile with down-thumb rate by turn class on the anatomy
page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(admin): profile dataCoverage now carries feedback
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(attachments): hold a turn's inline attachments to the Memory event quota (PR-6)
The stored unit is the message, not the file: a turn's inline attachments
are persisted as one AgentCore Memory event, base64-inflated 4/3 into a
blob payload and bounded by the 10 MB event quota, so ~7.5 MB of raw bytes
per turn is the break point. Past it create_message re-raises
SessionException — a hole in history. The per-file 4 MB gate and the SPA's
5-file cap do not protect: ~1.3–1.4% of prod attachment turns exceed it,
several with only 3–4 files (document-context-offload-validation.md, Claim 7).
- `_apply_inline_byte_budget`: first-fit in attachment order against
INLINE_ATTACHMENTS_MAX_TOTAL_BYTES (default 7.5 MB, derivation in the
constant's comment); images count. Trimmed files join the existing
oversized-note path with their own wording ("send in a follow-up").
- `_apply_message_file_cap`: the server side of the SPA's
MAX_FILES_PER_MESSAGE, applied to both `files` and `file_upload_ids`
before the S3 fetch. The resolver's silent `[:5]` becomes a note to the
user; the app_api's dead `max_files_per_message` now reads the same
shared constant.
- One content-free EMF metric, AttachmentTurnOverQuota (Bytes) in
AgentCoreStack/Compaction, plus a count-only log line.
- ATTACHMENT_TURN_GUARD_ENABLED=false restores the prior behaviour.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(spa): share artifacts from the library and the owner view page
Sharing an artifact used to be reachable only from the card inside the
conversation that produced it, which meant the library at /artifacts —
the page whose whole premise is that artifacts outlive their chat — had
no way to share one. Both surfaces now open the same
ArtifactShareModalComponent the card does, so there stays exactly one
place in the app that knows what an artifact share is.
The control is gated on `row.owned`, so it appears on your own rows and
never on the "Shared with you" ones: a received artifact has no artifact
id to share (the share id is the only handle you have on it), and
re-sharing someone else's grant is not something this model supports.
The view page deliberately has no ownership check of its own. It
resolves its artifact out of listLibrary(), which reads the caller's own
DynamoDB partition, so an id that is not yours renders "Artifact not
found" before the header exists at all — the boundary is the partition
key, and a comparison in the template would imply the guarantee lives in
the UI. A test pins that the control is absent when nothing resolved.
Both surfaces show HEAD, so they pin HEAD's version. Shares are
immutable and never follow HEAD, so the version rides in each control's
accessible name ("Share Quarterly plan, version 4") as well as the
dialog's caption — the library prints a version badge only when it is
> 1, so the control cannot rely on one being on screen.
Grid footer: raise the label-collapse threshold 19rem -> 21rem.
Found in the browser, not by a test. 19rem was tuned for FOUR controls
and fit with zero slack: at three columns on a 1280px window the
container measures exactly 306px and the labelled row needed exactly
306px. Adding Share pushed the row to 326px while leaving the container
2px ABOVE the collapse threshold, so the labels stayed visible and the
Delete icon was clipped. 21rem (336px) collapses them with 10px to
spare, and the labels still reappear at one column, so they are hidden
where they do not fit rather than removed. Verified against real data on
dev: zero overflow on every card, in both views.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(feedback): align with response-feedback spec — flag name, six reason buckets, explicit/implicit signal discriminator
RESPONSE_FEEDBACK_ENABLED per spec §5; reason codes are the spec's §6
buckets (still a closed enum — free text stays out by decision, recorded in
the spec's new §13); rows carry signal="explicit" and every thumb reader
filters on it so the §10 implicit phase needs no backfill and never sums in.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(feedback): retry-with-correction — the consequence behind a thumbs down
response-feedback spec §11 PR-1's other half: a down-thumb's reason row
offers "Retry with that in mind", which prefills the composer with a
correction template for the reason code (new ComposerDraftService; the
composer consumes drafts for its session and never submits). When the user
sends it, ChatRequestService hands the added user message to
MessageFeedbackService.consumePendingRetry, which PUTs retryMessageId — the
message's index, never its text — onto the F# row. The write is now an
update_item upsert so a later re-thumb keeps the link.
Open question 1 settled as a new turn rather than /steer (which targets a
running turn). The admin profile reports feedback.retried and reworkUsd
(spec §7 "rework cost": the thumbed call rows plus the retry turn's
consecutive assistant rows). Spec §13 updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(admin-costs): expect formatGap's minute-only form in the offload title
formatGap drops a zero seconds remainder, so a 420s cache gap renders as
"7m", not "7m 0s". The assertion was written against the latter and was
the only red check on this PR.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(admin-costs): keep DataCoverage.documents on DataCoverage
The merge resolution appended develop's `documents` member after the
interface's closing brace, landing it on FeedbackProfile and breaking the
AOT typecheck.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(feedback): implicit signals — copy and continue (response-feedback PR-2)
Spec §10: denser than thumbs, no UI cost. Copy and Continue in the message
actions rail POST a kind code to /sessions/{id}/messages/{messageId}/signals
(fire-and-forget, once per message per kind per page load). Rows share the
F# family under their own key — F#{session}#{message}#{kind} — with
signal="implicit", kind, and an ADDed count, so they never collide with the
thumb and every thumb reader keeps filtering them out. The profile reports
feedback.implicit as messages touched per kind, on its own line: explicit
and implicit are never summed.
Edit-and-resend has no SPA affordance to hook; abandonment is deferred by
design (spec §13). Spec status and §13 updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(documents): count PDF pages, not bytes, in documentTokens
`documentTokens` (offload spec §6.1) and the `DOCUMENT_OFFLOAD_MIN_TOKENS`
floor (§4C) both read a document's weight from the compaction estimator's
`bytes/4`. That is wrong for PDFs by an order of magnitude: Bedrock
dual-encodes each page as an image *and* a text layer — the property §3 of
the spec depends on — and the image channel has nothing to do with file size.
Measured on dev (session 61de2256, 2026-09-16): a 27,578-byte, 60-page PDF
produced a 109.1K-token first write against a 14.6K static prefix, so the
document was ~94K tokens. `bytes/4` reported 6,894 — 13.7x low. At that error
the spec's "document share of the prefix" reads 6% where the truth is ~87%,
which inverts the ship/abandon call the evaluation spec §4.2 reads off these
rows, and understates a measured 109.1K -> ~15K prefix drop.
New `apis/shared/files/document_tokens.estimate_document_tokens`:
pages x PDF_PAGE_TOKEN_ESTIMATE (default 1,500, env-tunable) for PDFs,
floored by bytes/4 so a byte-heavy scan can only score higher than before;
bytes/4 unchanged for formats Bedrock extracts to text. Page counting costs
~0.1ms even at 200 pages (measured), so no memoization is needed, and an
unparseable PDF falls back to today's number. On the dev canary it lands at
90,000 against 94,485 measured — 5% low, deliberately, because this number
gates an eviction.
Wired into every consumer that reports or decides on that quantity, so the
row and the offloader can never disagree: `document_context` (the `C#` row's
documentTokens and documentSliceTokens), `document_offload` (Candidate.tokens,
the min_tokens floor and the aged-slice ledger figure) and
`document_rehydration` (document_stripped). The compaction estimator is
deliberately NOT changed — it rescales to measured history and drives cut
thresholds, so it needs its own decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(documents): couple DOCUMENT_READ_ENABLED to the paths that promise the tool
The flag was read in exactly one place — `_document_tools_gate`, which decides
whether `document_read` is injected. Neither the restore path nor the live
offload consulted it, so pulling the one kill switch an operator would actually
reach for (the tool is misbehaving) left two worse-than-before behaviours:
1. restore kept emitting `<document-digest upload_id=…>`, advertising a
retrieval handle for a tool that would not be injected — the model is
invited to make a call it cannot make;
2. the live offload kept evicting document bytes from the prefix with the only
recovery path switched off. That is strictly worse than the pre-offload
world, where live bytes never left at all.
Spec §5 states the rule this violated: "a digest that points at a tool nobody
has is no better than today's placeholder."
Fix, split by whether dropping the bytes is optional:
- **Restore has to drop them** either way (Bedrock rejects duplicate document
names across a conversation), and a digest is still strictly more than the
pre-PR-3 placeholder — so it still renders, but `render_digest` gains
`include_handle` and the restore path passes `document_read_enabled()`. The
outline and abstract survive; the false affordance does not.
- **The live offload is optional**, so it stops entirely:
`offload_enabled_for` now also requires `document_read_enabled()`. Both
callers (the head-of-turn offload and restore-path slice ageing) already go
through that single gate, so one check covers both and there is no second
flag read to drift.
With the flag on, every rendered byte is identical to before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(feedback): eval sampling — down-thumbs feed AgentCore Evaluations (response-feedback PR-4)
Down-thumbed turns become the evaluation harness's input queue. Thumb rows
carry GSI1PK=FEEDBACK#down / GSI1SK=updatedAt on the existing
UserTimestampIndex, so the fleet's recent down-thumbs are one query with no
new index. POST /admin/feedback/evaluations/run judges up to N not-yet-judged
ones in a background task via bedrock_agentcore.evaluation.EvaluationClient
over the runtime log group (keyed by the runtime session id the chat proxy
pins); GET /admin/feedback/evaluations is the queue with verdicts.
Routing is spec §6: wrong → Correctness + Faithfulness, instructions →
InstructionFollowing, length → Conciseness, other/none → Helpfulness;
tool_failed gets no judge and is corroborated against the call's tool census
(ops, not model); outdated has no judge yet. The stored verdict is
per-evaluator value / rating / n / tokens — the judge's explanation quotes
the conversation, so it is dropped at summarisation, refused at the storage
write, and denylisted in the content policy. The profile shows
feedback.evaluations.
Opt-in: FEEDBACK_EVAL_SAMPLING_ENABLED defaults off (CDK
CDK_FEEDBACK_EVAL_SAMPLING_ENABLED=true) because the managed judge reads the
sampled conversation's spans — the scoping decision the evaluations spike
says to make per environment. CDK wires the runtime log group into app-api
and grants Logs Insights on it and aws/spans plus Evaluate/GetEvaluator;
inert until an environment opts in. Spec §13 updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Agent Templates: create-form prefill + admin-managed backend (#1149)
* chore(agents): integrate Round 1 template-prefill work (P1 hydration seam, P4 reconcile, P3/P5 picker+content)
* feat(agents): Phase 2 template prefill — read localStorage draft in create mode, reconcile tool refs, populate form dirty with flagged/dropped notice
* feat(agents): add Q&A, Roleplay and tool-notice demo templates + catalog-wide spec coverage
* feat(agents): pin a concrete default model (Claude Sonnet 5) on templates so the picker prefills a selection
* feat(agent-templates): admin-managed template catalog backend
Backend for the Agent Templates feature per docs/specs/AGENT_TEMPLATES_SPEC.md,
mirroring the system_prompts admin-catalog pattern:
- shared/agent_templates: models (AgentTemplate + admin/public wire shapes),
repository (DynamoDB CRUD, Scan listing), service, and an idempotent seed of
three NEUTRAL fork-safe templates (no org tool refs, model_id=null).
- app_api/admin/agent_templates: scope-guarded CRUD (admin.agent_templates).
- app_api/agent_templates: public GET /templates returning the client
TemplateCatalogEntry shape (camelCase draft + pitch), enabled-only, ordered,
[] on empty.
- admin.agent_templates RBAC scope (GROUP_AI_CONFIG, delegable); routers mounted;
AGENT_TEMPLATES config-cache key.
- CDK: AgentTemplatesTable in AdminTablesConstruct + SSM params, compute ref,
app_api env var + IAM grant, platform-stack wiring + backup list.
- Tests: 24 model/repo/service/seed + route tests; scope-coverage map updated.
Local only, no push. Client picker still reads the hardcoded array; the
data-source switch is the follow-on step.
* test(infra): add AgentTemplatesTable CDK assertions
Cover the AgentTemplatesTable + wiring already added in 57a7bd53 with
infra CDK tests (the backend commit added none):
- constructs.test.ts: AdminTablesConstruct 4->5 tables + PK/SK/no-GSI/
AWS_MANAGED assertion for the agent-templates table.
- platform-stack.test.ts: shared-table count 27->28; SSM param cap
49->51 for the table name+ARN publishes.
- app-api-environment.test.ts: DYNAMODB_AGENT_TEMPLATES_TABLE_NAME wiring.
- security-policy.test.ts: scoped AgentTemplatesTableAccess IAM grant
(never Action:* / Resource:*).
Tests only, no source changes. Local, no push.
* feat(agents): source template picker from live backend catalog
Swap the 'Start from a template' picker from the hardcoded AGENT_TEMPLATES
array to the public GET /templates/ endpoint via a new AgentTemplatesService.
Remove the org-specific template DATA from agent-templates.ts (keep the
TemplateDraft/TemplateCatalogEntry types + the localStorage contract key) so
no Boise/higher-ed content is baked into the client (forkability). The
select handoff (write draft to localStorage, route to /agents/new) and the
downstream prefill/reconcile/notice path are unchanged. Picker gains
loading/empty/error states. Specs updated to mock the service.
* feat(admin-ui): agent templates admin console (list + form + service)
Add the admin UI for the admin-managed Agent Templates catalog, consuming the
/admin/agent-templates CRUD routes on the backend branch:
- AdminAgentTemplatesService: list/get/create/update/delete (lazy resource).
- Manage list page: name/emoji/enabled/order, enable-disable, delete,
reorder (sort_order swap), mirroring manage-system-prompts + manage-models.
- Create/edit form page (new / edit/:id): agent-form field set (name, emoji,
description, instructions, visibility, tags, starters, model, bindings) plus
catalog fields (pitch, status, sort_order). Blank model id -> platform
default; model params + binding config preserved on edit.
- Routes under admin/agent-templates guarded by adminScopeGuard
data:{scope:'admin.agent_templates'}; nav entry in AI Configuration;
admin.agent_templates added to the client scope union.
- 13 vitest specs (service endpoints, list render/toggle/sort, form
buildPayload/create/edit) green via the Angular builder.
Local only.
* feat(admin): model dropdown for agent-template form
Replace the raw model-id text input in the admin Agent Templates create/edit
form with a friendlier select: a 'Platform default' option (modelConfig.modelId
= null, the default for new templates) plus the RBAC-filtered configured-model
catalog, reused from AgentService.loadBindable('model') (GET
/agents/bindable?kind=model) — the same source the create-agent form uses.
On edit the saved modelId preselects; a saved id no longer in the catalog is
shown as a clearly-labeled 'unavailable' option and preserved on save rather
than silently dropped. Spec covers platform-default, select-a-model, and
edit-preselect (valid + stale).
* feat(admin): emoji picker popup for agent-template form
Replace the plain emoji text input in the admin Agent Templates form with a
real emoji-picker popup. Extract the create-agent builder's inline emoji field
(emoji-mart in a CDK connected overlay) into a reusable ControlValueAccessor
component, app-emoji-picker (src/app/shared/emoji-picker), so it drops into the
reactive form via formControlName="emoji" — the form drives its value
(including edit-mode patchValue) and reads selections back; clearing emits ''.
agent-form is left as-is and can migrate to the shared component later.
* feat(agent-templates): drop template visibility; tool-binding picker
Change 1 — remove visibility as a template concept end-to-end. A template's
lifecycle is enabled/disabled; visibility (PRIVATE/PUBLIC/SHARED) is an agent
concept, meaningless on a template. Dropped from the backend model/repo/seed,
the public GET /templates/ TemplateDraft, the admin + client TS types, and the
admin form. Prefill no longer sets the create-agent form's visibility from the
template — applyAgentToForm falls back to the form's own default (PRIVATE) when
a draft carries none, so a created agent stays owner-private by default.
Change 2 — admin tool bindings pick from the enabled tool catalog instead of
raw ref text. Reuses AgentService.loadBindable('tool') (GET
/agents/bindable?kind=tool), same as the model dropdown. A tool binding renders
a dropdown (label shown, ref stored); a saved ref no longer in the catalog is
kept as a clearly-labelled 'unavailable' option and preserved on save. Other
binding kinds keep the text input.
Specs updated (backend + frontend), targeted runs green.
* chore(agent-templates): drop now-unused BindingPayload import in seed
* test(infra): register AgentTemplatesTable in GSI inventory + table count
The new AgentTemplatesTable bumps the stack's table inventory, tripping two
full-suite infra guards the targeted pre-push run didn't cover:
- gsi-inventory.json regenerated to include agent-templates (no GSIs).
- observability-dynamodb-alarms table count 27 -> 28 (the per-table throttle
alarm is auto-derived, so no construct change needed).
* docs(kaizen): queue the document-offload prod readout, and mark the epic built
The epic shipped (#1137–#1143, #1145) and was validated end to end (#1147 fixed
the two defects that sweep found). What is left is measurement, which the
Friday loop is the right home for. Three entries:
- **Decide the rollout arm before the release.** `DOCUMENT_OFFLOAD_ROLLOUT_PERCENT`
defaults to 100, which is the evaluation spec's own §4.2 "Ship" state, while
that spec is still Draft with no harness and its §2 gate written as a veto.
The bucket exists to give the evaluation concurrent B/C arms; at 100% there is
no control and only a before/after across the release boundary remains. This
is time-sensitive in a way the compaction veto item is not.
- **The prod readout — five numbers and what each decides**, including the
prediction that `document_offload` events will be ≈0 because `document_read`
is itself an `extra_tool`, so restore reaches the document every turn before
the live offload can. If so, all the benefit is PR-3 and PR-4's pinning is
inert. Also the first down-thumb-by-turn-class read, which is what PR-7 was
built for.
- **Recalibration plus the four findings still unfixed** — ReDoS in pattern
mode first (measured 14 s at 28 chars, exponential, model-supplied input),
then fitting `PDF_PAGE_TOKEN_ESTIMATE` from prod attach turns.
The originating [2026-09-16] epic entry is marked BUILT so review-prep does not
re-propose the build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): correct the rollout-percent entry — the bucket gates PR-4 only
I overstated this. `offload_enabled_for` is consulted in exactly two places:
the head-of-turn live offload and restore-path slice ageing. PR-3 — the
digest-on-restore that is the epic's actual behaviour change — is NOT bucketed;
it rides `DOCUMENT_REHYDRATE_ENABLED`, a plain global flag, as do PR-1, PR-2
and PR-6.
So the percent never was a control arm for PRs 1–3, and 'at 100% there is no
control' was wrong about the epic and right only about PR-4. Since readout
item 1 predicts PR-4 fires ~0 times, counting `document_offload` events at
100% answers the same question the A/B would have, more cheaply.
Entry now recommends shipping at 100%, records what that gives up, and names
`DOCUMENT_REHYDRATE_ENABLED=false` as the real rollback lever.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore(kaizen): weekly research scan 2026-09-18
Generated by the kaizen-research skill. Top 5 ideas appended to
docs/kaizen/review-queue.md for the kaizen-review-prep run later this morning.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(kaizen): weekly review prep 2026-09-18
Generated by kaizen-review-prep. Ranked agenda for the 10-15 min decision pass;
queue updated with prior-review outcomes.
Built queue-first and outcome-first: this morning's kaizen-research run fired
1h41m later than last week and was still running at prep time, so there is no
current research doc. The queue's Open entries are the skill's primary source,
so the review degrades to that; the pipeline gap is Proposal #5.
Outcome scorecard: last review converted 6 of 9 proposals, falsifying its own
"named carrier" thesis. The class that has never converted is the one requiring
a CLAUDE.md or skill-file edit (5th cycle).
Queue: 3 items resolved by outcome (nightly both halves, rate provenance,
GPT-6 Astra), 3 annotated as partially shipped, 3 new entries added.
Deliberately does NOT duplicate PR #1147's three queue entries, which are the
authoritative versions of the offload residue. An earlier draft of this review
also recommended moving DOCUMENT_OFFLOAD_ROLLOUT_PERCENT off 100; #1147's own
entry refutes that (PR-3, the epic's real behaviour change, is not bucketed at
all), so the recommendation is withdrawn and the correction is recorded in both
the review's Take and the queue entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(documents): bound document_read pattern mode against ReDoS
`_compile` passed a model-supplied regex straight to `re.compile`. A
nested-quantifier pattern then runs in exponential time — measured on the real
engine, `(a+)+$` against `"a"*n + "!"` takes 0.87 s at n=24, 3.5 s at 26 and
**14 s at 28**. Extracted PDF lines run 60–100 characters, so such a pattern
never returns. The scan runs in `asyncio.to_thread`, and CPython offers no way
to cancel a running `re.search` (signals are main-thread only), so the turn
hangs to the 600 s SSE timeout.
Two defences, because neither is sufficient alone:
1. `catastrophic_pattern` refuses the nested-quantifier family up front — an
unbounded quantifier applied to a group that itself contains one — and the
pattern is searched as literal text instead. That is the same graceful
degradation an invalid regex already got, and the payload carries a
`pattern_note` so a literal result is never presented as the regex one.
Deliberately narrow so it has no false positives: verified against 7
catastrophic shapes (all flagged) and 13 patterns a model would plausibly
write — `\d+`, `(invoice|receipt)`, `(foo)+`, `\$[\d,]+\.\d{2}` — none
flagged. Escapes and character classes are honoured, so `\(\w+\)+` and
`[(]a+[)]+` are correctly read as literals, not groups.
2. `DOCUMENT_READ_PATTERN_BUDGET_SECONDS` (default 2.0, `0` disables) bounds
the *walk* across lines and pages for anything the detector does not catch
(overlapping literal alternations like `(a|a)+`, backreference blowups). It
cannot interrupt one `re.search` — nothing can — so it is a backstop, not a
guarantee: a scan that runs out returns the matches it already has with
`timed_out` and a hint naming how far it got.
The tool description now also tells the model not to nest quantifiers, so the
common case is avoided rather than only corrected after the fact. A `regex`
module with native `timeout=` would be the complete fix; it is installed
transitively but not declared, and adding a dependency needs approval.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): mark the document_read ReDoS fixed, and record what is left
Fixed in 71b0fdfb. The residual is worth keeping in the queue rather than
closing outright: a single re.search still cannot be interrupted, so the
wall-clock budget is a backstop. The complete fix is the regex module's
native timeout=, which is a dependency decision (installed transitively,
not declared) and therefore needs approval rather than a quiet adoption.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(documents): close the last three findings from the offload validation sweep
**1. The inline-attachment budget had zero headroom.** base64 of N raw bytes is
`4*ceil(N/3)`, so the 7,500,000 default encoded to *exactly* 10,000,000 — the
AgentCore event quota itself — before the event's JSON envelope (role, content
keys, prompt text, per-file metadata, wrapper). A guard whose default sits on
the break point it exists to stay under does not prevent the failure it was
written for: `create_message` raises `SessionException` and leaves a hole in
history. Default is now 7,000,000, which encodes to 9,333,336 and leaves ~666 KB.
Still far above prod's p90 attachment cluster (2.58 MB), so no real turn is
newly trimmed. The existing test pinned the old number and its own comment
named the flaw ("lands on the quota, not over it") — rewritten to assert the
break point is real *and* that the default stays below it.
**2. The digest's 1,500-token cap was soft.** `render_digest` XML-escaped the
abstract *after* slicing it to the room left, so `&` -> `&` could render up
to 5x longer than the slice it was measured on — an all-`&` abstract rendered
388 tokens against a 100-token budget, and `digest.tokens` recorded the wrong
number with it. Now escaped first and cut on an entity-safe boundary
(`&` is never left as `&am`), with a final guard that drops the abstract
rather than overshoot. Only the opening tag — the `document_read` handle —
may outlive a budget too small for anything else.
**3. `prefixTokens.tools` absorbed the whole document on attachment turns.**
Diagnosed: `toolTokens` is a *residual* between two independently sourced
numbers — `full` (Strands' projection for the upcoming request) and `no_tools`
(our own CountTokens call) — so any disagreement about how a content block is
counted lands wholly in it. Bedrock understands a PDF page as an image *and* a
text layer; when the two sources disagree there, the document's entire weight
is attributed to tools. Measured on dev (session `61de2256`): `toolTokens` of
**106,756** where the real tools prefix was **12,516** — a 94,240 difference
against a document measured at ~94,485, i.e. the whole document.
The counting disagreement is Bedrock/Strands behaviour we do not control, so
the fix is to stop publishing a number we know to be untrustworthy: the split
is not computed while inline document or image bytes are in context, and is
taken on a later clean turn instead. An absent `prefixTokens` reads "not
tracked" (the ledger's convention); a wrong one silently corrupts every share
computed from it. A digest is text, so an attachment session still gets the
field from turn 2 onward, and an agent that already holds a trustworthy split
keeps using it — the document then lands in `messages`, where it belongs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): mark the last three offload findings fixed
Also records that prefixTokens.tools was NOT an off-by-one: toolTokens is a
residual between two independently sourced counts, and 106,756 - 12,516 =
94,240 against a ~94,485 document — the whole document landing in the tools
bucket. Carries forward the one open question (whether images disagree the
same way) for the prod readout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(feedback): fleet-level attribution — down-thumb rate by config arm (response-feedback PR-3)
The config-dimension attribution spec §7 calls the prize, and the compaction
spec §7.2 asked for: down-thumb rate joined against the choices we made,
across the fleet rather than one conversation at a time.
GET /admin/feedback/fleet?days= reads a trailing window off the
FEEDBACK#down / FEEDBACK#up partitions, joins each thumb to its C# cost row,
and splits it four ways: model, agent switch, distance from a compaction cut
(never / same call / 1-3 / 4+), and document turn class. Both reads are
capped and exceeding either is reported as coverage.truncated /
sessionsOmitted, never silently under-counted. /admin/feedback renders it
under the admin.costs scope.
Spec §9 is enforced, not promised: every arm carries its own n, an arm under
the floor returns downRate: null so there is no number to quote, and no
fleet-wide rate field exists at any level — the page states the gap between
the two furthest-apart reportable arms instead of a score. Open question 3
answered: the floor is 20, env-tunable via FEEDBACK_ARM_MINIMUM_N.
Skills is deliberately not an arm: enabledTools lives on the S# row, not the
C# row. Spec §13 updated with both deviations.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(admin-costs): down-thumb reason split on the per-session profile
The fleet view (#1152) reports the reason distribution across a period; the
per-session profile an admin drills into from it still did not. All six codes
(`wrong`, `instructions`, `length`, `tool_failed`, `outdated`, `other`) were
written to the `F#` row and round-tripped to the owner's own UI, but
`FeedbackProfile` exposed only counts — so the fleet could say *how much* and
the drill-down could not say *why this conversation*.
`_join_feedback` now aggregates them into `reasons`, which is the field that
turns "someone disliked this" into "these are tool_failed". Closed set only: an
unknown code (a row from a future schema) is dropped rather than surfaced, and
a reason on an up-thumb is ignored as meaningless.
Scope note: this began as a pair with a fleet-level aggregate, which #1152
landed first and better — generic config arms (model / agentSwitch /
callsSinceCompaction / turnClass) against my turn-class-only view, plus a
coverage block, and it deliberately refuses a fleet-wide rate where mine
emitted one. That work was dropped rather than duplicated; this is the half
#1152 left open.
SPA contract updated to match; `tsc --noEmit` clean.
Also queues the live-AWS test hang found while validating: one full-suite run
stalled 72 minutes on 4m26s of CPU holding four TCP connections to real AWS.
The test it stopped at passes in 1.9s alone and its whole slice is green, so it
is the victim of leaked state. Not reproducible on demand — one hang in seven
full runs the same day — which is itself the first thing to fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): diagnose the live-AWS test calls — 25 cases, hidden by fail-open
A socket guard (autouse patch of socket.connect, failing any off-box
destination) found 25 off-box connection attempts across 6 test files in one
run, all to real us-east-1 AWS.
The archetype, from test_skills_access.py: the test patches the role service,
but resolve_accessible_skill_ids then calls resolve_owned_skill_ids, which
builds a real DynamoDB client. Its except Exception returns [], catalog + []
== catalog, and the assertion passes either way. Fail-open production code plus
a partially-mocked test equals a silently real AWS call that never fails the
suite — 9196 tests pass with or without the connections.
Worse than a flake: ~/.aws/config carries a [default] profile, so these resolve
real credentials and are authenticated requests against real AWS from a unit
test run. The 72-minute hang was just the tail case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test: block off-box sockets, and fix the tests that were calling real AWS
The suite was not fully mocked. A socket guard found real, authenticated calls
to us-east-1 AWS from unit tests — `~/.aws/config` carries a `[default]`
profile, so these resolved working credentials. One run stalled 72 minutes on
4m26s of CPU holding four ESTABLISHED connections while botocore retried.
**Why it was invisible.** Service code here is deliberately fail-open — a user
should not lose their session because a table blipped. So a test that mocks one
dependency and misses a second gets a real client, a real request, a swallowed
exception, and a green assertion. The suite reported the same 9196 passed with
and without the connections. The archetype: `test_skills_access` patches the
role service, then `resolve_accessible_skill_ids` calls `resolve_owned_skill_ids`,
which builds a real skill-catalog repository; `catalog + [] == catalog`, so the
assertion passed either way.
**The guard** (`tests/conftest.py`) patches `socket.connect`/`connect_ex` and
blocks any non-localhost TCP destination. It sits at the socket layer rather
than in botocore because moto intercepts `before-send` itself, making handler
ordering ambiguous; moto never opens a real socket, so "left the box" is an
exact test. Only AF_INET/AF_INET6 **stream** sockets are guarded, so DNS and
AF_UNIX are untouched. Because the raise is swallowed by the code under test,
the violation is also recorded and asserted at teardown — that assert is the
only thing that makes this visible at all.
`AWS_TEST_ALLOW_OFF_BOX_SOCKETS=1` disables it.
**Fixed** by stubbing the second, unmocked dependency each path builds:
`test_skills_access` (the file's own `_owned()` helper existed and simply was
not used), `test_rbac_cache_service` + `test_app_role_service` (the tool-catalog
freshness snapshot), `test_kb_backend_parity` (CloudWatch `_publish`),
`test_role_agent_pins` (role admin service, `ToolCatalogService.get_tool`,
`MemorySpaceService.resolve_permission` — the latter two patched on the service
because the code receives an already-constructed one), `test_pbt_model_access_preservation`
(session metadata + the `document_read` gate), `test_resolver` (the AppRole
quota branch).
**Quarantined**, to be burned down: ten files under `tests/routes/`, listed in
`_OFF_BOX_QUARANTINE` with the recipe. Note what quarantine does and does not
do — the block applies to *every* test, so none of them can reach AWS from now
on; quarantine only suppresses the teardown failure, so the debt is bounded and
visible rather than load-bearing. No new offender can be added silently.
Full suite: 9196 passed, 3 skipped, 0 errors with the guard enforcing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): guard landed; record the burn-down list and the two traps
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(specs): assess the AuraX load-test A/B and plan the fixes
Validates the five fixes and three residuals from the 2026-09-15
40k-user A/B against develop: four confirmed, uploads-bucket CORS is a
deployment-config gap. Names the RetrieveMemoryRecords 30/s quota the
report calls 'conversation memory', and the per-message retrieval that
consumes it. Phased plan with a proof per item.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(infra): fail synth when the uploads bucket would have no CORS rule
buildCorsOrigins() returning empty used to be a console.warn; the
UserFilesBucket was then created without CORS and every browser upload
failed at S3. A production-mirror load-test environment shipped exactly
that. Throw instead, with CDK_ALLOW_NO_CORS_ORIGINS=true as the explicit
opt-out for a deployment with no browser front-end.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(inference-api): precompile bytecode and warm the first turn at start
uv sync ships no .pyc by default and /app/src was copied raw, so every
fresh AgentCore micro-VM compiled the dependency tree (sympy behind the
calculator tool included) on a new session's first message. Set
UV_COMPILE_BYTECODE=1 for the venv and compileall /app/src in the image.
A daemon-thread warm-up at lifespan start pulls the remaining lazy
imports and boto service-model loads off the request path; /ping
answers immediately. INFERENCE_WARMUP_ENABLED=false is the kill switch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(agents): bound CountTokens on the reply path, stop swapping the model id
Strands awaits one CountTokens call before every model call. Under load
the count throttles before the model does, and the SDK's shared client
retried it with backoff, so the reply waited out that backoff (p95 5.8s
on the 40k-user run). Count through a dedicated client with one attempt
and a 2s timeout: a throttle now costs one failed request before the
chars/4 heuristic. Pass the base model id to the API explicitly instead
of mutating config['model_id'] around the call, which was only safe
while a count and a stream could never overlap on one instance.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(agents): memoise the attribution split per session; cache four more tool families
Sessions whose injected tools keep them out of the agent cache rebuild
their Agent every turn and paid the two cold-start CountTokens calls
every turn on top of the SDK's own. Key the stable system/tools split by
(session, prompt digest, tool-spec digest) in a bounded process memo so
a rebuilt Agent adopts it. Promote Word, Excel, PowerPoint and workspace
tools to the cache-eligible set: each closes over (session_id, user_id)
only, exactly what the artifact arm proved safe in #839/#841.
Spreadsheet analysis still bypasses until assistant_id is in the key.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(admin-costs): pin the cache-bypass diagnosis to fixed sets, not today's promotion state
The rule derives its list from KEY_DESCRIBED_INJECTED_TOOL_IDS, so the
test broke the moment four more families were promoted. Test the
mechanics against patched sets, and keep one assertion on the live
state that names spreadsheet analysis as the only remaining bypass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(agents): cache spreadsheet-analysis sessions by carrying assistant_id in the key
The spreadsheet builders close over assistant_id, which the agent cache
key did not carry, so every session with analyze_spreadsheet or
list_spreadsheets enabled (2,669 of 3,565 prod sessions on the last
read) bypassed the cache and rebuilt its Agent, session manager and
AgentCore Memory restore on every turn. Add assistant_id to
_create_cache_key, stamp it on the construction snapshot, carry it on
PausedTurnSnapshot and replay it on resume so a paused agent is never
orphaned. A snapshot written before the field existed resumes with None,
misses the slot and rebuilds — the pre-existing eviction path, never a
wrong hit. Promote the spreadsheet family; every enabled_tools-gated
injected family is now cache-eligible.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test: burn down the off-box quarantine — no test reaches AWS, and the suite halves
The guard shipped with ten `tests/routes/` files quarantined. They are fixed,
the quarantine list and its bypass are deleted, and the guard is now
unconditional: any test that opens a non-localhost socket fails.
**The suite runs in 8:02, down from ~14-16 minutes.** That gap was the cost of
the problem — connection attempts to real AWS, each with botocore's connect
timeout and retries, on every run. Roughly half the suite's wall clock was
waiting on calls whose results were then thrown away by a fail-open `except`.
Most of the 51 cases came from a handful of shared seams, so the fixes live in
`tests/routes/conftest.py` as one autouse fixture rather than per file: the
inference path (session metadata, the `document_read` gate), the converse path
(managed-model routing, the rate-limit window, three quota-assignment lookups),
RBAC role resolution, agent-detail label resolution, and the artifact share
cascade on session delete. All are fail-open infrastructure lookups that no
route test asserts on; a test that cares patches its own, and that patch wins.
Two per-file fixes: `test_converse_cost_accounting`'s `mock_pricing` fixture
already existed and its docstring already said "without this mock the call hits
DynamoDB and fails" — it was simply opt-in, and the tests that did not request
it were calling the real table. It is autouse now. `test_sessions`' delete
cascade needed the artifact seam above.
Two order-dependence traps worth recording, both of which made these look
unreproducible: `tools/freshness` memoizes on a module-level TTL, and the
artifact share cascade only fires once an earlier test has left artifacts
"configured" — so several of these passed in isolation and failed only in
full-suite order.
Full suite: 9196 passed, 3 skipped, 0 errors, no quarantine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(kaizen): close the live-AWS entry — quarantine burned down, suite halved
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test: warm tiktoken before the guard arms, and write the rule into CLAUDE.md
CI caught what my machine could not. `csv_chunker` calls
`tiktoken.get_encoding("cl100k_base")`, which downloads the BPE vocabulary from
an external CDN on first use and caches it on disk. My cache was warm, CI's was
cold, so the guard correctly blocked a real network fetch and 16 tests failed.
Not an escaped AWS call — a legitimate asset fetch — but also a real network
dependency of the test run, which had been invisible precisely because every
developer machine has it cached. Warming it in `conftest.py` *before* the guard
arms keeps the guarded window hermetic without widening the allowlist, and makes
the dependency explicit rather than incidental. Best-effort: offline, the CSV
chunker tests now fail on their own terms rather than on a confusing socket
error. Verified against a cold `TIKTOKEN_CACHE_DIR`.
Also adds a **Tests** section to CLAUDE.md, since the rule is only useful if it
is where someone looks before writing the next test: no test may reach the
network; why the teardown assert is load-bearing (fail-open code swallows the
block, so the raise alone leaves a green suite); how to find the *second*,
unmocked dependency when the guard trips; the two module-level-memoization traps
that make escapes look order-dependent; and that shared stubs belong in the
directory's `conftest.py`.
Full suite: 9196 passed, 3 skipped, 0 errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(spec): plan the agent state feedback refinement
Records what the live indicator can honestly say beyond "Thinking" and
"Running <tool>", and sequences it into three PRs.
Two findings shape the plan. `thinking` and `tool_start` already cross
the wire and are dropped on the floor — `ToolInsightService` writes every
transition into a signal with zero readers, and the loader derives its
label from the content stream instead. That is not a style choice: the
status drain runs between yields of the agent stream, so a `tool_start`
queues for exactly the silence it exists to explain. Decoupling the drain
is therefore a precondition for the richer phases, not a cleanup task.
Also states the non-goals, so they are not re-litigated: no "responding"
phase, no "almost done thinking" (an inference wearing a fact's clothes),
no invented per-tool phrasing, and no persisted durations.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(chat): report how long the model spent thinking
The reasoning block's header said "Thinking" forever, including on a
block that finished nine minutes ago. It now reads "Thought for 17s"
once the model has stopped.
The span is client-observed — the parser has no server-side measurement
of thinking time — and it is stated as such in the code, because the tool
durations on the rail are measured inside Strands' event loop and the two
are not the same kind of number. It is sound here because the agent loop
starts a new Bedrock message at every tool round trip, so a reasoning
block never spans a tool call and cannot silently absorb its time.
The span closes at the first non-reasoning content in the same message,
with message_stop as the backstop for a cycle that reasoned and emitted
nothing else. Sub-second blocks read "<1s" rather than rounding up to a
second the model did not spend.
Live-only by construction: the duration rides `ContentBlock`, which only
the stream parser populates, so a reloaded conversation falls back to the
plain header with no persisted number — the same posture as the tool
rail. It sits at the block's top level rather than inside
`reasoningContent`, which is Bedrock's Converse shape.
docs/specs/agent-state-feedback.md PR-1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(attachments): auto-enable Spreadsheet Analysis for a session that holds a spreadsheet
CSV/XLSX never go inline (_partition_attachments) and the analysis
tools are opt-in in the picker, so on the default tool set an attached
spreadsheet was a dead end: the model told the user to find a sidebar
toggle. A session that holds a spreadsheet — this turn's attachment, or
a READY upload from an earlier turn — now gets list_spreadsheets and
analyze_spreadsheet appended to the turn's effective enabled_tools,
gated on the caller's RBAC grant via can_access_tool (enables, never
grants). Applied on the effective list so it flows into the cache key,
every builder, the attachment guidance and the paused-turn snapshot as
one value, and through the same seam on the MCP App dispatch paths so
they read the slot the real turns fill. Sticky per session (memoised
positive answer, one query otherwise), so the key does not flip between
the attach turn and the follow-up. The guidance for the remaining case
— a role that does not carry the tool — no longer points at a toggle
the user may not have. ATTACHMENT_TOOL_AUTOENABLE_ENABLED=false is the
kill switch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(agents): count the system prompt against a probe message, not an empty conversation
Bedrock CountTokens rejects an empty message list ('A conversation must
start with a user message' — verified against dev 2026-09-18), and
Strands swallows that into the chars/4 heuristic. So the systemTokens
partition of the context breakdown has been an estimate since it
shipped, never the authoritative count it looked like. Count the system
prompt against a fixed probe user message and subtract the probe's own
weight, a per-model constant measured once per process. Cold start is
now three counts (two after the first agent on a model), none of them
with an empty conversation.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(memory): retrieve long-term memory through a bounded client, and stop re-fetching the session's own summary per message
The SDK retrieves long-term memory on every user message, awaited
before the model call, through its shared read/write boto client with
default retries. Under load a throttled RetrieveMemoryRecords (30/s per
account by default) put boto's retry backoff on first-token latency —
the same mechanism as the CountTokens throttle fixed in #1154. Override
retrieve_customer_context with the SDK's exact contract (namespaces,
relevance filter, <user_context> block prepended to the last user
message) but through a dedicated one-attempt, 2s-timeout client; writes
keep their retries.
The third retrieval per message was the current session's own summary
namespace: AgentCore's summary of THIS conversation, re-injected into
every user message the model already holds the conversation for — paid
as input tokens on every turn and a lookup against the quota. Off by
default; MEMORY_SUMMARY_NAMESPACE_RETRIEVAL_ENABLED=true restores it.
Compaction still reads summaries where they matter, at checkpoint
advance. Lookups per user message: 3 → 2, none of them retried.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(agents): drain agent_status while the agent stream is silent
The status hook records model-call and tool-call boundaries from inside
Strands' event loop, which has no route to the SSE stream — so the
coordinator drains it from the emit loop. That loop only regains control
when the agent stream yields, and during tool execution the agent stream
yields nothing. A `tool_start` therefore waited out exactly the silence
it existed to explain and arrived bundled with its own `tool_end`:
measured on a three-tool browse turn, the indicator read "Thinking" for
all 4.5s and never named a tool.
`_merge_agent_status` races the agent stream against a short timer and
drains on every pass, so a transition recorded mid-silence leaves the
container while the tool is still running. The in-flight `__anext__` is
held across timeouts rather than re-requested — an async generator
cannot have two outstanding — and cancelled when the coordinator
abandons the stream, which unwinds `process_agent_stream` through the
cleanup-on-cancellation path it already documents and already took.
Its own kill switch, `AGENT_STATUS_LIVE_DRAIN_ENABLED`, rather than
riding AGENT_STATUS_ENABLED: that flag gates what the hook records, all
of it in-process, while this changes how the turn's stream is consumed,
so a regression here would be a streaming bug rather than a missing
status line. Off, the between-yields drain is restored exactly.
With the transitions now trustworthy, the SPA reads them:
- the loader prefers `agent_status` over the content stream for the
running tool, keeping the content derivation as the fallback — it
needs no round trip, so it is more current when it fires, and it is
all that is left if the drain is killed.
- a parallel batch is finally legible. `ToolInsightService` tracks the
matched tool_start/tool_end pairs, so three tools in flight read
"Running list_assignments and 2 more" instead of naming whichever
unresolved block happened to come first.
- `thinking` narrows the generic label to "Waiting for the model",
which rules out the agent build and head-of-turn context work that
precede it and read identically today.
docs/specs/agent-state-feedback.md PR-2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(spec): mark PR-1 shipped and record the loader-mount finding
PR-1 merged as #1159. Also records what building PR-2 turned up and the
plan did not anticipate: the loader stays mounted for the whole turn, so
a label keyed on the `thinking` phase would contradict text already on
screen. The gate that avoids it is a workaround — the real fix is that
the loader should not render under a streaming answer at all, which is a
separate change to when it renders rather than what it says.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: respond to the AuraX load-test report, finding by finding
Records our verdict on each of the five fixes and three residuals, what
we shipped for each (#1154, #1155, #1156, #1157, #1158), and what is
left. Two corrections the report should carry: the 30/s quota is
RetrieveMemoryRecords, not the restore path, and ListEvents has a fixed
20/s per-session cap no increase moves; and '6 lookups per message'
conflates a per-cold-start restore with a per-message retrieval. Two
deliberate divergences: we bounded the pre-model token count rather
than backgrounding it, because the same projection drives compaction
timing and therefore prompt-cache cost; and we scoped the attachment
fix to spreadsheets, since documents already route through
document_read. Also notes what their probe-message detail found:
systemTokens had been a chars/4 heuristic since it shipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: scope the deployment claim to what is verified in dev
Five PRs are merged; only #1154 and #1155 have a completed dev deploy.
The three merged since were superseded by concurrent deploys, so the
status table now separates 'verified in dev' from 'deploy pending'.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(files): start the pattern-scan budget after the document is opened
`_pdf_pattern` built its `_Budget` before calling `_open_pdf`, so lazily
importing the pypdfium2 native library and parsing the PDF were charged to a
budget whose docstring bounds "the walk across lines and pages". With enough of
the budget spent before the loop, the first `clock.out_of_time()` check breaks
immediately and the tool returns `match_count: 0`, `pages_searched: 0` and
`timed_out: True` — a timeout reported on a scan that never started, and an
empty answer to an ordinary pattern.
Measured, the open is a skim rather than a wipe-out: on a warm machine the lazy
import costs 46-95 ms and the parse is lazy in pypdfium2, so it does not scale
with the document (0.4 ms for 1000 pages; 68 ms for a 6 MB file with a broken
xref, the worst case that forces a full rebuild). That is 2-8% of the 2.0 s
default, so full exhaustion needs a much smaller configured budget or a
pathological cold-start import. The ordering is still wrong: it silently shrinks
the scan window by an amount that varies with container warmth, and the failure
it produces is indistinguishable from "no matches".
`_text_pattern` had the milder version of the same thing — `text.splitlines()`
inside the budget — now also moved ahead of the clock. The docx path already
extracted its paragraphs before calling `_text_pattern`, so it needed no change.
This loosens total wall time by the open cost, and deliberately adds no outer
bound: `_pdf_index`, `_pdf_pages` and the S3 fetch already run with no clock at
all, and index mode extracts text from up to 40 pages unbudgeted, so pattern
mode's new exposure is strictly smaller than what the other modes already accept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(chat): remove the parallel-batch count as dead code
PR-2 shipped "Running list_assignments and 2 more" on the reasoning that
`tool_start`/`tool_end` are a matched pair, so the count is real. The
count is real; the case never occurs.
`agent_factory.py` pins `tool_executor=SequentialToolExecutor()` so that
concurrent browser tools cannot start two Playwright sessions. A batch
therefore emits strictly interleaved start/end pairs and more than one
tool is never in flight. Verified on dev against a real three-tool
batch: the rail showed "3 tools", the loader showed one name throughout.
I reasoned from the Strands default (ConcurrentToolExecutor) without
checking that we override it, which we do, with a comment saying why.
`ToolInsightService.runningTools` keeps its list shape — it costs
nothing, it is what a concurrent executor would need, and it remains the
truthful source for the ONE running tool: a `toolUse` block starts
streaming when the model begins writing a tool's arguments, while
`tool_start` fires when the tool begins executing, a measured ~640ms
apart.
Also records the dev verification in the spec, including where the time
actually goes (3.75s before the first SSE byte on a warm turn), and
marks PR-3 blocked: the window it wants to narrate is one in which no
SSE channel is open, so it cannot be built as written.
docs/specs/agent-state-feedback.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(browser): hand the browser to the user so they can sign in
PR 1 of docs/specs/authenticated-web-assessment.md. `browse_web` can only
read what a logged-out Chromium can reach, which is the one thing every
target in the Library and VPAT requests fails. This adds the backend half of
the fix: the agent hits a login wall, calls `request_user_login`, the turn
pauses, the user signs in with their own hands, and the turn resumes on the
authenticated session.
Its own tool, not a `browse_web` action. RBAC granularity is exactly one
tool_id — a role's `grantedTools` rolls up into `enabled_tool_ids` and there
is no sub-tool gate — so an action would have shipped an interactive browser
inside our AWS account to every user who can browse, students included. As a
separate catalog entry it is granted to named roles, ships
`enabledByDefault: false`, and costs an ungranted user zero prefix tokens.
An interrupt, not an endpoint. The turn genuinely must stop, and a turn that
polls for login completion burns model calls to learn nothing.
`ask_user_question` already proved the pause/snapshot/resume path and Strands
routes tool- and hook-raised interrupts identically, so the
`PausedTurnSnapshot`, the resume route and the breadcrumb need no special case.
No URL anywhere near the model. `generate_live_view_url` signs with SigV4
query auth and caps at 300 seconds, so a URL in a tool result is re-emitted
truncated at the `?` (PR #1101) and dead before a human reacts either way.
`assert_no_url` enforces this on the interrupt, the SSE event and the
persisted row rather than trusting it, and the broken `live_view` action is
removed. app-api mints one per request in PR 2.
Also: the D4 projection onto the session-metadata row (app-api cannot read
`agent.state`), reaper pinning bounded by an abandonment deadline so a
walked-away-from takeover stops billing, and `BROWSER_TAKEOVER_ENABLED`.
Two deviations from the spec sketch, recorded in it: the event carries both
`sessionId` (conversation, as everywhere else) and `browserSessionId`, and
the conversation id plus the D4 write are stamped on in the streaming layer
so the tool stays a static…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backmerges
mainintodevelopafter the 1.24.0 release (#1244, squasheaefde28).VERSION,CHANGELOG.md,RELEASE_NOTES.md,README.md, manifests and lockfiles). No feature code differs.develop's copy, soplatform.ymlmerged with no diff.sync-version.sh --checkpasses, andorigin/mainis an ancestor of this branch.🤖 Generated with Claude Code