fix(pii): mask offloaded large payloads chunk-by-chunk instead of aborting at 16MB by TheodoreSpeaks · Pull Request #5810 · simstudioai/sim

TheodoreSpeaks · 2026-07-21T19:27:05Z

Summary

Block outputs past the 16MB inline materialization ceiling aborted the run before masking even started — the redaction path hydrated the whole offloaded value at once and the pre-flight size assert fired on the manifest's total byteSize
Large-array manifests now redact chunk-by-chunk: page one stored chunk at a time, mask, re-store, rebuild via the manifest writer (preview regenerated from masked items, bookkeeping recomputed + asserted). Peak heap stays ~one chunk regardless of payload size
Single refs up to the 64MB durable cap hydrate with a raised budget and run serially outside the concurrency pool (one 30MB hydration holds ~2-3x its size in transient heap; 4 at once would OOM the medium-1x trigger.dev machines)
Mask-batch chunk requests retry transient failures (network errors, 408/429/5xx honoring Retry-After, runtime-level timeouts, socket closes) with jittered backoff — a single ALB blip or Presidio pod restart no longer fails a whole payload's redaction, and on the execution-altering stages no longer aborts the run
Fixed a latent double-mask: a nested ref whose masked value shrinks back under the offload threshold got re-inlined and masked twice (wrong for custom patterns); string pass now runs before ref substitution
Sidecar: NER_ENTITIES now intersects the SpacyRecognizer's claimed entities with what the loaded models' NER labels can actually produce. Presidio's default mapping claims PHONE_NUMBER/AGE/ID/EMAIL for transformer backends spaCy can't emit, so any rule naming PHONE_NUMBER silently forced the full spaCy pass and the regex-only fast path never fired

Type of Change

Bug fix

Testing

End-to-end against a live stack (local Presidio + S3 + dev DB): a 26.2MB / 40k-record function output that previously aborted instantly now masks in ~3.2min on a single sidecar worker with the realistic entity set (all 265 batches on the regex fast path; ~15min projected on full NER before the sidecar fix). All 40k items verified: order/structure preserved across chunks, emails/cards/SSNs/phones masked, manifest preview regenerated from masked content, skip-vs-full span parity confirmed for phones. Unit suites updated: manifest branch had zero coverage (isLargeArrayManifest was stubbed false) — now covered chunk-wise incl. partial-chunk failure in both failure modes. lint, check:api-validation:strict, and full lib/execution+logs+guardrails suites pass.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

…sient mask failures A block output past the 16MB inline materialization ceiling aborted the run before masking even started: the redaction path hydrated the whole offloaded value at once, and the pre-flight size assert fired on the manifest's total byteSize. Large-array manifests now page one stored chunk at a time (materialize -> mask -> re-store, rebuilt via the manifest writer with preview derived from masked items), so peak heap stays ~one chunk regardless of payload size. Single refs up to the 64MB durable cap hydrate with a raised budget and run serially outside the concurrency pool. Mask-batch chunk requests now retry transient failures (network errors, 408/429/5xx, honoring Retry-After) with jittered backoff, so a single ALB blip or Presidio pod restart no longer fails a whole payload's redaction. Nested-ref masking now runs the string pass before ref substitution, fixing a latent double-mask when a masked nested value shrinks back inline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

Verified end-to-end against a 26MB / 40k-record offloaded output: the chunk-wise path masks it in ~54s on a single local Presidio worker where the old path aborted at the 16MB ceiling. The exercise surfaced two more transient error shapes the retry classifier missed — runtime-level request timeouts (undici's default 300s headers timeout, Bun's TimeoutError) and mid-flight socket closes — both of which previously failed the whole payload's redaction on the first occurrence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

…produce The registry's SpacyRecognizers claim every entity in Presidio's default NER-model mapping — including PHONE_NUMBER/AGE/ID/EMAIL, which exist for transformer de-identification backends and which no spaCy model can emit. The NER_ENTITIES derivation trusted that claim, so any request naming PHONE_NUMBER (present in nearly every redaction rule) silently forced the full spaCy pass and the regex-only fast path never fired. Intersect the claimed set with the entities the loaded models' actual NER labels map onto; the hard floor of core NER entities is unchanged, and a future backend that genuinely emits phone labels would re-gate automatically. Verified live: PHONE_NUMBER-only requests take nlp=skip with span parity against the full path, PERSON still forces NER, and a 26MB/40k-record block-output redaction with the realistic entity set runs entirely on the fast path (~3.2min vs ~15min projected full-NER on one worker). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

vercel · 2026-07-21T19:27:11Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 21, 2026 7:54pm

cursor · 2026-07-21T19:27:17Z

PR Summary

Medium Risk
Changes core PII redaction, large-payload materialization, and Presidio routing—incorrect behavior could scrub runs or leak PII, though behavior is heavily tested and failures still scrub or throw per mode.

Overview
Fixes offloaded large payloads failing at the 16MB inline materialization limit by redacting large-array manifests chunk-by-chunk (read one stored chunk, mask, append to a new manifest) instead of hydrating the whole manifest at once. Preview is rebuilt from masked items; item counts are asserted. Single refs and manifests with oversized chunks hydrate with the 64MB durable budget and run serially behind a shared gate so concurrent hydrations do not OOM trigger workers or deadlock on nested oversized refs.

Mask-batch HTTP now retries transient failures (5xx, 408/429 with Retry-After, network/timeouts) with jittered backoff (8 attempts); 4xx and bad response shapes fail immediately. Internal tokens are minted per attempt.

Fixes double-masking when a nested ref shrinks back inline by running the string mask pass before nested ref substitution. On the Presidio sidecar, NER_ENTITIES is intersected with labels spaCy can actually emit so rules naming PHONE_NUMBER (and similar transformer-only types) no longer force a full spaCy pass and block the regex-only fast path.

^{Reviewed by Cursor Bugbot for commit bd2a30a. Bugbot is set up for automated code reviews on this repo. Configure here.}

greptile-apps · 2026-07-21T19:29:43Z

Greptile Summary

This PR updates PII masking for large offloaded payloads. The main changes are:

Redacts large-array manifests one stored chunk at a time.
Serializes hydration of oversized single values and chunks.
Retries transient masking-service failures with jittered backoff.
Prevents nested offloaded values from being masked twice.
Narrows spaCy processing to entities the loaded models can produce.
Adds coverage for large manifests, partial failures, retries, and oversized values.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.
The retry behavior remains bounded and preserves the existing failure handling.
Chunk rebuild failures continue to use the established ownership and cleanup lifecycle.

Important Files Changed

Filename	Overview
apps/pii/server.py	Limits spaCy-required entities to labels supported by the loaded NER models while retaining the core entity set.
apps/sim/lib/guardrails/mask-client.ts	Adds bounded retries for transient HTTP and transport failures, including Retry-After support and fresh tokens per attempt.
apps/sim/lib/guardrails/mask-client.test.ts	Adds coverage for retryable responses, network errors, timeouts, retry exhaustion, token renewal, and deterministic failures.
apps/sim/lib/logs/execution/pii-large-values.ts	Processes manifests chunk-by-chunk, serializes oversized hydration, and masks strings before replacing nested references.
apps/sim/lib/logs/execution/pii-large-values.test.ts	Adds coverage for oversized values, manifest rebuilding, masked previews, nested references, and partial failures.
apps/sim/lib/logs/execution/pii-redaction.ts	Updates masking-path documentation to reflect transient retry behavior.

_{Reviews (4): Last reviewed commit: "fix(pii): fail fast on a null mask-batch..." | Re-trigger Greptile}

… refs Review finding: a manifest whose packer emitted a chunk past the inline ceiling (one item larger than the chunk target) hydrates that chunk with the raised 64MB budget inside the REF_CONCURRENCY pool, so several such manifests could hydrate oversized blobs concurrently — the exact heap scenario the serial path exists to prevent. The serial gate now covers any ref whose hydration can exceed the inline ceiling: oversized single refs and manifests containing an oversized chunk. Normally-chunked manifests stay pooled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

TheodoreSpeaks · 2026-07-21T19:35:28Z

@greptile

TheodoreSpeaks · 2026-07-21T19:35:30Z

@cursor review

…tion passes Review finding: the serial gate was per-resolveReplacements invocation, so nested oversized refs discovered inside different pooled parents each got their own pool and could hydrate oversized blobs concurrently. A shared promise-chain gate now threads through the options from the entry points, and a reentrancy flag lets a gated ref's own nested oversized work run directly instead of deadlocking on the hold. Covered by a cross-parent max-in-flight assertion and a nested-oversized deadlock regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

TheodoreSpeaks · 2026-07-21T19:47:15Z

@greptile

TheodoreSpeaks · 2026-07-21T19:47:16Z

@cursor review

A 200 response with a null JSON body threw TypeError on the data.masked read, which the retry classifier treats as transient — burning the full retry budget on a deterministic shape failure. Null-guard the body so it throws the non-retryable shape error immediately. Also swap the gate test's inline setTimeout promise for sleep() to satisfy check:utils, which failed CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1JYstmLHk9qMGyBDqYRcJ

TheodoreSpeaks · 2026-07-21T19:54:32Z

@greptile

TheodoreSpeaks · 2026-07-21T19:54:33Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bd2a30a. Configure here.}

TheodoreSpeaks and others added 3 commits July 21, 2026 11:20

greptile-apps Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread apps/sim/lib/guardrails/mask-client.ts

Comment thread apps/sim/lib/logs/execution/pii-large-values.ts

cursor Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread apps/sim/lib/logs/execution/pii-large-values.ts Outdated

vercel Bot temporarily deployed to Preview July 21, 2026 19:34 Inactive

cursor Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread apps/sim/lib/logs/execution/pii-large-values.ts

vercel Bot temporarily deployed to Preview July 21, 2026 19:47 Inactive

cursor Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread apps/sim/lib/guardrails/mask-client.ts

vercel Bot temporarily deployed to Preview July 21, 2026 19:54 Inactive

cursor Bot reviewed Jul 21, 2026

View reviewed changes

TheodoreSpeaks merged commit bcad4e3 into staging Jul 21, 2026
20 checks passed

TheodoreSpeaks deleted the fix/pii-large-payload branch July 21, 2026 22:03

Uh oh!

Conversation

TheodoreSpeaks commented Jul 21, 2026

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

greptile-apps Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

TheodoreSpeaks commented Jul 21, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 21, 2026 •

edited

Loading

cursor Bot commented Jul 21, 2026 •

edited

Loading

greptile-apps Bot commented Jul 21, 2026 •

edited

Loading