Skip to content

fix(ai-service): enforce PII scrubbing on the humanitarian verification path - #452

Open
Degentle12 wants to merge 1 commit into
ChainForgee:mainfrom
Degentle12:fix/pii-scrubbing-humanitarian-verification
Open

fix(ai-service): enforce PII scrubbing on the humanitarian verification path#452
Degentle12 wants to merge 1 commit into
ChainForgee:mainfrom
Degentle12:fix/pii-scrubbing-humanitarian-verification

Conversation

@Degentle12

Copy link
Copy Markdown

Closes #430

Problem

/v1/ai/humanitarian/verify transmitted raw recipient evidence (names, locations, phones, emails, IDs) straight to OpenAI/Groq. The PII scrubber was only reachable as an opt-in /anonymize endpoint, so nothing enforced anonymization before the provider call.

Changes

  • Mandatory preprocessing stage in HumanitarianVerificationService: aid_claim, every supporting_evidence entry, and string-valued context_factors are scrubbed before any prompt is constructed, so only masked tokens ever reach a provider.
  • Fail closed: new PII_SCRUBBING_ENABLED setting (default true). When scrubbing is disabled or raises, the request is rejected with success=false and no provider call is made — raw text is never sent unredacted.
  • Response envelope: verification results now include a pii_scrubbing block (applied, anonymized, pii_summary) so callers can distinguish scrubbed input from raw; raw text remains on the backend (human-review) side only.
  • Audit: when PII_DECISIONS_ENABLED=true, each verification persists an aggregate-only record (entity/token counts + non-reversible SHA-256 fingerprint, never text) to the pii_decisions store, mirroring the /v1/ai/anonymize audit path.
  • Docs: the AI service README documents which endpoints scrub PII, which do not, and the residual-risk posture of the regex/spaCy detector.

Verification

New tests in tests/test_humanitarian_pii_scrubbing.py:

  • the mocked provider payload contains no raw PII for seeded evidence (names/phones/emails/locations are masked);
  • fail-closed when the scrubber is unavailable — provider is never called;
  • fail-closed when scrubbing is disabled — service-level and route-level;
  • aggregate audit record is persisted and never stores raw or anonymized text;
  • PII-free evidence passes through unchanged.

Full suite: 256 passed. flake8 --select=E9,F63,F7,F82 clean; test_setup.py passes.

Out of scope

Improving scrubber detection precision and backend-side metadata scrubbing (#213) remain separate concerns as scoped in the issue.

…on path

The PII scrubber was only reachable as an opt-in /anonymize endpoint, so
/v1/ai/humanitarian/verify transmitted raw recipient evidence (names,
locations, phones, emails, IDs) to OpenAI/Groq unredacted. Wire the
scrubber in as a mandatory preprocessing stage before prompt construction:

- Scrub aid_claim, supporting_evidence, and string-valued context_factors
  before any provider call; only masked tokens reach the provider.
- Fail closed: when PII_SCRUBBING_ENABLED=false or scrubbing itself fails,
  the request is rejected and no provider call is made.
- Expose a pii_scrubbing block (applied/anonymized/pii_summary) in the
  verification response so callers can tell scrubbed input from raw; raw
  text stays on the backend side only.
- Persist aggregate-only scrub metadata (counts + SHA-256 fingerprint,
  never text) to the pii_decisions store when enabled.
- Document the scrubbing posture and residual risk in the AI service README.

Closes ChainForgee#430
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pii_scrubber is never applied on the LLM verification path: raw evidence text is sent to OpenAI and Groq

1 participant