Skip to content

Harden API + fix bugs (v3.1) - #7

Merged
willscott-v2 merged 3 commits into
mainfrom
fix/hardening-and-bugs
Jul 21, 2026
Merged

willscott-v2 merged 3 commits into
mainfrom
fix/hardening-and-bugs

Conversation

@willscott-v2

Copy link
Copy Markdown
Collaborator

Supersedes #6, which GitHub auto-closed when its stacked base branch (feature/si-design-system, now merged via #5) was deleted. Identical content, retargeted to main.

Security

  • SSRF guards (api/chunk.js): http/https only; DNS-resolve the host and reject private/reserved ranges (127/10/172.16/192.168, 169.254 metadata, 100.64 CGNAT, IPv6 loopback/ULA/link-local); every redirect hop re-validated (manual redirect, max 5); 5MB response cap; content-type allowlist.
  • XSS: escapeHtml now escapes " and ' — closes the title="${fragment}" attribute injection from scraped heading ids.
  • Slack injection (api/feedback.js): escape &<> in message/email/pageUrl/user-agent; pageUrl must be http(s), length-capped.
  • Input validation: allowlist chunkSize/strategy/extract/format/tokenizer → 400 not 500; overlap bounded 0–200.
  • Error leak: stop returning raw error.message.

Bugs

  • Overlap slider was permanently stuck on Auto — unreachable else branch; verified fixed.
  • Dead fetch timeout — node-fetch v3 ignored timeout:; replaced with native fetch + AbortSignal.timeout(25s), node-fetch dependency removed.
  • extract=defuddle now fails 422 instead of silently falling back to cheerio.

Housekeeping

  • Removed scripts/build.js + vercel buildCommand; GTM-4G43 hardcoded (shared SI container).
  • Deleted ~1,900 lines of unused root design-system files.
  • README: MIT license, searchinfluence/getchunks URLs, honest data-security FAQ.

Rate limiting — done separately

Two Vercel Firewall rules are already live on the project (/api/chunk 20/60s/IP, /api/feedback 5/600s/IP, deny-on-exceed). Applied via the Firewall API; no code in this PR.

Testing

Local harness + Playwright: SSRF/scheme/validation rejections, live extraction, http→https redirect following, slider fix (25→request), XSS containment.

🤖 Generated with Claude Code

willscott-v2 and others added 3 commits July 21, 2026 13:12
Security:
- SSRF guards in api/chunk.js: http/https only, DNS-resolve + reject
  private/reserved IP ranges (incl. 169.254 metadata, CGNAT, IPv6 ULA),
  per-hop redirect re-validation, 5MB body cap, content-type allowlist
- escapeHtml now escapes quotes — closes title="${fragment}" attribute
  XSS from scraped heading ids
- Slack mrkdwn escaping in feedback.js kills <!channel> pings + link
  spoofing; pageUrl scheme-checked and length-capped
- Input validation with allowlists (400 not 500); error details no longer
  leak to clients

Bugs:
- Overlap slider was permanently stuck on Auto (unreachable else branch)
- Dead node-fetch timeout replaced with native fetch + AbortSignal.timeout;
  node-fetch dependency removed
- extract=defuddle now fails 422 instead of silently falling back to cheerio

Housekeeping:
- Removed scripts/build.js + vercel buildCommand; GTM-4G43 (shared SI
  container, matches grader/ontologizer) hardcoded; GTM_SETUP.md rewritten
- Deleted ~1,900 lines of unused root design-system files
- README: MIT license, searchinfluence/getchunks URLs, v3/v3.1 history,
  honest data-security FAQ

Verified locally: SSRF/scheme/validation rejections, live extraction,
http->https redirect following, slider fix, and XSS containment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@willscott-v2
willscott-v2 merged commit 3a785d3 into main Jul 21, 2026
3 checks passed
@willscott-v2
willscott-v2 deleted the fix/hardening-and-bugs branch July 21, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant