Skip to content

Replace external ruleset with curated subset of gitleaks' MIT-licensed rules #56

Description

@fahadsiddiqui

Problem

data/regex_list_1.json is a generic third-party DLP sample list (Arista/Cisco configs, US bank routing numbers, Box links, CVE IDs, etc.) with mixed quality — much of it irrelevant to a coding-assistant use case. After the coding-only keyword filter, only 30 of 66 entries survive, and (per #3) some of those are broken. The dataset is ballast rather than a curated asset.

Fix

Replace or supplement data/regex_list_1.json with a converted subset of gitleaks' MIT-licensed ruleset (https://github.com/gitleaks/gitleaks), which is actively maintained, vastly higher precision, and purpose-built for this exact use case (secrets in code/text).

Suggested approach:

  • Write a one-time conversion script (scripts/convert-gitleaks-rules.ts or similar) that maps gitleaks TOML rules into this project's external-rule JSON schema
  • Preserve the existing stable-ID scheme so config references (disabledRules, etc.) don't break for existing users
  • Vet every converted rule through the existing safe-regex2 + empty-match checks before inclusion
  • Drop routing-number/networking-config style entries that don't apply to this product's threat model

Acceptance criteria

  • data/regex_list_1.json (or its replacement) is derived from gitleaks' ruleset with clear attribution/license notice
  • All included rules pass safe-regex2 vetting with zero rejections at load time
  • Conversion script is committed so the ruleset can be refreshed later
  • Existing tests (rule-loading, FP corpus) still pass

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions