Skip to content
#

crows-pairs

Here are 4 public repositories matching this topic...

Language: All
Filter by language

Reproduction pack for "Auditing Bias Detection Under Source Shift" (AACL-IJCNLP 2026, main conference). 10,304 audited sentences from three sources with counterfactual pairs, a two-phase LLM audit validated against human annotation, and a pair-aware, leave-one-source-out evaluation protocol.

  • Updated Nov 2, 2026
  • Jupyter Notebook

Déjà vu Bias: your bias benchmark has been here before. CrowS-Pairs and StereoSet leaked into OLMo-1.7's pretraining data through the instruction-tuning supply chain (SuperNI → FLAN → Tulu → Dolma), up to 1,728 copies per sentence. We trace the leak into the training shards and measure its effect on bias scores: none detectable.

  • Updated Nov 18, 2026
  • Python

Add this topic to your repo

To associate your repository with the crows-pairs topic, visit your repo's landing page and select "manage topics."

Learn more