Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ name = "add_profanity"
path = "src/add_profanity.rs"
required-features = ["regex"]

[[example]]
name = "leetspeak_replacements"
path = "examples/leetspeak_replacements.rs"
required-features = ["customize"]

[features]
default = ["censor", "context"]
censor = ["arrayvec", "bitflags", "lazy_static", "itertools", "unicode-normalization", "rustc-hash"]
Expand Down
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,27 @@ If you want to add custom profanities or safe words, enable the `customize` feat
}
```

Digit-heavy text, like zip codes, can cause false positives via digit-to-letter substitutions (e.g. `9` → `g`, `0` → `o`). Zip codes obviously aren't a censor target, but got hit in practice in a chat bot's gas price feature which included zip code lookup. You can remove specific mappings from the default `Replacements` table:


```rust
#[cfg(feature = "customize")]
{
use rustrict::Replacements;

// SAFETY: call once, synchronously, before any concurrent access to censoring
// (e.g. at startup, before spawning any tasks that censor text).
unsafe {
let replacements = Replacements::customize_default();
replacements.remove('9', 'g');
replacements.remove('0', 'o');
// ...repeat for any substitution causing false positives in your use-case.
}
}
```

See [`examples/leetspeak_replacements.rs`](examples/leetspeak_replacements.rs) for a runnable version.

If your use-case is chat moderation, and you store data on a per-user basis, you can use `rustrict::Context` as a reference implementation:

```rust
Expand Down
22 changes: 22 additions & 0 deletions examples/leetspeak_replacements.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
//! Demonstrates removing specific digit/letter substitutions from the default
//! `Replacements` table. Useful when digit-heavy text (zip codes, numeric IDs,
//! prices, etc.) is decoding into flagged words via leetspeak-style
//! substitutions (e.g. `9` -> `g`, `0` -> `o`) that don't apply to your use-case.
//!
//! Run with: `cargo run --example leetspeak_replacements --features customize`

use rustrict::Replacements;

fn main() {
// SAFETY: call once, synchronously, before any concurrent access to censoring
// (e.g. at startup, before spawning any tasks that censor text).
unsafe {
let replacements = Replacements::customize_default();
replacements.remove('9', 'g');
replacements.remove('0', 'o');
// ...repeat for any substitution causing false positives in your use-case.
}

// Digits still match themselves; only the letter interpretations are removed.
println!("Replacements customized; '9' no longer maps to 'g', '0' no longer maps to 'o'.");
}