Skip to content

fix(scrubbing): leave query params the SDK does not redact untouched #1200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
solnic merged 1 commit into from
Sep 11, 2026
Merged

fix(scrubbing): leave query params the SDK does not redact untouched #1200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 31 additions & 9 deletions lib/sentry/scrubber.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,15 +332,37 @@ defmodule Sentry.Scrubber do
keys = param_keys(opts)

query
|> URI.query_decoder()
|> Enum.map(fn {key, value} ->
cond do
sensitive_key?(key, keys) -> {key, @scrubbed_value}
is_binary(value) and value =~ credit_card_regex() -> {key, @scrubbed_value}
true -> {key, value}
end
end)
|> URI.encode_query()
|> String.split("&")

@whatyouhide whatyouhide Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you not using URI.decode_query/1 here (and below) because it'd lose ordering?

@solnic solnic Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we want to preserve original state, order included, and also prevent cases like ending up with ********* placeholder being converted to %2A%2A%2A%2A%2A%2A%2A%2A%2A.

|> Enum.map_join("&", &scrub_query_pair(&1, keys))
end

# Only a redacted pair is rewritten. Everything the SDK keeps is passed
# through exactly as it arrived, so the reported query string still matches
# what the client sent rather than a re-encoding of it.
defp scrub_query_pair(pair, keys) do
case String.split(pair, "=", parts: 2) do
[raw_key, raw_value] ->
if redact_param?(decode_www_form(raw_key), decode_www_form(raw_value), keys) do

@sentry sentry Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The scrub_query_string implementation calls String.downcase on keys that may be non-UTF-8 binaries, causing a UnicodeConversionError and crashing the scrubber.
Severity: HIGH

Suggested Fix

The call to String.downcase within sensitive_key? should be wrapped in a try/rescue block to handle potential UnicodeConversionError. When an error is caught, the function should treat the key as a non-sensitive binary and proceed without attempting to downcase it, preventing the crash.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: lib/sentry/scrubber.ex#L345

Potential issue: The `scrub_query_string` function processes query strings by decoding
them with `decode_www_form`. If a key contains percent-encoded non-UTF-8 bytes (e.g.,
`%FF%FE`), `URI.decode_www_form` produces a non-UTF-8 binary. This binary is then passed
to `sensitive_key?`, which calls `String.downcase`. `String.downcase` raises a
`UnicodeConversionError` on non-UTF-8 input. The existing `rescue` block only catches
`ArgumentError`, so the `UnicodeConversionError` is unhandled and will crash the
scrubber process. This will cause a test at `test/sentry/scrubber_test.exs:176` to fail.

Did we get this right? 👍 / 👎 to inform future reviews.

raw_key <> "=" <> @scrubbed_value
else
pair
end

[_without_value] ->
pair
end
end

defp redact_param?(key, value, keys) do
sensitive_key?(key, keys) or value =~ credit_card_regex()
end

# Scrubbing runs while an error is already being reported, so malformed
# percent-encoding must not raise. The raw form is still fine to match on.
defp decode_www_form(value) do
URI.decode_www_form(value)
rescue
ArgumentError -> value

@whatyouhide whatyouhide Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When does this raise?

@solnic solnic Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question - in my testing nowhere, but BugBot pointed out that if it did, we'd have an unwanted crash. It is very defensive, I have to admit - do you think it's 100% safe to remove that rescue?

@whatyouhide whatyouhide Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say so yes.

end

@doc """
Expand Down
3 changes: 1 addition & 2 deletions test/plug_capture_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,6 @@ defmodule Sentry.PlugCaptureTest do

@token "SEKRIT-TOKEN-VALUE"
@redacted Sentry.Scrubber.scrubbed_value()
@encoded_redacted URI.encode_www_form(Sentry.Scrubber.scrubbed_value())

setup %{bypass: bypass} do
Application.put_env(:sentry, PhoenixEndpointWithUrlScrubber,
Expand Down Expand Up @@ -420,7 +419,7 @@ defmodule Sentry.PlugCaptureTest do

assert [%{"exception" => [%{"value" => value}]}] = SentryTest.collect_sentry_events(ref, 1)

assert value =~ ~s(query_string: "token=#{@encoded_redacted}")
assert value =~ ~s(query_string: "token=#{@redacted}")
end

test "redacts a route parameter named like a credential", %{ref: ref} do
Expand Down
4 changes: 2 additions & 2 deletions test/sentry/live_view_hook_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,11 +260,11 @@ defmodule Sentry.LiveViewHookTest do
params_breadcrumb = Enum.find(context.breadcrumbs, &(&1.category == "web.live_view.params"))

refute params_breadcrumb.data.uri =~ "supersecret"
assert params_breadcrumb.data.uri =~ "password=%2A%2A%2A%2A%2A%2A%2A%2A%2A"
assert params_breadcrumb.data.uri =~ "password=#{Sentry.Scrubber.scrubbed_value()}"
assert params_breadcrumb.data.uri =~ "visible=ok"

refute context.request.url =~ "supersecret"
assert context.request.url =~ "password=%2A%2A%2A%2A%2A%2A%2A%2A%2A"
assert context.request.url =~ "password=#{Sentry.Scrubber.scrubbed_value()}"
end

test "raises ArgumentError when :scrubber is not an MFA tuple" do
Expand Down
16 changes: 6 additions & 10 deletions test/sentry/plug_context_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,15 +120,15 @@ defmodule Sentry.PlugContextTest do
conn = conn(:get, "/test?password=hunter2")
call(conn, [])

assert "http://www.example.com/test?password=#{encoded_scrubbed_value()}" ==
assert "http://www.example.com/test?password=#{Sentry.Scrubber.scrubbed_value()}" ==
Sentry.Context.get_all().request.url
end

test "scrubs query string by default" do
conn = conn(:get, "/test?password=hunter2&hello=world")
call(conn, [])

assert "password=#{encoded_scrubbed_value()}&hello=world" ==
assert "password=#{Sentry.Scrubber.scrubbed_value()}&hello=world" ==
Sentry.Context.get_all().request.query_string
end

Expand All @@ -143,21 +143,21 @@ defmodule Sentry.PlugContextTest do
conn = conn(:get, "/test?password=hunter2&hello=world")
call(conn, url_scrubber: fn _conn -> raise "custom scrubber bug" end)

assert "http://www.example.com/test?password=#{encoded_scrubbed_value()}&hello=world" ==
assert "http://www.example.com/test?password=#{Sentry.Scrubber.scrubbed_value()}&hello=world" ==
Sentry.Context.get_all().request.url

assert "password=#{encoded_scrubbed_value()}&hello=world" ==
assert "password=#{Sentry.Scrubber.scrubbed_value()}&hello=world" ==
Sentry.Context.get_all().request.query_string
end

test "falls back to the default URL scrubber when a custom scrubber returns a non-binary" do
conn = conn(:get, "/test?password=hunter2&hello=world")
call(conn, url_scrubber: fn _conn -> %{unexpected: "value"} end)

assert "http://www.example.com/test?password=#{encoded_scrubbed_value()}&hello=world" ==
assert "http://www.example.com/test?password=#{Sentry.Scrubber.scrubbed_value()}&hello=world" ==
Sentry.Context.get_all().request.url

assert "password=#{encoded_scrubbed_value()}&hello=world" ==
assert "password=#{Sentry.Scrubber.scrubbed_value()}&hello=world" ==
Sentry.Context.get_all().request.query_string
end

Expand Down Expand Up @@ -296,8 +296,4 @@ defmodule Sentry.PlugContextTest do
defp call(conn, opts) do
Plug.run(conn, [{Sentry.PlugContext, opts}])
end

defp encoded_scrubbed_value do
URI.encode_www_form(Sentry.Scrubber.scrubbed_value())
end
end
35 changes: 35 additions & 0 deletions test/sentry/scrubber_test.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,13 @@ defmodule Sentry.ScrubberTest do
assert Scrubber.scrub_url("http://example.com/foo") == "http://example.com/foo"
end

test "leaves everything it does not redact exactly as it arrived" do
url = "http://example.com/reset%20me?password=x&keep=hello%20there&flag"

assert Scrubber.scrub_url(url) ==
"http://example.com/reset%20me?password=#{Scrubber.scrubbed_value()}&keep=hello%20there&flag"
end

test "preserves scheme, host, port, and path" do
scrubbed = Scrubber.scrub_url("https://example.com:8443/p?secret=x")
assert scrubbed =~ "https://example.com:8443/p?"
Expand Down Expand Up @@ -175,6 +182,34 @@ defmodule Sentry.ScrubberTest do
refute scrubbed =~ "hunter2"
assert scrubbed =~ "keep=ok"
end

test "leaves the placeholder readable rather than percent-encoding it" do
scrubbed = Scrubber.scrub_query_string("password=hunter2")

assert scrubbed == "password=#{Scrubber.scrubbed_value()}"
refute scrubbed =~ "%2A"
end

test "passes through params it keeps byte for byte" do
assert Scrubber.scrub_query_string("greeting=hello%20there") == "greeting=hello%20there"
assert Scrubber.scrub_query_string("greeting=hello+there") == "greeting=hello+there"
assert Scrubber.scrub_query_string("a=1&&b=2") == "a=1&&b=2"
assert Scrubber.scrub_query_string("flag&visible=ok") == "flag&visible=ok"
end

test "keeps the key as it was sent when redacting its value" do
assert Scrubber.scrub_query_string("Reset%2DToken=abc") ==
"Reset%2DToken=#{Scrubber.scrubbed_value()}"
end

test "does not raise on malformed percent-encoding" do
assert Scrubber.scrub_query_string("a=%ZZ") == "a=%ZZ"
end

test "still redacts credit-card-shaped values" do
assert Scrubber.scrub_query_string("card=4111+1111+1111+1111&keep=a+b") ==
"card=#{Scrubber.scrubbed_value()}&keep=a+b"
end
end

describe "scrub/1 with no registered scrubber" do
Expand Down
Loading