Resolve set_config() arguments from the Bind message and let Postgres answer it - #1298
Resolve set_config() arguments from the Bind message and let Postgres answer it#1298IgorOhrimenko wants to merge 11 commits into
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This is a bug in the parser. It should be able to extract |
|
Thanks — done. The parser now pulls
Making those calls take the Tests added at three levels: parser (bound values, NULL, both fallbacks), protocol (the full extended-protocol exchange — fails without the |
|
Related: #1302 fixes a third way session state survives checkin — a client's SQL-level |
ca3a653 to
b623a6f
Compare
|
The failing job here is The same job passed on this exact code six minutes earlier, and it has failed on I also measured whether the extra test database this PR adds to #1303 replaces the fixed sleep with a poll, which should stop this from flipping. |
|
Same comment as on #1299. We should not be using the "dirty" flag here. We should keep track of parameters and change them as needed for each client. I think for this particular scenario it would be easier to:
This is all that's needed for our existing logic to handle the parameter diff. For this to work, you may just need to add |
b623a6f to
c2c6302
Compare
|
Done, both parts. The parser reads And it is forwarded rather than intercepted: take a server, record the parameter change on that connection, send the statement, let Postgres answer. That removes the invented response along with its protocol bug — describing the portal claimed no rows and then sent one, so libpq rejected the reply with This is stacked on #1299, which carries the recording mechanism; the diff here shows both, its last two commits are this PR. Arguments that still don't resolve — no Bind, a non-text parameter, an expression — leave the statement an ordinary query, as today. Covering that case would need a way to record "this parameter is now something we can't name", and I didn't want to invent one here. Say the word if you'd rather it were handled. |
|
I reproduced this independently against Setup: #1299 fixes the mid-transaction #1298 fixes bind-parameter The gap: Where that bites, stated precisely — it's topology-dependent, and I measured both
Measured on this branch, single primary, no replicas:
Adding a replica to the same config flips the first row to clean, which is what This is pre-existing, not something this PR introduces — the PR touches 12 files I think the reason CI doesn't surface it is the new test config, which pins
Sensible for determinism, but One pattern closes it: /// `SELECT set_config(...)` changes session state exactly like `SET`, but it is
/// a function call inside a `SELECT`, so it never matches the statement-start
/// patterns above.
static CMD_SET_CONFIG: &[&str] = &[r"(?i)\bset_config\b"];
fn cmd_base_patterns() -> impl Iterator<Item = String> {
CMD_BASE
.iter()
.map(|cmd| format!("{}{}", COMMENT_PREFIX, cmd))
.chain(CMD_SET_CONFIG.iter().map(|s| s.to_string()))
}Chaining into One note on priority. The CREATE POLICY tenant_isolation ON patients
USING (org_id = current_setting('app.current_org_id')::uuid);and Separately, and maybe worth a docs line: after a targeted |
c2c6302 to
066be97
Compare
|
Thank you — this is the most useful thing anyone has done to this PR. Measuring You were right about the test config, and that was the part I'd got wrong: I Fixed, and the fixture now says so: two copies of the same single-primary I also added a test for the silent mode you described, because it deserved to Both of those fail at Which is why I'd rather not take your patch into this PR. You found it, you Two things I found while confirming your results, both outside what that patch
All three leak at And thank you for the |
A SET or RESET the client sends once a server is attached changes that server's session, but we never wrote it down. What we did instead was clear the whole parameter cache whenever a CommandComplete said RESET, which trades one problem for another: the cache is what tells us to undo the change for the next client, and a ROLLBACK undoes the RESET anyway. pg_dump -t <table> walks straight into it: SET search_path TO '', then RESET search_path inside its transaction, then ROLLBACK. Postgres brings the empty search_path back, the cache no longer mentions it, and the next client gets the connection with unqualified names silently broken. A SET has the same hole in the other direction: BEGIN, a query, SET statement_timeout, COMMIT — the setting stays on the server and nothing resets it for whoever comes next. So record the change where it happens. RESET now has the transaction handling SET always had (reset vs reset_transaction), so a rollback restores what it cleared and a commit makes it permanent, and the server connection keeps the same record its client does. The existing parameter diff then does the rest: the next client is handed a precise RESET for what it doesn't want, instead of a connection nobody dares reuse. The CommandComplete fallback stays for RESETs we don't see coming — with the query parser off, that is still all we have.
Runs against a database with a single server connection, so the next client always gets the connection the previous one used.
The keys still have to be lifted out of the maps before we can reset them, but there is no reason to clone the ones we are about to drop.
The parser only understood constants, so a parameterized call went
through untouched and whatever it changed stayed on the connection:
SELECT pg_catalog.set_config($1, $2, false)
Read $n from the Bind message instead. The is_local argument is decoded
from either wire format. Arguments that still don't resolve — no Bind
message, a parameter that isn't text, an expression — leave the statement
an ordinary query, same as today: it runs, and we don't know what it
changed.
`SELECT set_config(...)` was intercepted and answered locally, which meant inventing a response for a query the client asked Postgres to run: the tag was SET where Postgres sends SELECT 1, and describing the portal claimed no rows and then sent one, which libpq rejects outright. It is a query, so treat it as one — take a server, record what the statement changes on that connection, and forward it. The client gets Postgres' own answer, and the next client gets the connection with that parameter reset. Renamed the flag that marks these statements: it no longer describes how we imitate a SELECT, it says the statement is one.
The test pinned down the value PgDog made up while pretending to run the statement: the SetParam it parsed, so NULL for a reset. Postgres resets the setting and answers with the value it landed on, and that is what the client gets now that the statement reaches it.
The fixture pinned the leak database to level "on", which is the one level that always parses. Everything a statement has to get past to reach the parser at the default level went untested, and set_config() is a function call inside a SELECT: it matches no statement-start keyword, so the gate drops it and the value stays on the connection. A second copy of the same single-primary database at "auto" covers that path. Both copies run every test. The new test covers the half of the leak that reports nothing: row-level security keyed on a custom GUC is how multi-tenant applications isolate tenants, set_config() is how that GUC gets set, and a value that outlives its client makes the next one read as the previous tenant. Reads go through a plain role because the pooler's user is a superuser and superusers ignore RLS.
main dropped the second parser in pgdogdev#1324, and this branch still carried the changes for both. Resolving the rebase left one cfg_select! block and a stray blank line behind.
066be97 to
ff3c387
Compare
It passed in CI and failed locally, which means it was answering a question it never asked: with a different server connection there is nothing for the first client to have left behind, and the assertion holds for the wrong reason. Compare pg_backend_pid() across the two clients, and separate the GUC outliving its client from row-level security failing to filter, so a failure says which of the two happened.
|
On the two dead ends you listed for the
The full startup set is 15: So Custom GUCs are not, and you're right about And with two Why the if !changed_params.is_empty() {
for (name, value) in changed_params.iter() {
context.params.insert(name.clone(), value.clone());
}i.e. the change is recorded as the client's new intent rather than as state left on the connection. At the next checkout That's a code read, not a tested fix — I haven't tried closing it, and it's close enough to the parameter-tracking rework in your thread that it may already be covered by whatever shape that takes. Splitting it that way: the reported GUCs look reachable with the existing machinery, and the custom ones need something else entirely. I opened #1327 for the regex gate, so the |
Problem
set_config()interception only understood constant arguments, so a parameterized call went through untouched:Whatever it changed stayed on the connection, and the next client inherited it —
42P01on unqualified names until the connection was recycled.The interception itself had a second problem.
SELECT set_config(...)was answered locally, which meant inventing a response for a query the client asked Postgres to run: the tag wasSETwhere Postgres sendsSELECT 1, and describing the portal claimed no rows and then sent one, which libpq rejects withD message without prior T.Fix
Read
$nfrom the Bind message, so a parameterizedset_config()resolves like a constant one. Theis_localargument is decoded from either wire format.Then stop faking the answer. It is a query, so treat it as one: take a server, record what the statement changes on that connection (#1299's mechanism), and forward it. The client gets Postgres' own reply, and the next client gets the connection with that parameter reset — the invented response and its protocol bug are gone rather than fixed.
Arguments that still don't resolve — no Bind message, a parameter that isn't text, an expression such as
set_config('search_path', current_setting('x'), false)— leave the statement an ordinary query. That is unchanged from today: it runs, and we don't know what it changed. Covering it would need a way to say "this parameter is now something we can't name", which I left out of this PR.The flag marking these statements is renamed: it no longer describes how we imitate a
SELECT, it says the statement is one.Testing
$1,$2,$3resolve to the bound values; a NULL parameter becomes a reset; a binaryis_localis decoded; arguments with no Bind, a non-UTF-8 value, or an expression stay an ordinary query.integration/python/test_session_params_leak.py::test_set_config_bound_params: a client poisons the pool through bound parameters, a later client must see a cleansearch_path. Fails on Track a client's parameter changes on the server connection #1299 alone, passes here.RESET "search_path"when the connection is handed over — no synthesized rows, noRESET ALL.Tests now cover both parser levels
@Bougerous reproduced this independently and pointed out that the test config
here pinned
pgdog_leaktolevel = "on"— the one level that skips the regexgate, so nothing exercised the path a statement takes at the default level.
That was a fair hit:
set_config()is a function call inside aSELECT, itmatches no statement-start pattern, and on a single-primary cluster
autoleaves it to the gate, which drops it.
So the fixture is now two copies of the same single-primary database differing
only in parser level —
pgdog_leakaton,pgdog_leak_autoatauto— andevery test runs against both.
A second test covers the half of this that reports nothing. Row-level security
keyed on a custom GUC is how multi-tenant applications isolate tenants, and
set_config()with a bound parameter is how that GUC gets set; a value thatoutlives its client makes the next one read as the previous tenant, with no
error at all. Reads go through a plain role, since the pooler's own user is a
superuser and superusers ignore RLS.
This leaves two tests failing at
auto, and they stay that way until thegate learns about
set_config(). That fix belongs to @Bougerous, who found itand offered the patch — it is one pattern in
regex_parser.rs, and it is notthis PR's code (this PR doesn't touch that file). Merging his change first
turns both tests green here with nothing else to do.