Bind named parameters by the name the statement declares - #352
Open
RagingKore wants to merge 5 commits into
Open
Bind named parameters by the name the statement declares#352RagingKore wants to merge 5 commits into
RagingKore wants to merge 5 commits into
Conversation
A clean clone needed two builds. The first downloaded the libraries and copied none of them; a second build then picked them up. Three separate defects produced that. The download guard was the existence of obj/runtimes/<rid>/native. It recorded no version, so raising the DuckDB release left every machine that had built once on the old engine, and a checkout targeting v1.5.5 kept running v1.2.0 and failed 127 tests with EntryPointNotFoundException. MakeDir also ran before DownloadFile, so a download that failed left an empty directory that satisfied the guard and every later build skipped in silence. The libraries were staged under obj, which the SDK lists in DefaultItemExcludes. Every item glob over them matched nothing and reported nothing, so the files reached no output directory and a cold dotnet pack shipped DuckDB.NET.Bindings.Full with no native library in it. The download now runs during restore, which is the only phase that precedes evaluation, where item globs expand. The hook is a NuGet target rather than the public Restore target, because Restore only runs on the project the command names and a solution build would skip every project inside it. It carries no compatibility guarantee, so if a later SDK renames it the symptom is a missing library on a clean clone. Staging moves to artifacts/natives/<version>/<rid>/native. That is outside obj, so globs see it, and the version in the path makes the directory the cache key: a bump lands in a fresh folder and cannot inherit the previous engine. Incrementality follows the SDK cache file convention, where a value rather than a source file drives the decision, as CreateGeneratedAssemblyInfoInputsCacheFile does in Microsoft.NET.GenerateAssemblyInfo.targets. The download cache file is written last, so a failure leaves it older than the inputs and the next build retries. The fetch also completes before the installed payload is cleared, so a download that fails over a dead network leaves the working engine in place. Bindings declares the libraries as Content in the project body. Content flows across a ProjectReference, so the test, benchmark and sample projects each get the engine from the reference they already had and their own globs are gone. Copying is unconditional because every project that runs needs the engine beside its assembly; only packing stays gated on BuildType. DuckDbVersion moves to Directory.Build.props and drives the artifact URL, the staging path and both packages' release notes, which each carried the version as hand typed prose. The release and nightly download targets differed in one step and are now one target. DuckDbArtifactRoot still wins when set, which the nightly CI job does through the environment.
A NuGet package writes a runtimeTargets entry into deps.json and the host then
probes runtimes/{rid}/native on its own. A project reference produces no such
entry, so every project in this repository loaded the library itself: the test
project through a module initializer, the benchmarks through a second copy of
the same code, and the samples by compiling the test project's file through a
Compile Include that reached across projects.
DuckDBNativeLibrary registers a DllImportResolver for the Bindings assembly,
which is where DuckDbLibrary and its imports live. One copy of the runtime
identifier logic now serves every consumer, in this repository and downstream.
The resolver returns zero for a name it does not own and for a file it cannot
find, so default probing still applies and a library already present on the
system still loads. It passes a relative path with
DllImportSearchPath.AssemblyDirectory, which keeps it working under single file
publish where Assembly.Location is empty, and the four argument overload applies
the platform naming convention so one name matches libduckdb.dylib,
libduckdb.so and duckdb.dll.
Raising the version used to touch three files and nothing checked that they agreed. Commit fd45dbb updated the artifact URL and left both packages' release notes on the previous version. It is one property now, so say where it is.
The named branch of BindParameters looped over the parameter collection and asked duckdb_bind_parameter_index for each ParameterName. A name that the statement did not declare returned a failure that the loop discarded, so the value bound nothing and the caller got no report. Closes Giorgi#203. A ParameterName can now carry the SQL prefix that other ADO.NET providers accept: an entry named "$PARM1" binds a statement that declares $PARM1. Only "$" is stripped, because a quoted parameter name can start with "@" and stripping that character would let an entry named "@foo" answer for a declared "foo". The loop now runs over the statement. It reads each declared name with the new duckdb_parameter_name binding, finds the value in the collection, and throws InvalidOperationException that names the parameter when no entry supplies it. This changes behavior. A declared parameter with no supplied value now throws before execution, where it previously bound nothing and let the engine report a later error that named nothing. Code that relied on the old behavior was already producing a failed query. IndexOf(string) gains the same tolerance, so a caller who adds "$id" can retrieve it by "id".
Five facts became two theories. The bodies were identical apart from one literal, and each case keeps the comment that ties it to the edge case it defends.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #351. That one needs to merge first. The first 13 files in this diff belong to it. This PR's own change is 8 files.
Closes #203.
TL;DR
This did not bind:
The statement declares the parameter as
PARM1, without the marker, so aParameterNameof$PARM1matched nothing. Other ADO.NET providers accept the prefix. Now this one does.Fixing it exposed a second defect. This bound nothing at all and reported nothing:
The engine then failed with a message naming no parameter. It now throws
InvalidOperationException: No value supplied for parameter 'name'.before execution.The bug
No
else. A name the statement does not declare returns a failure the loop discards. Both cases above are that missing branch.The loop also asks the wrong question. "Does the statement want this value?" has no sensible failure action, because unused values are legal. "Do I have a value for this parameter?" does.
The fix
The loop runs over the statement instead. It reads each declared name through a new
duckdb_parameter_namebinding, finds the value in the collection, and throws when none supplies it.Matching makes two passes: an exact match over the whole collection first, then one that strips a leading
$. Two passes because a quoted name can itself start with$:One pass would let that entry answer for a declared
foo.Only
$is stripped.@is DuckDB's absolute-value operator, soSELECT @namebinds a column reference and no@name can ever be declared.IndexOf(string)gains the same tolerance, so a caller who adds$idcan retrieve it byid.Behaviour change
A declared parameter with no supplied value now throws before execution instead of letting the engine fail later with a message that named nothing. Code relying on the old behaviour was already producing a failed query. One assertion in
ParameterCollectionTests.BindMultipleValuesInvalidOrderTestchanged accordingly.Positional binding is untouched.
Benchmarks
ParameterBindingBenchmarkjoins the existing suite. It caught a quadratic allocation in the prefix pass during review: the stripped name was materialised withSubstringonce per candidate, per declared parameter.The numbers below are the prefixed path, which is the one that changed.
NamedExactwas the baseline and moved on neither axis.Allocation is the whole gain. Time is unchanged, because a prepare and execute dominates the operation.
Gen0drops from 2.4414 to 0.4883 collections per 1000 operations at 32 parameters, and the prefixed path now allocates exactly what the exact path does.The old code materialised the stripped name with
Substringonce per candidate, per declared parameter, so the cost grew with the square of the collection size. The new code compares a span and allocates nothing.Verification
30 tests in
NamedParameterTests.cs: prefixed and unprefixed names, quoted names that start with$or consist of digits, repeated names, names differing only in case, multi-statement commands, and every case that must throw or must be ignored.NamedParameterTests