[WIP][SPARK-59685][SQL] Keep clause-free SQL/JSON constructor canonical SQL bound to the built-in - #58938
Open
ganeshashree wants to merge 1 commit into
Open
[WIP][SPARK-59685][SQL] Keep clause-free SQL/JSON constructor canonical SQL bound to the built-in#58938ganeshashree wants to merge 1 commit into
ganeshashree wants to merge 1 commit into
Conversation
ganeshashree
force-pushed
the
SPARK-59685
branch
6 times, most recently
from
September 22, 2026 07:35
9aa2d4f to
6183c0c
Compare
…L bound to the built-in A clause-free JSON_VALUE / JSON_QUERY / JSON_EXISTS / JSON_ARRAY call routes through function resolution (SPARK-59144), but these built-ins' canonical `sql` drops default clauses, so a call that is the built-in only because of a default clause renders clause-free SQL that, reparsed under a shadowing PATH, binds a same-named routine instead of the built-in. Render the default clause (RETURNING STRING, or FALSE ON ERROR for JSON_EXISTS) so canonical SQL reparses back to the built-in; `usePrettyExpression` renders the clean clause-free form (dropping the round-trip-only clause and rendering children in place, so nested splice decisions are preserved) and auto-generated column names are unaffected. Generated-by: Claude Code (Opus 4.8) Co-authored-by: Isaac <no-reply@databricks.com>
ganeshashree
force-pushed
the
SPARK-59685
branch
from
September 22, 2026 08:49
6183c0c to
054b5d5
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Since SPARK-59144, a clause-free SQL/JSON function call routes through function resolution, so a same-named routine on the SQL PATH can shadow the built-in. However these built-ins' canonical
sqlomits default clauses, so a call that is a built-in only because of a default clause renders as clause-free SQL; reparsing that under a shadowing PATH binds the routine instead of the built-in.This PR renders the default clause (
RETURNING STRING, orFALSE ON ERRORforJSON_EXISTS) in canonicalsqlso the rendering reparses back to the built-in. A sharedRoutedSqlJsonExpressiontrait letsusePrettyExpressionkeep rendering the clean clause-free form for auto-generated column names (which are never reparsed), so display names stay unaffected.Why are the changes needed?
Without this, a canonical-SQL round-trip (e.g. persisting and reloading a view) can silently rebind a built-in SQL/JSON function to a user routine that shadows it on the SQL PATH, changing results.
Does this PR introduce any user-facing change?
Yes, within unreleased master only. Canonical SQL of a clause-free SQL/JSON constructor now includes its default clause. Auto-generated column names are unchanged -- both top-level and nested (children are rendered in place, so a nested constructor's name gains no synthetic
FORMAT JSON).How was this patch tested?
New unit tests in
JsonValueSuite/JsonQuerySuite/JsonExistsSuite/JsonArraySuite: each asserts a default-clause constructor's canonical SQL reparses to the built-in under a shadowing PATH (including the zero-argJSON_ARRAY()case), that the emitted SQL round-trips and evaluates correctly, and that top-level and nested auto-generated column names stay clean. No golden-file changes.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)