Make parameter length based on underlying column length except for complex ops#2627
Open
simonsabin wants to merge 18 commits intoAzure:mainfrom
Open
Make parameter length based on underlying column length except for complex ops#2627simonsabin wants to merge 18 commits intoAzure:mainfrom
simonsabin wants to merge 18 commits intoAzure:mainfrom
Conversation
simonsabin
requested review from
Aniruddh25,
RubenCerna2079,
aaronburtle,
neeraj-sharma2592,
ravishetye,
rohkhann,
sezal98 and
sourabh1007
as code owners
aaronburtle
reviewed
Mar 20, 2025
aaronburtle
reviewed
Mar 20, 2025
aaronburtle
reviewed
Mar 20, 2025
aaronburtle
reviewed
Mar 20, 2025
aaronburtle
reviewed
Mar 20, 2025
src/Service.Tests/SqlTests/GraphQLQueryTests/MsSqlGraphQLQueryTests.cs
Outdated
Show resolved
Hide resolved
aaronburtle
reviewed
Mar 20, 2025
Contributor
aaronburtle
left a comment
aaronburtle
left a comment
There was a problem hiding this comment.
This looks great, thank you for raising and then solving this issue. Just had a few nits, and a question.
Collaborator
Author
|
It’s to cover the edge case of matching with the LiKE that you get with contains and starts with, and also the escaping of values that occurs in the predicates. If you don’t you can end up with false positives and false negatives
Simon Sabin
________________________________
From: aaronburtle ***@***.***>
Sent: Thursday, March 20, 2025 9:19:26 PM
To: Azure/data-api-builder ***@***.***>
Cc: Simon Sabin ***@***.***>; Author ***@***.***>
Subject: Re: [Azure/data-api-builder] Make parameter length based on underlying column length except for complex ops (PR #2627)
@aaronburtle commented on this pull request.
________________________________
In src/Service.Tests/DatabaseSchema-MsSql.sql<#2627 (comment)>:
@@ -514,7 +514,7 @@ SET IDENTITY_INSERT books ON
INSERT INTO books(id, title, publisher_id)
VALUES (1, 'Awesome book', 1234),
(2, 'Also Awesome book', 1234),
-(3, 'Great wall of china explained', 2345),
+(3, 'Great wall of china explained]', 2345),
I am curious, why the extra "]"?
—
Reply to this email directly, view it on GitHub<#2627 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAJHM2Z7OZVBM5HTNT3F6SD2VMWF5AVCNFSM6AAAAABZOI6YHOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDOMBUGEZTOMRTG4>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Co-authored-by: aaronburtle <93220300+aaronburtle@users.noreply.github.com>
Co-authored-by: aaronburtle <93220300+aaronburtle@users.noreply.github.com>
Collaborator
Author
|
applied suggested changes |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
|
Pipeline failures are just whitespace format. |
Aniruddh25
reviewed
Mar 24, 2025
Aniruddh25
reviewed
Mar 24, 2025
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
aaronburtle
requested review from
Alekhya-Polavarapu,
JerryNixon,
anushakolan,
rusamant,
souvikghosh04,
stuartpa and
vadeveka
as code owners
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
There was a problem hiding this comment.
Pull request overview
Updates MSSQL parameter creation to stabilize query plan reuse by sizing string parameters based on underlying column length, with an escape hatch for “complex” string operations that expand values (LIKE patterns/escaping).
Changes:
- Capture MSSQL column length in SQL metadata (
ColumnSize) and flow it throughDbConnectionParam. - Extend parameterization to support a
lengthOverridepath (intended for LIKE-based operations). - Add/adjust MSSQL GraphQL tests and MSSQL test schema to exercise string filter scenarios.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Service.Tests/SqlTests/GraphQLQueryTests/MsSqlGraphQLQueryTests.cs | Adds new GraphQL filter tests for string params (simple and complex ops). |
| src/Service.Tests/DatabaseSchema-MsSql.sql | Changes books.title to varchar(30) and adjusts seed data to include a special character. |
| src/Core/Services/MetadataProviders/SqlMetadataProvider.cs | Populates column length for MSSQL columns via schema metadata. |
| src/Core/Resolvers/MsSqlQueryExecutor.cs | Sets SqlParameter.Size for certain string SqlDbTypes based on new length metadata. |
| src/Core/Resolvers/CosmosQueryStructure.cs | Updates MakeDbConnectionParam signature to include lengthOverride for parity with base. |
| src/Core/Resolvers/BaseQueryStructure.cs | Adds lengthOverride argument and flows parameter length from model metadata into DbConnectionParam. |
| src/Core/Models/GraphQLFilterParsers.cs | Plumbs lengthOverride through GraphQL filter parsing for LIKE-based operators. |
| src/Core/Models/DbConnectionParam.cs | Adds Length to represent intended parameter size. |
| src/Config/DatabasePrimitives/DatabaseObject.cs | Adds GetLengthForParam and stores ColumnDefinition.Length. |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
thanks for perservering with this |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
aaronburtle
force-pushed
the
paramlength
branch
from
dfad4c3 to
3189c7e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why make this change?
Closes #2626
What is this change?
Where a parameter is linked to an underlying column in the data model the length is stored
When the query is executed the length is used to ensure parameters have consistent lengths
For certain operations the value is padded with escape characters or %s, these need to be allowed for. In those cases the length is set to -1 (max) in order that the extra characters are allowed for AND consistent lengths are maintained
How was this tested?
Checking the resultant query is using the right length was done using trace. Not sure with the architecture whether it would be possible to mock at that level.
Sample Request(s)
See new tests TestFilterParamForStringFilterWorkWithComplexOp and TestFilterParamForStringFilterWorkWithNotContains