Implement support for SQL Server VectorSearch() by roji · Pull Request #37536 · dotnet/efcore

roji · 2026-01-19T19:19:22Z

Closes #36384
Closes #37535

...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs

src/EFCore.Relational/Query/RelationalSqlTranslatingExpressionVisitor.cs

roji · 2026-01-19T23:18:59Z

/azp run

azure-pipelines · 2026-01-19T23:19:07Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

This pull request implements support for SQL Server's VECTOR_SEARCH() function, which enables approximate, index-driven vector similarity search. The PR also addresses the need to allow table-valued functions to accept non-scalar parameters.

Changes:

Implements VectorSearch() extension method on DbSet for SQL Server
Relaxes TableValuedFunctionExpression to accept Expression instead of SqlExpression parameters
Refactors JsonEachExpression and SqlServerOpenJsonExpression to use dedicated Json properties instead of indexing Arguments

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
test/EFCore.SqlServer.FunctionalTests/Query/Translations/VectorTranslationsSqlServerTest.cs	Adds functional tests for VectorSearch with placeholder Experimental attributes that need correction
src/Shared/EFDiagnostics.cs	Adds new experimental diagnostic constant EF9105 for SqlServerVectorSearch
src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.cs	Changes exception throwing to ProcessUnknownMethod call for better extensibility
src/EFCore.Sqlite.Core/Query/Internal/SqliteQueryableMethodTranslatingExpressionVisitor.cs	Updates to use new JsonEachExpression.Json property
src/EFCore.Sqlite.Core/Query/Internal/SqliteQuerySqlGenerator.cs	Updates to use new JsonEachExpression.Json property
src/EFCore.Sqlite.Core/Query/Internal/SqlExpressions/JsonEachExpression.cs	Refactors to use primary constructor and dedicated Json property
src/EFCore.SqlServer/Query/Internal/SqlServerTypeMappingPostprocessor.cs	Updates to use SqlServerOpenJsonExpression.Json property
src/EFCore.SqlServer/Query/Internal/SqlServerSqlTreePruner.cs	Updates to use SqlServerOpenJsonExpression.Json property
src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs	Implements VectorSearch translation logic with naming issues in variable names
src/EFCore.SqlServer/Query/Internal/SqlServerQuerySqlGenerator.cs	Adds special SQL generation logic for VECTOR_SEARCH() function
src/EFCore.SqlServer/Query/Internal/SqlServerJsonPostprocessor.cs	Updates to use SqlServerOpenJsonExpression.Json property
src/EFCore.SqlServer/Query/Internal/SqlExpressions/SqlServerOpenJsonExpression.cs	Refactors to use dedicated Json property instead of Arguments[0]
src/EFCore.SqlServer/Properties/SqlServerStrings.resx	Adds error message for VectorSearchRequiresColumn
src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs	Auto-generated designer file for new error message
src/EFCore.SqlServer/Extensions/SqlServerQueryableExtensions.cs	New file defining VectorSearch extension method and VectorSearchResult struct with minor documentation typo
src/EFCore.SqlServer/Extensions/SqlServerDbFunctionsExtensions.cs	Minor documentation formatting improvement
src/EFCore.SqlServer/EFCore.SqlServer.csproj	Adds NoWarn for EF9105 experimental diagnostic
src/EFCore.Relational/Query/SqlExpressions/TableValuedFunctionExpression.cs	Changes Arguments type from IReadOnlyList to IReadOnlyList
src/EFCore.Relational/Query/RelationalSqlTranslatingExpressionVisitor.cs	Adds optimization to reduce member accesses on NewExpression
src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs	Removes assertion that required properties to be nullable

Files not reviewed (1)

src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

src/EFCore.SqlServer/Extensions/SqlServerQueryableExtensions.cs

src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs

test/EFCore.SqlServer.FunctionalTests/Query/Translations/VectorTranslationsSqlServerTest.cs

src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs

roji · 2026-01-22T07:16:31Z

/azp run

azure-pipelines · 2026-01-22T07:16:43Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

Copilot reviewed 19 out of 20 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.cs

Closes dotnet#36384 Closes dotnet#37535

src/EFCore.Relational/Query/SqlExpressions/TableValuedFunctionExpression.cs

src/EFCore.SqlServer/Extensions/SqlServerQueryableExtensions.cs

AndriySvyryd · 2026-02-03T22:19:11Z

src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs

+                    translatedMetric = _sqlExpressionFactory.ApplyTypeMapping(
+                        translatedMetric, _typeMappingSource.FindMapping("varchar(max)"));


I wonder if there's a better way of accomplishing this

You mean avoiding the N? Do you have a specific problem in mind with this, or just verbose/annoying?

Just too verbose and it feels like there's à way to refactor it to make it more readable

src/EFCore/Properties/CoreStrings.resx

Copilot

Pull request overview

Copilot reviewed 21 out of 23 changed files in this pull request and generated 7 comments.

Files not reviewed (2)

src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported
src/EFCore/Properties/CoreStrings.Designer.cs: Language not supported

Copilot · 2026-02-04T10:55:28Z

src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs

+                        ]);
+
+                    // We have the VECTOR_SEARCH() function call. Modify the SelectExpression and shaper to use it and project
+                    // the appropriate things.


The code uses EF1001 warning suppression for internal API usage (SetTables), which is intentional and acceptable in provider code. However, consider whether this internal API access could be avoided by using a more public mechanism, or ensure this dependency is documented as potentially fragile across EF Core versions.

Suggested change

// the appropriate things.

// the appropriate things.

// Note: SetTables is an internal EF Core API. Its usage here is intentional provider code and may need to be

// revisited if EF Core internals change in future versions.

Copilot · 2026-02-04T10:55:28Z

src/EFCore.Relational/Query/SqlExpressions/TableValuedFunctionExpression.cs

+        var arguments = NewArrayInit(
+            typeof(Expression),
+            Arguments.Select(a => ((IRelationalQuotableExpression)a).Quote()));


The cast to IRelationalQuotableExpression assumes all Arguments implement this interface. While this is currently safe because the only non-SqlExpression arguments passed are TableExpression instances (which inherit from TableExpressionBase and implement IRelationalQuotableExpression), this could fail at runtime if future code passes other Expression types. Consider adding a defensive check or documentation clarifying this assumption.

Suggested change

var arguments = NewArrayInit(

typeof(Expression),

Arguments.Select(a => ((IRelationalQuotableExpression)a).Quote()));

Microsoft.EntityFrameworkCore.Utilities.Check.DebugAssert(

Arguments.All(a => a is IRelationalQuotableExpression),

"All TableValuedFunctionExpression arguments must implement IRelationalQuotableExpression.");

var arguments = NewArrayInit(

typeof(Expression),

Arguments.Cast<IRelationalQuotableExpression>().Select(a => a.Quote()));

Copilot · 2026-02-04T10:55:29Z

...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs

@@ -2918,9 +2918,6 @@ private Expression CreateGetValueExpression(
            Type type,
            IPropertyBase? property = null)
        {


The removal of this assertion weakens safety checks for reading values from the database. The original assertion validated that when reading nullable values without a specified property, the type must be nullable. Without this check, there's a risk of attempting to read null values into non-nullable types, which could cause runtime errors. Consider either restoring this assertion or adding a comment explaining why this safety check is no longer necessary.

Suggested change

{

{

Microsoft.EntityFrameworkCore.Utilities.Check.DebugAssert(

!nullable

|| property is not null

|| !type.IsValueType

|| Nullable.GetUnderlyingType(type) is not null,

"When reading a nullable value without property metadata, the CLR type must be nullable.");

Copilot · 2026-02-04T10:55:29Z

src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs

+                    select.ReplaceProjection(new Dictionary<ProjectionMember, Expression>
+                    {
+                        [valueProjectionMember] = entityProjection,
+                        [distanceProjectionMember] = new ColumnExpression("Distance", vectorSearchFunction.Alias, typeof(double), _typeMappingSource.FindMapping(typeof(double)), nullable: false)


Consider verifying that the distance projection is correctly typed and nullable. While the code specifies nullable: false, vector search distance values could potentially be null in some edge cases (e.g., if the underlying index is unavailable or corrupted). Review SQL Server's VECTOR_SEARCH documentation to confirm the Distance column is always non-null.

Copilot · 2026-02-04T10:55:29Z

src/EFCore.SqlServer/Properties/SqlServerStrings.resx

    <value>Vector property '{propertyName}' is on '{structuralType}' which is mapped to JSON. Vector properties are not supported within JSON documents.</value>
  </data>
+  <data name="VectorSearchRequiresColumn" xml:space="preserve">
+    <value>VectorSearch() requires a valid vector column.</value>


The error message should be more specific about what constitutes a "valid vector column". Consider enhancing it to explain that VectorSearch requires a direct property reference to a vector column, not a computed expression or nested property access. This would provide better guidance to developers encountering this error.

Suggested change

<value>VectorSearch() requires a valid vector column.</value>

<value>VectorSearch() requires a direct reference to a vector property column, not a computed expression or nested property access.</value>

Copilot · 2026-02-04T10:55:29Z

src/EFCore.SqlServer/Extensions/SqlServerQueryableExtensions.cs

+        int topN)
+        where T : class
+        where TVector : unmanaged
+    {


There is no input validation for the topN parameter. According to the XML documentation (line 28), topN must be a positive integer, but there's no check to enforce this constraint. Consider adding validation to reject non-positive values before they're passed to SQL Server, which would provide clearer error messages to developers.

Suggested change

{

{

if (topN <= 0)

{

throw new ArgumentOutOfRangeException(nameof(topN));

}

Copilot · 2026-02-04T10:55:29Z

src/EFCore.SqlServer/Extensions/SqlServerQueryableExtensions.cs

+    public static IQueryable<VectorSearchResult<T>> VectorSearch<T, TVector>(
+        this DbSet<T> source,
+        Expression<Func<T, TVector>> vectorPropertySelector,
+        TVector similarTo,
+        [NotParameterized] string metric,
+        int topN)
+        where T : class
+        where TVector : unmanaged
+    {


There is no validation for the metric parameter. While NotParameterized prevents parameterization, the method doesn't verify that the metric value is one of the valid options (e.g., 'cosine', 'euclidean', 'dot'). Invalid metric values will only fail at SQL Server execution time with potentially less clear error messages. Consider validating against known metric values or at least checking for null/empty strings.

See dotnet/efcore#37536 See dotnet/efcore#37538 See dotnet/efcore#37578

roji force-pushed the SqlServerVectorSearch branch 3 times, most recently from ab8971f to 6364719 Compare January 19, 2026 19:28

roji commented Jan 19, 2026

View reviewed changes

...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs Show resolved Hide resolved

src/EFCore.Relational/Query/RelationalSqlTranslatingExpressionVisitor.cs Outdated Show resolved Hide resolved

roji marked this pull request as ready for review January 19, 2026 23:18

roji requested a review from a team as a code owner January 19, 2026 23:18

Copilot AI review requested due to automatic review settings January 19, 2026 23:18

Copilot started reviewing on behalf of roji January 19, 2026 23:19 View session

Copilot AI reviewed Jan 19, 2026

View reviewed changes

roji force-pushed the SqlServerVectorSearch branch from 6364719 to 233af64 Compare January 20, 2026 06:12

roji mentioned this pull request Jan 20, 2026

SQL Server: Support for full-text search TVFs (CONTAINSTABLE/FREETEXTTABLE) #37489

Closed

6 tasks

roji mentioned this pull request Jan 28, 2026

Implement SQL Server full-text search TVFs #37578

Merged

Copilot AI review requested due to automatic review settings January 28, 2026 18:06

roji force-pushed the SqlServerVectorSearch branch from 233af64 to 0bdd842 Compare January 28, 2026 18:06

Copilot started reviewing on behalf of roji January 28, 2026 18:07 View session

Copilot AI reviewed Jan 28, 2026

View reviewed changes

src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.cs Show resolved Hide resolved

Implement support for SQL Server VectorSearch()

be80ffc

Closes dotnet#36384 Closes dotnet#37535

roji force-pushed the SqlServerVectorSearch branch from 0bdd842 to be80ffc Compare January 28, 2026 18:20

AndriySvyryd approved these changes Feb 3, 2026

View reviewed changes

Address review comments

f12d852

Copilot AI review requested due to automatic review settings February 4, 2026 10:45

Copilot started reviewing on behalf of roji February 4, 2026 10:45 View session

roji enabled auto-merge (squash) February 4, 2026 10:46

Copilot AI reviewed Feb 4, 2026

View reviewed changes

roji merged commit d19634d into dotnet:main Feb 4, 2026
15 of 16 checks passed

roji deleted the SqlServerVectorSearch branch February 4, 2026 11:59

dotnet-maestro bot mentioned this pull request Feb 5, 2026

[main] Source code updates from dotnet/efcore dotnet/dotnet#4676

Merged

roji added a commit to roji/EntityFramework.Docs that referenced this pull request Feb 5, 2026

Document new SQL Server vector and full-text search features

fb9ba6b

See dotnet/efcore#37536 See dotnet/efcore#37538 See dotnet/efcore#37578

roji mentioned this pull request Feb 5, 2026

Document new SQL Server vector and full-text search features dotnet/EntityFramework.Docs#5258

Merged

roji added a commit to roji/EntityFramework.Docs that referenced this pull request Feb 5, 2026

Document new SQL Server vector and full-text search features

bf61efe

See dotnet/efcore#37536 See dotnet/efcore#37538 See dotnet/efcore#37578

roji added a commit to roji/EntityFramework.Docs that referenced this pull request Feb 5, 2026

Document new SQL Server vector and full-text search features

41f8813

See dotnet/efcore#37536 See dotnet/efcore#37538 See dotnet/efcore#37578

roji added a commit to dotnet/EntityFramework.Docs that referenced this pull request Feb 7, 2026

Document new SQL Server vector and full-text search features (#5258)

453bf75

See dotnet/efcore#37536 See dotnet/efcore#37538 See dotnet/efcore#37578

		translatedMetric = _sqlExpressionFactory.ApplyTypeMapping(
		translatedMetric, _typeMappingSource.FindMapping("varchar(max)"));

-        {
+        {
+            Microsoft.EntityFrameworkCore.Utilities.Check.DebugAssert(
+                !nullable
+                || property is not null
+                || !type.IsValueType
+                || Nullable.GetUnderlyingType(type) is not null,
+                "When reading a nullable value without property metadata, the CLR type must be nullable.");

	<value>VectorSearch() requires a valid vector column.</value>
	<value>VectorSearch() requires a direct reference to a vector property column, not a computed expression or nested property access.</value>

-    {
+    {
+        if (topN <= 0)
+        {
+            throw new ArgumentOutOfRangeException(nameof(topN));
+        }

Conversation

roji commented Jan 19, 2026

Uh oh!

Uh oh!

Uh oh!

roji commented Jan 19, 2026

Uh oh!

azure-pipelines bot commented Jan 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

roji commented Jan 22, 2026

Uh oh!

azure-pipelines bot commented Jan 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AndriySvyryd Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

roji Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

AndriySvyryd Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments