Skip to content

Comments

Implement support for SQL Server vector indexes#37538

Merged
roji merged 6 commits intodotnet:mainfrom
roji:SqlServerVectorIndex
Feb 5, 2026
Merged

Implement support for SQL Server vector indexes#37538
roji merged 6 commits intodotnet:mainfrom
roji:SqlServerVectorIndex

Conversation

@roji
Copy link
Member

@roji roji commented Jan 19, 2026

Closes #37281

@roji roji force-pushed the SqlServerVectorIndex branch from 6e92827 to ff9db2e Compare January 19, 2026 23:27
@roji
Copy link
Member Author

roji commented Jan 20, 2026

/azp run

@roji roji marked this pull request as ready for review January 20, 2026 06:13
@roji roji requested a review from a team as a code owner January 20, 2026 06:13
Copilot AI review requested due to automatic review settings January 20, 2026 06:13
@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements support for SQL Server vector indexes, a preview feature in SQL Server 2025. The implementation includes model building APIs, migration generation, scaffolding/reverse engineering support, and validation.

Changes:

  • Added fluent API for creating vector indexes via HasVectorIndex() extension methods with configurable metrics (cosine, euclidean, dot) and index types (DiskANN)
  • Implemented migration SQL generation for CREATE VECTOR INDEX and DROP INDEX statements with proper formatting
  • Added scaffolding support to reverse-engineer vector indexes from existing databases via sys.vector_indexes system view
  • Added model validation to ensure vector indexes have exactly one column
  • Updated test utilities to properly manage connection state during test execution

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/EFCore.SqlServer/Extensions/SqlServerEntityTypeBuilderExtensions.cs Added HasVectorIndex() methods for typed and untyped entity builders
src/EFCore.SqlServer/Extensions/SqlServerIndexExtensions.cs Added extension methods for getting/setting vector index metric and type annotations
src/EFCore.SqlServer/Metadata/Builders/VectorIndexBuilder.cs New builder class for configuring vector index properties (non-generic)
src/EFCore.SqlServer/Metadata/Builders/VectorIndexBuilder1.cs` New builder class for configuring vector index properties (generic)
src/EFCore.SqlServer/Metadata/Internal/SqlServerAnnotationNames.cs Added annotation constants for VectorIndexMetric and VectorIndexType
src/EFCore.SqlServer/Metadata/Internal/SqlServerAnnotationProvider.cs Added logic to provide vector index annotations for table indexes
src/EFCore.SqlServer/Infrastructure/Internal/SqlServerModelValidator.cs Added validation to ensure vector indexes have single column
src/EFCore.SqlServer/Migrations/SqlServerMigrationsSqlGenerator.cs Implemented SQL generation for CREATE VECTOR INDEX with metric and type options
src/EFCore.SqlServer/Scaffolding/Internal/SqlServerDatabaseModelFactory.cs Added GetVectorIndexes method to scaffold vector indexes from database, excluded vector indexes from regular index query
src/EFCore.SqlServer/Properties/SqlServerStrings.resx Added error message for vector index single column requirement
src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs Generated code for new error message
test/EFCore.SqlServer.FunctionalTests/Migrations/MigrationsSqlServerTest.cs Added tests for creating and dropping vector indexes with different configurations
test/EFCore.SqlServer.FunctionalTests/TestUtilities/SqlServerTestStore.cs Fixed connection state management to restore original state after command execution
Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji roji force-pushed the SqlServerVectorIndex branch from 3d7c966 to 936b8d4 Compare January 20, 2026 09:03
Copilot AI review requested due to automatic review settings January 20, 2026 09:03
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 2 comments.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji roji force-pushed the SqlServerVectorIndex branch from 936b8d4 to 5232b11 Compare January 20, 2026 09:23
/// </param>
/// <param name="name">The name to assign to the index.</param>
/// <returns>A builder to further configure the vector index.</returns>
public static VectorIndexBuilder<TEntity> HasVectorIndex<TEntity>(
Copy link
Member Author

@roji roji Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndriySvyryd take a look at this builder/metadata proposal, and feel free to push any changes directly to the PR. Some notes:

  • Defining a distance metric (cosine, euclidean...) is mandatory, so I'm using that as the metadata indication that an index is a vector index.
  • Vector indexes aren't just some extra facets on regular indexes - they're a completely different type of index (the options when creating a vector index are very different from those of regular indexes). As a result, I'm proposing that we have a separate VectorIndexBuilder to only expose the relevant APIs, even if metadata-wise it's just an index.
    • Interestingly the SQL Server sys.vector_indexes table that's used to scaffold inherits sys.indexes, as if vector indexes are an extension to regular indexes; but their creation syntax is very different. In any case, we can change this in the future, making VectorIndexBuilder extend IndexBuilder if we really want to.

@roji roji force-pushed the SqlServerVectorIndex branch from 5232b11 to 2d7d979 Compare January 20, 2026 09:58
Copilot AI review requested due to automatic review settings January 20, 2026 09:58
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 5 comments.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji roji force-pushed the SqlServerVectorIndex branch from 2d7d979 to 38c0ffa Compare January 20, 2026 10:22
Copilot AI review requested due to automatic review settings January 20, 2026 10:39
@roji roji force-pushed the SqlServerVectorIndex branch from 38c0ffa to a3d18e0 Compare January 20, 2026 10:39
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji roji force-pushed the SqlServerVectorIndex branch from a3d18e0 to 25c32c1 Compare January 20, 2026 12:32
Copilot AI review requested due to automatic review settings January 20, 2026 15:05
@roji roji force-pushed the SqlServerVectorIndex branch from 25c32c1 to 42d0420 Compare January 20, 2026 15:05
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji
Copy link
Member Author

roji commented Jan 22, 2026

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI review requested due to automatic review settings February 4, 2026 19:42
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported

@roji roji force-pushed the SqlServerVectorIndex branch from 33d481b to ef783e4 Compare February 4, 2026 20:19
@roji roji requested a review from AndriySvyryd February 4, 2026 20:19
@roji
Copy link
Member Author

roji commented Feb 4, 2026

@AndriySvyryd ready for re-review. Check especially the last commit with the added tests (fully AI-generated).

@roji roji enabled auto-merge (squash) February 4, 2026 20:19
Comment on lines +282 to +296
if (index.GetVectorMetric() is null or "")
{
throw new InvalidOperationException(
SqlServerStrings.VectorIndexRequiresMetric(
index.DisplayName(),
index.DeclaringEntityType.DisplayName()));
}

if (index.GetVectorIndexType() is "")
{
throw new InvalidOperationException(
SqlServerStrings.VectorIndexRequiresType(
index.DisplayName(),
index.DeclaringEntityType.DisplayName()));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for these

{
b.Property(e => e.Vector).HasColumnType("vector(3)");

if (b is IInfrastructure<EntityTypeBuilder<VectorIndexEntity>> genericBuilder)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't cast to IInfrastructure here, add extension methods that mimic the new fluent API

Comment on lines 142 to 143
index.AddAnnotation("SqlServer:VectorIndexMetric", "cosine");
index.AddAnnotation("SqlServer:VectorIndexType", "DiskANN");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not used at runtime, so they should be filtered out in SqlServerCSharpRuntimeAnnotationCodeGenerator

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am noting that if we do implement a unified vector search API, we'd likely need at least the similarity measure at runtime. But we'll cross that bridge when we get to it.

Copilot AI review requested due to automatic review settings February 4, 2026 23:19
@roji
Copy link
Member Author

roji commented Feb 4, 2026

@AndriySvyryd you comments should be addressed - mainly by AI.

I suggest that for these things, which are in your area and where you know exactly what you want, it's probably better for you to push changes directly into the PR rather than ask - just faster/easier with less roundtrips.

@roji roji requested a review from AndriySvyryd February 4, 2026 23:20
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 1 comment.

Files not reviewed (1)
  • src/EFCore.SqlServer/Properties/SqlServerStrings.Designer.cs: Language not supported


Assert.Same(table.Columns.Single(c => c.Name == "Vector"), indexColumn);
Assert.Equal("COSINE", index[SqlServerAnnotationNames.VectorIndexMetric]);
Assert.Equal("DiskANN", index[SqlServerAnnotationNames.VectorIndexType]);
Copy link

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test expects VectorIndexType to be "DiskANN" even though UseType() was never called. This assertion assumes that SQL Server defaults vector indexes to type "DiskANN" when the TYPE clause is omitted. While this may be correct based on SQL Server's behavior, it should be verified that SQL Server actually provides this default. If SQL Server does not default to "DiskANN", this assertion will fail.

Suggested change
Assert.Equal("DiskANN", index[SqlServerAnnotationNames.VectorIndexType]);

Copilot uses AI. Check for mistakes.
@AndriySvyryd
Copy link
Member

I suggest that for these things, which are in your area and where you know exactly what you want, it's probably better for you to push changes directly into the PR rather than ask - just faster/easier with less roundtrips.

That's not how this works, if you submit a PR you are signing up as a Copilot copy-paster 😆

Hopefully, CCA will be able to work on fork PRs soon.

@roji roji merged commit e72cb67 into dotnet:main Feb 5, 2026
15 of 16 checks passed
@roji roji deleted the SqlServerVectorIndex branch February 5, 2026 01:43
@roji
Copy link
Member Author

roji commented Feb 5, 2026

That's not how this works, if you submit a PR you are signing up as a Copilot copy-paster 😆

Yeah I'm aware :)

Jokes aside, it's less that I care about Copilot copy-pasting, more about whether the results coming out of Copilot will be exactly what you want...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQL Server: Support vector indexes

2 participants