Add IncludeGraph support for full object graph bulk insert

[Copilot]

Review feedback addressed:

• GraphBulkInsertOrchestrator.cs - Allow IncludeGraph for Oracle/MySQL providers when all keys are client-generated (no database-generated keys)
• GraphBulkInsertOrchestrator.cs - Include many-to-many join record inserts in TotalInsertedCount
• GraphMetadata.cs - Add entityType.FindPrimaryKey() == null check to filter out keyless entities
• GraphTestsBase.cs - Fix comment that incorrectly said "1000 blogs" when it should say "50 blogs"
• GraphBulkInsertOrchestrator.cs - Renamed hasAnyGeneratedKeys to hasAnyDatabaseGeneratedKeys for clarity

Original prompt

Implementation Plan: IncludeGraph Support for Full Object Graph Bulk Insert

1. Overview

Goal

Add the ability to bulk insert a full graph of objects (entities with their navigation properties/relationships) while maintaining the library's high-performance characteristics across all supported providers (PostgreSQL, SQL Server, SQLite, MySQL, Oracle).

This will fix issue #2

Current State

• The library currently explicitly does not support navigation properties (documented in docs/limitations.md and README roadmap)
• Current architecture handles flat entity insertion only via TableMetadata and ColumnMetadata
• Entity relationships like Student ↔ Course (many-to-many) already exist in test entities but aren't bulk-insertable

---

2. Changes to BulkInsertOptions

2.1 Add IncludeGraph Option

public class BulkInsertOptions
{
    // ... existing options ...

    /// <summary>
    /// When enabled, recursively inserts all reachable entities via navigation properties.
    /// This includes one-to-one, one-to-many, many-to-one, and many-to-many relationships.
    /// Default: false (only the root entities are inserted).
    /// </summary>
    public bool IncludeGraph { get; set; } = false;

    /// <summary>
    /// Maximum depth for graph traversal when IncludeGraph is enabled.
    /// Use 0 for unlimited depth. Default: 0.
    /// </summary>
    public int MaxGraphDepth { get; set; } = 0;

    /// <summary>
    /// Navigation properties to explicitly include when IncludeGraph is enabled.
    /// If empty and IncludeGraph is true, all navigation properties are included.
    /// </summary>
    public HashSet<string>? IncludeNavigations { get; set; }

    /// <summary>
    /// Navigation properties to explicitly exclude when IncludeGraph is enabled.
    /// </summary>
    public HashSet<string>? ExcludeNavigations { get; set; }
}

---

3. New Graph Processing Infrastructure

3.1 Create GraphMetadata Class

Location: src/PhenX.EntityFrameworkCore.BulkInsert/Metadata/GraphMetadata.cs

Purpose: Analyze EF Core's IEntityType to extract navigation property metadata, FK relationships, and dependency order.

internal sealed class GraphMetadata
{
    // Navigation property info (FK columns, principal/dependent entities)
    public IReadOnlyList<NavigationMetadata> Navigations { get; }
    
    // Topological sort order for insertion (respecting FK constraints)
    public IReadOnlyList<Type> InsertionOrder { get; }
    
    // Detected relationship types
    public bool HasManyToMany { get; }
    public bool HasOneToOne { get; }
    public bool HasOneToMany { get; }
}

3.2 Create NavigationMetadata Class

Location: src/PhenX.EntityFrameworkCore.BulkInsert/Metadata/NavigationMetadata.cs

internal sealed class NavigationMetadata
{
    public string PropertyName { get; }
    public INavigation Navigation { get; }
    public Type TargetType { get; }
    public bool IsCollection { get; }
    public bool IsManyToMany { get; }
    public string[] ForeignKeyColumns { get; }
    public string[] PrincipalKeyColumns { get; }
    
    // For many-to-many: join table info
    public string? JoinTableName { get; }
    public JoinTableMetadata? JoinTable { get; }
}

3.3 Create GraphEntityCollector Class

Location: src/PhenX.EntityFrameworkCore.BulkInsert/Graph/GraphEntityCollector.cs

Purpose: Walk the object graph, collect all entities by type, and detect cycles.

internal class GraphEntityCollector
{
    /// <summary>
    /// Traverses the entity graph and returns entities grouped by type in insertion order.
    /// </summary>
    public GraphCollectionResult Collect<T>(
        IEnumerable<T> rootEntities,
        DbContext context,
        BulkInsertOptions options) where T : class;
}

internal class GraphCollectionResult
{
    // Entities by type, in topological order (parents before children)
    public IReadOnlyDictionary<Type, IList<object>> EntitiesByType { get; }
    
    // Insertion order (respecting FK dependencies)
    public IReadOnlyList<Type> InsertionOrder { get; }
    
    // Many-to-many join records to insert after both sides are inserted
    public IReadOnlyList<JoinRecord> JoinRecords { get; }
}

3.4 Create GraphBulkInsertOrchestrator Class

Location: src/PhenX.EntityFrameworkCore.BulkInsert/Graph/GraphBulkInsertOrchestrator.cs

Purpose: Coordinate multi-table insertions with FK propagation.

internal class GraphBulkInsertOrchestrator
{
    /// <summary>
    /// Orchestrates the bulk insert of an entity graph.
    /// 1. Collect and sort entities
    /// 2. Insert in dependency order (parents first)
    /// 3. Propagate generated keys to dependent FKs
    /// 4. Insert join table records for many-to-many
    /// </summary>
    public async Task<GraphInsertResult<T>> InsertGr...

</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[Copilot]

Pull request overview

This PR adds comprehensive support for bulk inserting entity graphs with navigation properties, addressing issue #2. The implementation enables users to insert entire object graphs (entities with their related navigation properties) while maintaining the library's performance characteristics through topological sorting and efficient FK propagation.

Changes:

• Added IncludeGraph option with supporting configuration options (MaxGraphDepth, IncludeNavigations, ExcludeNavigations)
• Implemented graph processing infrastructure including metadata analysis, entity collection with cycle detection, and multi-table orchestration
• Added comprehensive test coverage across PostgreSQL, SQL Server, and SQLite providers with test entities for various relationship types
• Updated documentation to reflect new capabilities and mark the feature as complete in the roadmap

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
src/PhenX.EntityFrameworkCore.BulkInsert/Options/BulkInsertOptions.cs	Added four new options for graph insert configuration
src/PhenX.EntityFrameworkCore.BulkInsert/Metadata/NavigationMetadata.cs	Navigation property metadata extraction and FK relationship analysis
src/PhenX.EntityFrameworkCore.BulkInsert/Metadata/GraphMetadata.cs	Entity graph analysis and topological sorting for insertion order
src/PhenX.EntityFrameworkCore.BulkInsert/Graph/GraphEntityCollector.cs	Graph traversal with cycle detection and entity collection
src/PhenX.EntityFrameworkCore.BulkInsert/Graph/GraphBulkInsertOrchestrator.cs	Multi-table insertion orchestration with generated ID propagation
src/PhenX.EntityFrameworkCore.BulkInsert/Extensions/PublicExtensions.DbSet.cs	Integration point that routes to graph orchestrator when IncludeGraph is enabled
src/PhenX.EntityFrameworkCore.BulkInsert/Abstractions/IBulkInsertProvider.cs	Added SupportsOutputInsertedIds property for provider capability detection
src/PhenX.EntityFrameworkCore.BulkInsert/BulkInsertProviderUntyped.cs	Default implementation of SupportsOutputInsertedIds
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/Blog.cs	Test entity for one-to-many relationships
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/Post.cs	Test entity for one-to-many and many-to-many relationships
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/Tag.cs	Test entity for many-to-many relationships
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/BlogSettings.cs	Test entity for one-to-one relationships
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/Category.cs	Test entity for self-referencing hierarchical relationships
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContext/TestDbContext.cs	Configuration of relationship mappings for test entities
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Graph/GraphTestsBase.cs	Comprehensive test suite covering various graph scenarios
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Graph/GraphTestsPostgreSql.cs	PostgreSQL-specific test runner
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Graph/GraphTestsSqlServer.cs	SQL Server-specific test runner
tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Graph/GraphTestsSqlite.cs	SQLite-specific test runner
docs/graph-insert.md	Comprehensive documentation of the graph insert feature
docs/documentation.md	Option documentation for the new graph insert options
docs/limitations.md	Updated to indicate navigation properties are now supported
README.md	Usage examples and roadmap update

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[PhenX]

@copilot add tests to ensure that inserted entities are linked to there original objects (with generated primary keys or not)

[Copilot]

@copilot add tests to ensure that inserted entities are linked to there original objects (with generated primary keys or not)

Added 3 new tests in commit 8e4dfe6:

• InsertGraph_OriginalEntitiesLinked_WithGeneratedKeys - Verifies original entity references retain their generated IDs and FK relationships after insertion
• InsertGraph_OriginalEntitiesLinked_WithClientGeneratedKeys - Tests entities with ValueGeneratedNever() (GUID keys) maintain their pre-set IDs
• InsertGraph_MultipleRootEntities_OriginalEntitiesLinked - Verifies multiple root entities with children all maintain proper linking and relationships

[PhenX]

@copilot Update the benchmark project to test include graph option. Do not write a new benchmark, only a boolean property that I can change to true if I want to benchmark Include graph insertion. When set to false (the default) the benchmark should run exactly the same way as it does now

[Copilot]

@copilot Update the benchmark project to test include graph option. Do not write a new benchmark, only a boolean property that I can change to true if I want to benchmark Include graph insertion. When...

Added UseIncludeGraph property to LibComparator class in commit f811609. Set it to true to benchmark with IncludeGraph enabled. Defaults to false for unchanged benchmark behavior.

[Copilot]

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[PhenX]

@copilot implement the suggestions provided in the latest review

[Copilot]

@copilot implement the suggestions provided in the latest review

Implemented the review suggestions in commit d927f1f:

1. SetInverseNavigation - Now checks if inverse is a collection and skips to avoid invalid cast
2. Large scale test - Reduced to 50 blogs × 10 posts × 3 tags to prevent CI timeouts
3. Docs - Clarified that join table population requires explicit join entity types, and Oracle/MySQL limitation is specific to database-generated key propagation
4. Sync flag - Now properly passed through to provider calls instead of hardcoding false
5. Join entities - Added generic helper method and skip dictionary-based join entities with a warning

[Copilot]

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[PhenX]

@copilot implement suggestions from latest review with these choices :

• Update the code so that graph insert works even for oracle and mysql, only if all the keys are generated.
• Update totalInserted so that it includes many to many inserts.
• Check for entityType.IsKeyless in GraphMetadata

[Copilot]

@copilot implement suggestions from latest review with these choices :

• Update the code so that graph insert works even for oracle and mysql, only if all the keys are generated.
• Update totalI...

Implemented the requested changes in commit f21b3a8:

1. Oracle/MySQL support for client-generated keys - Validation now only throws when entities have database-generated keys AND provider doesn't support returning IDs. Graph inserts with client-generated keys (GUIDs) work on all providers.

2. TotalInsertedCount includes join records - InsertJoinRecords now returns the count of inserted join records, which is added to totalInserted.

3. Keyless entity filter - Added entityType.FindPrimaryKey() == null check in GraphMetadata to properly filter out keyless entities.

4. Fixed test comment - Updated "1000 blogs" to "50 blogs".