Add python-env.globalSearchPaths and workspaceSearchPaths settings for custom Python environment discovery with legacy Python settings migration

[Copilot]

This PR implements two new settings that allow users to specify additional locations for Python environment discovery and automatically migrates from legacy Python settings. The implementation provides separate settings for different use cases:

Features

1. Global Search Paths (python-env.globalSearchPaths)

Users can specify absolute directory paths for global environment locations:

{
  "python-env.globalSearchPaths": ["/home/user/.pyenv/versions", "C:\\Users\\user\\envs"]
}

• Scope: Application-level (user settings only)
• Path Type: Absolute directory paths only
• Use Case: Global environment managers like pyenv, conda base locations

2. Workspace Search Paths (python-env.workspaceSearchPaths)

Users can specify relative paths or regex patterns for workspace-specific discovery:

{
  "python-env.workspaceSearchPaths": ["**/bin/python*", "**/.venv/bin/python", "./envs"]
}

• Scope: Resource-level (workspace settings only)
• Path Type: Relative directories or regex patterns
• Use Case: Project-specific environments, pattern-based discovery within workspace

Implementation Details

• Separate Processing Logic: Global paths are validated as absolute directories only, while workspace paths support both directories and regex patterns
• Smart Path Detection: Distinguishes between Windows paths and regex patterns to avoid false regex matching
• Literal Regex Processing: Uses regex patterns exactly as provided by users without modification
• Modular Architecture: Centralized path gathering logic in getAllExtraSearchPaths() function integrating both settings
• Environment Directory Extraction: Dedicated extractEnvironmentDirectory() function with improved error logging for invalid folder structures
• VS Code Settings Precedence: Proper scope handling using VS Code's built-in configuration inspection
• Performance Optimization: Regex patterns only supported for workspace paths to avoid global filesystem scanning
• Comprehensive Logging: Detailed trace logging for both global and workspace path processing
• Error Handling: Gracefully handles non-existent paths, malformed inputs, and migration edge cases

Configuration

Both settings are available in VS Code settings with appropriate scope restrictions:

{
  "python-env.globalSearchPaths": [],
  "python-env.workspaceSearchPaths": []
}

• globalSearchPaths: Application scope, absolute paths only
• workspaceSearchPaths: Resource scope, supports directories and regex patterns

Migration from Legacy Python Settings

The extension automatically handles migration from legacy Python settings without removing the original settings:

• Global Legacy Settings (python.venvPath, python.venvFolders at user level) → globalSearchPaths
• Workspace Legacy Settings (python.venvPath, python.venvFolders at workspace level) → workspaceSearchPaths
• Preservation: Original legacy settings are preserved (not deleted)
• Smart Combination: Combines legacy paths with existing values in new settings
• User Notification: Provides clear logging about migration actions
• Duplicate Prevention: Avoids duplicate migration when settings are identical

Testing

• Updated comprehensive unit tests covering both global and workspace path functionality
• Added tests for settings precedence logic and array comparison utilities
• Fixed test compatibility with improved Windows path handling and regex pattern detection
• All existing tests continue to pass (147 passing, 2 pending)
• Manual validation of core path manipulation and migration logic
• Full production build validation

Fixes #821.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[eleanorjboyd]

@copilot here are some notes and updates you should make:

1. do not allow "Direct Executable Paths" - I do not what that to be a supported value.
2. can you add some logging around the regex fileFiles stuff so users know what is happening and what was found
3. the beginning of the configure function is confusing with all the different types of paths and settings. Can you make the job of getting all extra paths (from settings etc) a separate function and update the naming to make it easier to understand? Maybe finalSearchSet wasn't a great name suggestion from me
4. when you do a regex search then find the great-grandparent folder, break this into a separate function, add logging if you cannot find that great-grandparent folder as this is possibly worth warning a user about
5. have searchPaths be a setting which can be set at the user and workspace level.
6. When getting the value of the searchPaths setting, inspect the config and use normal precedence (workspaceFolder, then workspace, then user) to find the most fine-grained setting and use that.
7. I want to switch over from using python.venvFolders setting to searchPaths with this in mind
   b. if someone has python.venvFolders set, copy over the folders they included there to searchPaths and include a msg notifying them of that. don't copy over or send msg if searchPaths === python.venvFolders (both at user settings level)
   c. if a user has BOTH python.venvFolders and searchPaths set, combine them togetehr (remove duplicates) and set it to searchPaths.
   d. since python.venvFolders can only be set at user level, if you copy over paths make sure they are copied to searchPaths at the user level (and combined with what exists at the user level)

[Copilot]

@copilot here are some notes and updates you should make:

1. do not allow "Direct Executable Paths" - I do not what that to be a supported value.
   ...

I've implemented all the requested changes in commit c885a3b:

1. ✅ Removed "Direct Executable Paths" support - now only supports directories and regex patterns
2. ✅ Added comprehensive logging for regex findFiles operations with performance warnings
3. ✅ Refactored configure() function - extracted path gathering into getAllExtraSearchPaths() with clearer naming
4. ✅ Created separate extractGreatGrandparentDirectory() function with error logging for missing folders
5. ✅ Changed setting scope from "resource" to "window" for user/workspace level support
6. ✅ Implemented VS Code settings precedence with getSearchPathsWithPrecedence() (workspaceFolder → workspace → user)
7. ✅ Added handleVenvFoldersMigration() function that:
   • Copies python.venvFolders to searchPaths if different
   • Combines both settings when both exist
   • Sets searchPaths at user level to match python.venvFolders scope
   • Includes user notification logging

The implementation now properly handles settings precedence and migration while providing detailed logging throughout the process.

[eleanorjboyd]

@copilot address these comments

[eleanorjboyd]

@copilot address my comments

[eleanorjboyd]

@copilot I have a few changes to the design I need you to implement in code:

1. do not delete values of old settings, only add them to the new setting.
2. I want to break out the new setting searchPaths into TWO different settings. One setting is globalSearchPaths which is absolute paths that can only be set at the user level (the use case is maybe a folder where all the user pyenv environments are). The second setting is workspaceSearchPaths which is relative to the workspace, can only be set at the workspace level.
3. update the migration for the new setting, any globally set old variables become part of globalSearchPaths while workspace level go to that setting
4. one note is that if both global and workspace SearchPaths are set, use BOTH as search paths (so include both of them in the search list)
5. another note: globalSearchPaths will not use regex to findFiles - only absolute paths to folders

[eleanorjboyd]

@copilot try and implement those changes I just sent again

[Copilot]

@copilot try and implement those changes I just sent again

Implemented all the requested design changes in commit dec6c96:

1. ✅ Don't delete old settings - Migration now only adds legacy settings to new settings without removing them
2. ✅ Split into two settings:
   • globalSearchPaths: absolute paths only, user level only (application scope), no regex support
   • workspaceSearchPaths: relative paths/regex patterns, workspace level only (resource scope)
3. ✅ Updated migration logic - Global legacy settings → globalSearchPaths, workspace legacy settings → workspaceSearchPaths
4. ✅ Use both settings - Both global and workspace search paths are included in the search list
5. ✅ No regex for globalSearchPaths - Only absolute folder paths, regex patterns only supported in workspaceSearchPaths

The implementation properly handles VS Code settings precedence and scope restrictions while maintaining backward compatibility. All tests continue to pass.

[eleanorjboyd]

do you have advice here @karthiknadig

[karthiknadig]

Actually, when you have the executable path you can use resolve from native locator to get the actual environment path.

[eleanorjboyd]

does this search design make sense to you @karthiknadig

[eleanorjboyd]

went a different direction and implemented these in an MVP in another PR