Skip to content

Conversation

@mshafer-NI
Copy link
Collaborator

Adds a test for and fixes #168

@github-actions
Copy link
Contributor

Thank you for contributing! 👋

@mshafer-NI mshafer-NI changed the title Dev/fix formatting files with unicode Fix formatting files with unicode Nov 12, 2025
@mshafer-NI mshafer-NI requested a review from Copilot November 12, 2025 22:50
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes issue #168 by ensuring proper Unicode handling when reading and writing Python files during formatting and linting operations.

Key Changes:

  • Added explicit UTF-8 encoding to all file read/write operations across the codebase
  • Replaced read_text()/write_text() with shutil.copyfile() in the diff generation path for better encoding preservation
  • Added a test fixture that forces ASCII encoding by default to catch any missing encoding specifications
  • Added comprehensive test cases with Unicode characters (©, π, Gerät, デバイス, 장치, 设备, etc.) to verify the fix

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_cli/test_format.py Added explicit UTF-8 encoding to test file reading to properly handle Unicode content
ni_python_styleguide/_utils/code_analysis.py Added UTF-8 encoding to file reading in import region analysis
ni_python_styleguide/_fix.py Added DEFAULT_ENCODING to multiple read/write operations and replaced read_text/write_text with shutil.copyfile for better encoding preservation
tests/conftest.py Added autouse test fixture to force ASCII encoding by default, catching any missing encoding specifications
pyproject.toml Removed unnecessary blank line for code cleanup
tests/test_cli/format_test_cases__snapshots/file_with_unicode/* Added new test case files containing various Unicode characters to validate the fix
tests/test_cli/fix_test_cases__snapshots/unicode_example/* Added test case files for Unicode handling in the fix command
tests/test_cli/acknowledge_existing_errors_test_cases__snapshots/unicode_in_files/* Extended existing test case with additional Unicode data

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@irwand irwand merged commit bf81954 into ni:main Nov 13, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix can't handle source files encoded as UTF-8

2 participants