Fix pandas 3 compatibility

[Copilot]

Pandas 3 Compatibility - COMPLETE ✅

Summary

Successfully fixed all originally reported pandas 3.0 compatibility issues. All three failing tests now pass with both pandas 2.x and pandas 3.x.

Original Failing Tests (Now Fixed)

1. ✅ test_get_annotation_str - Fixed by using IsOneOf to test both possible annotation paths
2. ✅ test_annotation_is_pandas_series - Fixed by supporting both module paths
3. ✅ test_ensure_running_start_time_is_datetime - Fixed by applying dtype conversions

Changes Made

1. UDF annotation handling (openeo/udf/run_code.py):

   • Updated _annotation_is_pandas_series() to recognize both pandas 2.x ("pandas.core.series.Series") and pandas 3.x ("pandas.Series") module paths
   • Maintains cross-version compatibility for UDFs written in different pandas versions

2. Test updates (tests/udf/test_run_code.py):

   • Made tests version-agnostic using dirty_equals.IsOneOf to validate both pandas 2.x and 3.x module paths
   • Tests validate that annotation strings match either expected value
   • Comments embedded inline within IsOneOf call for clarity

3. DataFrame type enforcement (openeo/extra/job_management/_manager.py):

   • Enhanced normalize_df() to apply dtype conversions only when current dtype doesn't match required dtype
   • For string columns (dtype="str"), converts to "object" to maintain backward compatibility
   • Prevents errors when setting string values into columns with incompatible inferred types (e.g., float64)

4. Version constraint (setup.py):

   • Removed pandas<3.0.0 constraint to allow pandas 3.x

Testing Results

✅ All originally failing tests pass with pandas 3.0.0
✅ All originally failing tests pass with pandas 2.3.3 (backward compatibility verified)
✅ All UDF tests pass (122 passed, 6 skipped)
✅ All job manager tests pass (29 passed)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Pandas 3 compatibility</issue_title>
<issue_description>pandas 3 was released earlier this week and unit tests seem to be failing on that

=========================== short test summary info ============================
FAILED tests/extra/job_management/test_manager.py::TestMultiBackendJobManager::test_ensure_running_start_time_is_datetime[2024-09-01T09:00:00Z-2024-09-01T09:00:00Z-nan-2024-09-01T20:00:00Z-finished-21600] - TypeError: Invalid value '2024-09-01T10:00:00Z' for dtype 'float64'
FAILED tests/udf/test_run_code.py::test_get_annotation_str[Series-pandas.core.series.Series] - AssertionError: assert 'pandas.Series' == 'pandas.core.series.Series'
  
  - pandas.core.series.Series
  + pandas.Series
FAILED tests/udf/test_run_code.py::test_annotation_is_pandas_series - AssertionError: assert False is True
 +  where False = _annotation_is_pandas_series('pandas.core.series.Series')
=========== 3 failed, 2609 passed, 9 skipped, 165 warnings in 30.19s ===========

</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Pandas 3 compatibility #856

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.