Skip to content

Conversation

@edoyango
Copy link
Collaborator

@edoyango edoyango commented Jan 8, 2026

The tests covers all the operation classes in operations/{dask,numpy,xarray}/values.py. Some important changes:

  • refactored xarray fillna to behave like numpy fillna
    • before, xarray fillna only filled nans, whereas numpy fillna also filled in positive and negative infinities.
  • completed the implementation of dask fillna
    • This was raising a NotImplementedError, and just needed to add a missing arg in da.nan_to_num to make it work.
  • use the Replace class instead of replace_value class from pyearthtool.data.transforms.mask as replace_value doesn't exist anymore.
  • Renamed ForceNormalised operation to Clip since that matches the functions in dask/numpy/xarray
    • also refactored the classes to use the corresponding clip function
    • This is probably the most opinionated change here, happy to change back if it's a problem.
  • fixed a problem in xarray.Derive where Derive.apply_func was also updating the input dataset.
  • fixed a warning in pyearthtools.data.transforms.derive by replacing the deprecated Dataset.drop with Dataset.drop_vars

Clip is more appropriate as there is an equivalent dask/numpy/xarray
function. Additionally, no normalisation is occuring.
This aligns xarray FillNan with numpy FillNan
Clip is more appropriate as there is an equivalent dask/numpy/xarray
operation.
when creating a new variable with Derive, the new variable
was being added to the input dataset. This commit fixes
that by making a shallow copy of the input and adding the
variables to that shallow copy before returning it.
Clip is more appropriate as there is an equivalent dask/numpy/xarray
operation.
@coveralls
Copy link

coveralls commented Jan 8, 2026

Pull Request Test Coverage Report for Build 20835553598

Details

  • 151 of 152 (99.34%) changed or added relevant lines in 7 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.9%) to 65.91%

Changes Missing Coverage Covered Lines Changed/Added Lines %
packages/data/src/pyearthtools/data/transforms/derive.py 7 8 87.5%
Totals Coverage Status
Change from base Build 20305027296: 0.9%
Covered Lines: 10711
Relevant Lines: 15893

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants