ECC-2102: Improved high-level interface for BUFR

[tomas-kral]

Description

This PR is an overhaul of the high-level interface for working with BUFR files.

Some of the highlights of this new implementation are:

• Automatic packing and unpacking (though users can still do it manually if they so desire).
• A hierarchical view of the data section blocks which allows to easily navigate complex BUFR messages with multiple (and possibly nested) replications.
• Returning numpy masked arrays by default.
• Values of multi-rank keys in compressed multi-subset messages are represented as 2-dimensional arrays.
• A number of convenience methods: as_dict() for inspecting the contents of BUFR messages, get_datetime() which combines values date/time-related keys into np.datetime64 objects, copy(subsets=...) to extract subsets from messages based on different criteria, etc.

Also, you may notice that there are several workarounds (especially in the Coder class) for some of the known (and documented) shortcomings of the ecCodes library when it comes to BUFR en/decoding. These workarounds were necessary in order to provide more consistent behaviour and get rid of some of the wards in the low-level API.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 1.21664% with 2517 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.55%. Comparing base (10c48c8) to head (662aac2).

Files with missing lines	Patch %	Lines
eccodes/highlevel/_bufr/coder.py	0.00%	445 Missing ⚠️
eccodes/highlevel/_bufr/data.py	0.00%	438 Missing ⚠️
eccodes/highlevel/_bufr/tree.py	0.00%	413 Missing ⚠️
eccodes/highlevel/_bufr/helpers.py	0.00%	352 Missing ⚠️
eccodes/highlevel/_bufr/message.py	0.00%	268 Missing ⚠️
eccodes/highlevel/_bufr/header.py	0.00%	192 Missing ⚠️
eccodes/highlevel/_bufr/tables.py	0.00%	181 Missing ⚠️
eccodes/highlevel/_bufr/common.py	0.00%	168 Missing ⚠️
eccodes/highlevel/_bufr/view.py	0.00%	50 Missing ⚠️
eccodes/highlevel/_bufr/__init__.py	0.00%	3 Missing ⚠️
... and 4 more

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #134      +/-   ##
===========================================
- Coverage    33.93%   29.55%   -4.39%     
===========================================
  Files           16       34      +18     
  Lines         2705     6101    +3396     
  Branches       201      734     +533     
===========================================
+ Hits           918     1803     +885     
- Misses        1768     4276    +2508     
- Partials        19       22       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shahramn]

@tomas-kral
I know why the build ci/downstream-ci-hpc is failing.
It is because you are calling
base_path = Path(eccodes.codes_definition_path(), 'bufr', 'tables', '0')

But if the environment variable ECCODES_DEFINITION_PATH is defined then this would fail. The function
eccodes.codes_definition_path()
returns the path which may contain several directories. It acts similar to the Unix PATH env. var i.e.,
ECCODES_DEFINITION_PATH=dir1:dir2:dir3

So if you want to the get the full file path of a given BUFR element.def file, you should call
char* grib_context_full_defs_path(grib_context* c, const char* basename)

Where the first argument can be NULL and the basename argument would be:
bufr/tables/0/wmo/24/element.table

[tomas-kral]

Ah, I see, haven't thought of that. Okay, no problem, let me try calling the other function then. Thanks for the tip by the way!