Update tests

[rigoudyg]

Update tests : coverage + add test for scripts
coverage is now trully computed for command-line test script

New test script work alone but not with others.
@sol1105 do you have an idea why there is an issue?

[sol1105]

@rigoudyg It does not seem to accept the temporary directories and configuration. For test_cli.py it seems to work however, though the approach is the same. I'll try to figure it out.

[sol1105]

@rigoudyg After merging my PR #135, you can modify your setup_method methods in the following way to make it work. It will however only work when running the tests from the repository root. Should the scripts folder be added in the same way as the cli scripts? At least some of them?

# add the import statement
+        import data_request_api.utilities.config as dreqcfg

# update setup_method
     
             ...
             yaml.dump(config, fh)
+        dc._dreq_res = self.temp_config_file.parent
+        dc.versions = {"tags": [], "branches": []}
+        dreqcfg.CONFIG_FILE = self.temp_config_file
+        dreqcfg.CONFIG = {}
+        #dreqcfg.load_config(reload=True) # alternatively
         dc.load(self.version)

For some reason dc.versions remains cached even between test scripts. I do not know why, I guess it is a pytest feature, so this will reset it. Another issue was that a cached config was used and it was not read from disk again. Then, dc._dreq_res needs to be set explicitly, since it was set to the default when dc was imported. Lastly, I had an issue in test_cli.py where I used os.chdir and the directory change persisted after the test. I was not aware that this is the case with pytest. I revert the CWD now after each test where I change it.

[rigoudyg]

@sol1105 Thank you very much for investigating this issue! I will have a try using your PR.
The idea of the new test script was to test the scripts as we can the the command line and I try to do it as we do it for command line.
On the add of script directories, we can speak of that on Thursday's meeting

[rigoudyg]

@sol1105 I have merge your pull request and it works correctly right now on my laptop. Thanks !

But I still have an issue when it is launched by actions.
The coverage fails as the command lines coverages remains at 0% even if they are launched.
@wolfiex @sol1105 Do you have an idea of the differences?
If I run the same command as the one which is launched for action, it works well on my laptop.

[rigoudyg]

@wolfiex The issue with github tests

[sol1105]

@rigoudyg Looking at the action logs, all tests pass, but the test coverage is considered less than the 50% threshold. The actions skip quite a few tests (that base on test data that you might only have locally available?), which might be the reason why the test coverage is sufficient when you run the tests locally, and not when run by github actions?

[rigoudyg]

@sol1105 When I launch locally, the percentage found is more than 50%: ~64%
For those tests, the official releases are used are are downloaded on a test directory, so it should not be an issue.
The tests work well but it seems that the the coverage of subprocesses it not taken into account correctly for test_cli.py as it is done for test_scripts.py (coverage remains at 0% for command-line scripts on github while it is from 61% to 94% locally)

Results obtained locally here:
Name Stmts Miss Branch BrPart Cover

/etc/python3.10/sitecustomize.py 5 1 0 0 80%
data_request_api/data_request_api/init.py 1 0 0 0 100%
data_request_api/data_request_api/command_line/init.py 0 0 0 0 100%
data_request_api/data_request_api/command_line/compare_variables.py 147 9 66 9 91%
data_request_api/data_request_api/command_line/config.py 25 2 16 1 88%
data_request_api/data_request_api/command_line/estimate_dreq_volume.py 218 49 94 19 73%
data_request_api/data_request_api/command_line/export_dreq_lists_json.py 85 26 34 7 61%
data_request_api/data_request_api/command_line/get_variables_metadata.py 28 1 6 1 94%
data_request_api/data_request_api/content/init.py 0 0 0 0 100%
data_request_api/data_request_api/content/consolidate_export.py 315 117 232 32 62%
data_request_api/data_request_api/content/dreq_content.py 251 44 124 27 78%
data_request_api/data_request_api/content/dump_transformation.py 351 35 200 22 88%
data_request_api/data_request_api/content/mapping_table.py 5 0 0 0 100%
data_request_api/data_request_api/query/init.py 0 0 0 0 100%
data_request_api/data_request_api/query/data_request.py 701 15 302 17 97%
data_request_api/data_request_api/query/dreq_classes.py 205 73 76 11 62%
data_request_api/data_request_api/query/dreq_query.py 549 143 318 65 70%
data_request_api/data_request_api/query/vocabulary_server.py 145 4 64 4 96%
data_request_api/data_request_api/utilities/init.py 0 0 0 0 100%
data_request_api/data_request_api/utilities/config.py 83 7 24 2 92%
data_request_api/data_request_api/utilities/decorators.py 27 0 10 0 100%
data_request_api/data_request_api/utilities/logger.py 50 12 20 6 69%
data_request_api/data_request_api/utilities/parser.py 6 0 2 0 100%
data_request_api/data_request_api/utilities/tools.py 29 4 6 2 83%
data_request_api/data_request_api/version.py 13 3 2 1 73%
scripts/check_consolidation2.py 342 342 212 0 0%
scripts/check_consolidation.py 87 87 24 0 0%
scripts/check_consolidation_and_transformation.py 416 416 264 0 0%
scripts/check_variables_attributes.py 78 3 22 1 96%
scripts/database_transformation.py 39 3 4 1 91%
scripts/workflow_example.py 20 20 2 0 0%
scripts/workflow_example_2.py 77 4 16 2 94%

TOTAL 4298 1420 2140 230 64%
Required test coverage of 50.0% reached. Total coverage: 63.56%
= 167 passed, 10 skipped, 1 xfailed, 59 warnings in 1755.03s (0:29:15) =