Most of the tox commands are no longer working

[shoeffner]

• Your issue may already be reported! Please search on the issue tracker before creating one.
  • Could not find anything
• Is this something you can debug and fix? Send a pull request! For more information, see the Contributor Guide.
  • Yes, I already worked on it
• We as maintainers foster an open and welcoming environment. Be respectfull, supportive and nice to each other! :)

Prerequisites

• Are you running the expected version of pyDataverse? (check via pip freeze).
  • main branch

Bug report

1. Describe your environment

• OS: MacOS Sonoma, M1
• pyDataverse: main branch
• Python: 3.8, 3.9, 3.10, 3.11, 3.12
• Dataverse: 6.3

2. Actual behaviour:

• tox -e py311 (should be roughly equivalent to running the docker-compose tests, which default to python 3.11) would not work, as the environment is not specified.
• Additionally, the tox.ini lists many dependencies directly or assuming an aptly named requirements.txt, which were removed with the transition to poetry.
• The docs cannot be built with tox -e docs because of that
• I have to pass DV_VERSION and BASE_URL; in fact my current invocation is:

  env $(grep -v '^#' local-test.env | xargs) tox -e docs
  

  where I added a few variables (API_TOKEN, DV_VERSION, BASE_URL, API_TOKEN_SUPERUSER) to local-test.env. (This is also required for me to run the tests with VSCode, but that's a different story. For DV_VERSION specifically, see also the discussion in Update DV_VERSION to 6.3 #197).

3. Expected behaviour:

That it works.

4. Steps to reproduce

1. tox -e docs
2. Alternatively, env $(grep -v '^#' local-test.env | xargs) tox -e docs with the relevant vars set (DV_VERSION, BASE_URL, API_TOKEN, API_TOKEN_SUPERUSER)

5. Possible solution

Either of the following:

• Update tox.ini
• Remove tox.ini

As I was working on #193 , when I tried to build it, I realized that tox -e docs would not work right now, plus some instructions referencing tox would not be valid.
I have now already started on updating the tox.ini, but I can also imagine removing it entirely is an option.

There are a number of things to discuss should the tox.ini be updated:

• Envs py36, py37 – remove: These two versions are EOL (3.8 is still supported on non-EOL Ubuntu versions though), so I think it is okay to drop them unless there are reasons to keep them around.
• Envs py39, py310, py311, py312 – add: These versions are currently in use and should/could be added. At least py311 should be added as that is used as a default in the tests.
• Envs py313 – maybe add: Not yet in widespread use, but available already.
• Envs pylint, flake8, flake8_docs, flake8_tests – remove: Instead, add ruff or lint and use the specified ruff. I am not sure if ruff handles everything the _docs and _tests envs would handle, but flake8 is not listed in the pyproject.toml anyways.
• Env black – remove: Editors can often use black already, pre-commit runs it, and ruff could also be used instead. Not much reason to keep the target around, I think.
• Env pre-commit – remove: Except for the --all-files, this is run on each commit anyways, someone who does not want to set it up to run for commits is not likely to run tox -e pre-commit either, I assume.
• Env dist_install – remove: This does not really seem to do much except running a pip freeze and printing the pyDataverse version.
• Env pypi, pypi_test: I don't know about these two. I would say they are not really needed, but if someone or some automation uses them to check the versions on the respective PyPI instances, they might be good to stay.
• Env packaging: Uses twine which is not listed in the pyproject.toml. I don't know how releases are currently done, so this might or might not be good.

For most envs, the deps parameter should be replaced with

commands_pre = 
  poetry lock --no-update
  poetry install --with=...

What do you think? Which envs should stay, which should be updated, which shouldn't? Or should the tox.ini be removed completely?

[shoeffner]

I found #181 which would at least also impact building the docs, so maybe the tox -e docs should be changed to already employ that -- or that should become a follow-up.