Naming conventions for categories/types of test.

[TomMonks]

This is a point of discussion. We can chat first before you deciding if you want to make changes. I know you have used the taxonomy for the RAP materials! I think on balance it is worth it.

Alignment with The Turing Way

I think the current way tests are named in the tutorial: unit, functional, and back is potentially a little confusing if people are for example familiar with the Turing Way or general software testing. I think we should consider aligning the tutorial with The Turing Way as it gives it a lot of credibility.

For ref: https://book.the-turing-way.org/reproducible-research/testing/testing-integrationtest/

Turing breaks these down into

• Unit -> which aligns with your unit category
• Integration -> which I believe overlaps with your functional category. i.e. testing two or more components together.
• System -> which aligns with the "end 2 end" tests in your functional category
• Regression -> aligns perfectly with your back testing
• Smoke -> as discussed in (should we include a "smoke test" example to complement the functional tests? #17) this overlaps with your functional tests. Its a simple fast system test run first to stop other tests from running if it fails.

Scenarios are relevant across all types of test

I would consider if the scenarios currently listed in functional tests are a separate initial page that explains that each test type (unit/integration/system/regression" should cover. A few ways to describe this

• Clean or Positive tests: where code is given a valid input and returns valid outputs.
• Dirty or Negative or Error tests: where code fails as expected.
• Edge or Extreme tests

The Clean, Dirty, Edge distinction is something that I learnt a long time ago from Code Complete 2 by McConnell (we are talking over 20 years ago!). My understanding is that most people lean into Clean tests rather than trying to generate errors.

I think your Variation tests are a sub class of Clean. For research code my view is that they are very important! So I'd emphasise that a bit more.