WIP: Parallel cucumbers

[MarcelloPerathoner]

Issue

Fixes #7307 - CI runs up from 0:45h to 4:30h

Improvements

Parallel execution of cucumber tests tremendously improves running times.
CI builds are down to 0:25h.

Note: when using --parallel N make sure there are N contiguous free ports at the
configured port number (eg. at ports 5000--5000+N).

A lot more can be configured with profiles (in cucumber.mjs). Usage examples:

$ npx cucumber-js -p dev -p ch -p mmap --format html:/tmp/testlog.html --parallel 8
$ npx cucumber-js -p ci -p mld -p directly --parallel 8
$ npm test -- --parallel $JOBS --fail-fast

On github CI full logs, also containing the outputs from all binaries, are published to
https://reports.cucumber.io/.

Code refactored and simplified. ~150 LoC less.

Other Fixes

During development other problems were found and fixed:

• The datastore load method now works as expected: one and the same osrm-routed will
  be running in the background for the whole duration of the test. Previous behaviour: for
  each scenario an osrm-routed would be spawned, loaded with datastore, and then killed
  again.

• osrm-datastore on macOS now correctly waits for other processes to detach from shared
  memory before exiting. Previous behaviour: osrm-datastore slept for 50ms and then
  boldly announced: 'All clients switched.' This caused tests to fail randomly because
  queries were executed against old data.

• The data watchdog in osrm-routed now will emit the message: 'updated facade' right
  after the update (previous behaviour: right before the update), so that it can be used
  for process synchronization.

• osrm::util::exception is now caught in osrm binaries. Previous behaviour: it was not
  caught and caused a SIGABRT (and a core dump).

Breaking changes

If you run the tests using the npx cucumber-js ... calling method you must add either
-p dev or -p ci as the first profile, eg.:

$ npx cucumber-js -p dev -p mld -p mmap --parallel 8

instead of:

$ OSRM_LOAD_METHOD=mmap npx cucumber-js -p mld

Old cached files are not cleaned automatically anymore because that interfered with
concurrency. There is no hook in cucumber that runs before the workers start.

We don't support changing the load method in mid-run anymore. (eg. "Given data is loaded
with datastore" no longer works). Reason: With the datastore load method an
osrm-routed will be constantly running in the background. No second osrm-routed can
open the same port again.

Retired Environment Variables

• OSRM_CONNECTION_RETRIES
• OSRM_CONNECTION_EXP_BACKOFF_COEF

Other changes

Removed one cache level. CH and MLD are processed and cached separately.

The outputs of osrm binaries are not written into lots of small files anymore. Instead
use the --format html:test/logs/testlog.html commandline argument to get one big HTML
file with all outputs embedded.

New configuration variables

New environment variables and/or cucumber worldParameters entries were introduced. The
whole configuration is done in cucumber.mjs.

Environment Variable	worldParameters	Defaults to
CUCUMBER_TIMEOUT	timeout	5000	Scenario timeout in ms.
CUCUMBER_HTTP_TIMEOUT	httpTimeout	2000	HTTP timeout in ms.
CUCUMBER_TEST_PATH	testPath	test	The test directory
CUCUMBER_PROFILES_PATH	profilesPath	profiles	The profiles directory
CUCUMBER_LOGS_PATH	logsPath	test/logs	The logs directory
CUCUMBER_CACHE_PATH	cachePath	test/cache	The cache directory
OSRM_BUILD_DIR	buildPath	build	Path to the binaries
OSRM_LOAD_METHOD	loadMethod	datastore	Data load method
OSRM_ALGORITHM	algorithm	ch	Routing algorithm
OSRM_IP	ip	127.0.0.1	IP Address
OSRM_PORT	port	5000	IP Port

New tags

Tag	A scenario thus tagged ...
@isolated	will not run while any other scenario is running in parallel
@with_(datastore|directly|mmap)	will be executed iff the load method matches
@no_(datastore|directly|mmap)	will be executed unless the load method matches
@with_(ci|mld)	will be executed iff the algorithm matches
@no_(ci|mld)	will be executed unless the algorithm matches

Tasklist

• CHANGELOG.md entry (How to write a changelog entry)
• TODO: clean up and remove some unrelated stuff
• update relevant Wiki pages
• review
• adjust for comments

Requirements / Relations

Link any requirements here. Other pull requests this PR is based on?

[afarber]

Thanks for working on speeding up the CI! The parallel cucumber approach looks promising.

One concern: could we keep the CI matrix changes out of this PR? Switching from macos-13 to macos-15-intel drops test coverage for users on Ventura/Sonoma, and adding clang-20 (which isn't released yet) seems unrelated to the parallelization work. These changes could be discussed separately if there's a reason to drop older platform support.

Also spotted a few bugs in osrm_loader.js - there are references to an undefined err variable in the error handlers (lines 45, 118, 132) that would crash if osrm-routed fails.

[MarcelloPerathoner]

The PR is still WIP. I will cleanup before releasing it.

BTW clang-21 is current. The reason to drop macos-13 is that github does not support it anymore.