Make each product use their own gRPC Device Server ports in the System Tests

[SaarthakMohanNI]

• This contribution adheres to CONTRIBUTING.md.

- [ ] I've updated CHANGELOG.md if applicable.

- [ ] I've added tests applicable for this pull request

What does this Pull Request accomplish?

Several different gRPC device server processes are created when running system tests, but all these different processes default to the same port, 31763. This collision is causing tests to fail to run.

List issues fixed by this Pull Request below, if any.

AB#3048787

What testing has been done?

[SaarthakMohanNI]

Note that I'm not 100% sure yet that each product doesn't run multiple gRPC tests simultaneously

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.35%. Comparing base (6a67d01) to head (6bd7478).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2086   +/-   ##
=======================================
  Coverage   91.34%   91.35%           
=======================================
  Files          66       66           
  Lines       16292    16292           
=======================================
+ Hits        14882    14883    +1     
+ Misses       1410     1409    -1     

Flag	Coverage Δ
codegenunittests	84.95% <ø> (ø)
nidcpowersystemtests	94.59% <ø> (+0.04%)	⬆️
nidcpowerunittests	89.53% <ø> (ø)
nidigitalsystemtests	92.26% <ø> (ø)
nidigitalunittests	68.44% <ø> (ø)
nidmmsystemtests	92.72% <ø> (ø)
nifakeunittests	87.24% <ø> (ø)
nifgensystemtests	94.86% <ø> (ø)
nimodinstsystemtests	73.85% <ø> (ø)
nimodinstunittests	94.20% <ø> (ø)
niscopesystemtests	92.94% <ø> (ø)
niscopeunittests	43.20% <ø> (ø)
nisesystemtests	91.50% <ø> (ø)
niswitchsystemtests	82.03% <ø> (ø)
nitclksystemtests	94.87% <ø> (ø)
nitclkunittests	98.26% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a67d01...6bd7478. Read the comment docs.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ni-jfitzger]

Note that I'm not 100% sure yet that each product doesn't run multiple gRPC tests simultaneously

Despite claims to the contrary in the niscope system tests, I'm pretty sure we don't parallelize execution of a tests for a given API. The "parallel-mode" argument passed to coverage merely changes the name of the coverage report file produced. See https://coverage.readthedocs.io/en/7.6.12/cmd.html

In order to run tests in parallel for an API, we would need to be using xdist.
See https://stackoverflow.com/questions/45733763/how-to-run-pytest-tests-in-parallel

[bkeryan]

FYI, you can configure it to use a dynamic port and read the port number from stdout. That's what InstrumentStudio's GrpcDeviceServerActivationService does.

[ni-jfitzger]

We considered it, but we felt this was safer, particularly with multiple instances of gRPC Device Server being kicked off at the same time. Our internal VM Build process launches up to 4 instances at once, repeatedly while testing up to 4 APIs in parallel, so they could, theoretically pick the same port.

[bkeryan]

That shouldn't happen. The server specifies a listening port of 0 when calling bind(), which tells the OS to allocate a port from the ephemeral port range. Then the server asks the OS which port it's using and prints that port number to stdout. This should only fail when all of the ports in the ephemeral port range are in use (and there are thousands of these).