Add GenerateAndBuildOnly Option

[lihaMSFT]

This PR adds an option to run toolchain generator and builder without actually executing the built binary.

This way we can attach other tools such as perfview and Visual Studio Performance Profiler to profile the benchmarks.

[dnfadmin]

All CLA requirements met.

[adamsitnik]

Hi @lihaMSFT

Thank you for your contribution! The PR LGTM, but I would like to better understand what you are trying to achieve.

This way we can attach other tools such as perfview and Visual Studio Performance Profiler to profile the benchmarks.

BDN provides a possibility to implement your own plugin, by implementing the IDiagnoser iterface. It allowed us to implement few profiler plugins:

• EtwProfiler which mimics PerfView and produces an .etl trace file that you can open with PerfView
• ConcurrencyVisualizerProfiler that mimics Concurrency Visualizer and produces a trace file that you can open both in Concurrency Visualizer in VS and in PerfView
• EventPipeProfiler which is a cross-platform profiler that produces a trace file which can be opened with https://www.speedscope.app/
• PerfCollectProfiler that internally uses perfcollect to profile benchmarked code on Linux

What are these built-in profilers missing? Perhaps we could just implement a new one that would be integrated with Visual Studio Profiler?

Thanks,
Adam

[lihaMSFT]

Hello Adam,

VS Performance Profiler supports multiple types of profiling: CPU, instrumentation, object allocation and has a large API surface. We can discuss a potential integration in the future.

Thanks.