Skip to content

Enable linux-x64 for TechEmpower SPMI scenarios #123742

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
EgorBo wants to merge 3 commits into
base: main
Choose a base branch
from
+10 −243
Draft

Enable linux-x64 for TechEmpower SPMI scenarios #123742

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cc4153d
Enable linux-x64 for TechEmpower SPMI scenarios
EgorBo Jan 29, 2026
b3b30e2
clean up
EgorBo Jan 29, 2026
0c1d0bc
Update superpmi_aspnet2.py
EgorBo Jan 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 1 addition & 227 deletions eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,232 +255,6 @@ extends:
jobParameters:
testGroup: outerloop

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: pmi
collectionName: libraries

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: crossgen2
collectionName: libraries

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: run
collectionName: realworld

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: run
collectionName: benchmarks

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: run_pgo
collectionName: benchmarks

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: run_pgo_optrepeat
collectionName: benchmarks

#
# Collection of coreclr test run
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/templates/runtimes/run-test-job.yml
buildConfig: checked
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: superpmi
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
SuperPmiCollect: true
unifiedArtifactsName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-collect-job.yml
buildConfig: checked
platforms:
- linux_arm64
- linux_x64
- windows_x64
- windows_arm64
helixQueueGroup: ci
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testGroup: outerloop
liveLibrariesBuildConfig: Release
collectionType: nativeaot
collectionName: smoke_tests

#
# Collection of libraries test run: normal
# Libraries Test Run using Release libraries, and Checked CoreCLR
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/libraries/run-test-job.yml
buildConfig: Release
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: superpmi
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testScope: innerloop
liveRuntimeBuildConfig: Checked
dependsOnTestBuildConfiguration: Release
dependsOnTestArchitecture: x64
scenarios:
- normal
SuperPmiCollect: true
SuperPmiCollectionName: libraries_tests
unifiedArtifactsName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_Checked
helixArtifactsName: LibrariesTestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_Checked
unifiedBuildConfigOverride: checked
# Default timeout is 150 minutes, which is too low for osx-arm64 queue.
timeoutInMinutes: 300

#
# Collection of libraries test run: no_tiered_compilation
# Libraries Test Run using Release libraries, and Checked CoreCLR
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/libraries/run-test-job.yml
buildConfig: Release
platforms:
- osx_arm64
- linux_arm
- linux_arm64
- linux_x64
- windows_x64
- windows_x86
- windows_arm64
helixQueueGroup: superpmi
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
jobParameters:
testScope: innerloop
liveRuntimeBuildConfig: Checked
dependsOnTestBuildConfiguration: Release
dependsOnTestArchitecture: x64
scenarios:
- no_tiered_compilation
SuperPmiCollect: true
SuperPmiCollectionName: libraries_tests_no_tiered_compilation
unifiedArtifactsName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_Checked
helixArtifactsName: LibrariesTestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_Checked
unifiedBuildConfigOverride: checked
# Default timeout is 150 minutes, which is too low for osx-arm64 queue.
timeoutInMinutes: 300

#
# Collection of aspnet/TechEmpower benchmarks (crank-based): aspnet2
#
Expand All @@ -491,7 +265,7 @@ extends:
platforms:
- osx_arm64
- linux_arm64
# - linux_x64 # currently runs of disk space too often.
- linux_x64
- windows_x64
- windows_arm64
helixQueueGroup: ci
Expand Down
25 changes: 9 additions & 16 deletions src/coreclr/scripts/superpmi_aspnet2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,10 @@ def setup_and_run_crank_agent(workdir: Path):
os.environ['DOTNET_CLI_TELEMETRY_OPTOUT'] = '1'
os.environ['DOTNET_MULTILEVEL_LOOKUP'] = '0'
os.environ['UseSharedCompilation'] = 'false'
os.environ['NUGET_PLUGINS_CACHE_PATH'] = str(workdir / "NUGET_PLUGINS_CACHE_PATH")
os.environ['NUGET_PACKAGES'] = str(workdir / "NUGET_PACKAGES")
os.environ['NUGET_HTTP_CACHE_PATH'] = str(workdir / "NUGET_HTTP_CACHE_PATH")
os.environ['NUGET_SCRATCH'] = str(workdir / "NUGET_SCRATCH")

print("Installing tools ...")
logs_dir.mkdir(parents=True, exist_ok=True)
Expand All @@ -172,24 +176,17 @@ def setup_and_run_crank_agent(workdir: Path):
# Create a Localhost.yml to define the local environment since we can't access the PerfLab.
yml = textwrap.dedent(
f"""
variables:
applicationAddress: 127.0.0.1
loadAddress: 127.0.0.1
applicationPort: {CRANK_PORT}
applicationScheme: http
loadPort: {CRANK_PORT}
loadScheme: http

profiles:
Localhost:
variables:
serverAddress: "{{{{applicationAddress}}}}"
jobs:
application:
endpoints:
- "{{{{applicationScheme}}}}://{{{{applicationAddress}}}}:{{{{applicationPort}}}}"
- "http://127.0.0.1:{CRANK_PORT}"
load:
endpoints:
- "{{{{loadScheme}}}}://{{{{loadAddress}}}}:{{{{loadPort}}}}"
- "http://127.0.0.1:{CRANK_PORT}"

""")
localhost_yml.write_text(yml, encoding="utf-8")

Expand All @@ -212,6 +209,7 @@ def setup_and_run_crank_agent(workdir: Path):
"--log-path", str(logs_dir),
"--build-path", str(build_dir),
"--dotnethome", str(dotnethome_dir),
"--build-timeout", "30",
],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, stdin=subprocess.DEVNULL,
creationflags=creation_flags,
Expand Down Expand Up @@ -249,11 +247,6 @@ def run_crank_scenario(crank_app: Path, scenario_name: str, framework: str, work
"--application.runtimeVersion", "10.0.0-rtm.25513.102",
"--application.sdkVersion", "10.0.100-rtm.25513.102",

"--application.collectDependencies", "false",
"--application.options.collectCounters", "false",
"--load.options.reuseBuild", "true",
"--load.variables.duration", "45",
"--load.variables.warmup", "15",
"--load.job", "bombardier", # Bombardier is more cross-platform friendly (wrk is linux only)
]

Expand Down
Loading