diff --git a/content/install-guides/kernel-build.md b/content/install-guides/kernel-build.md new file mode 100644 index 0000000000..c507de096c --- /dev/null +++ b/content/install-guides/kernel-build.md @@ -0,0 +1,235 @@ +--- +title: Compiling Linux Kernels for Arm +additional_search_terms: +- linux kernel +- tuxmake +- fastpath +- Image.gz +- modules.tar.xz +minutes_to_complete: 60 +author: Geremy Cohen +official_docs: https://tuxmake.org/ +test_images: +- ubuntu:latest +test_maintenance: true +weight: 1 +tool_install: true +multi_install: false +multitool_install_part: false +layout: installtoolsall +--- + +This guide walks you through building and installing Linux kernels on Arm cloud VM instances using utility scripts available at [`arm_kernel_install_guide`](https://github.com/geremyCohen/arm_kernel_install_guide). + +## What do I need before building Arm kernels? + +Before you begin, choose a cloud provider, and spin up an instance with the following characteristics: + +### Cloud Provider ### +This guide uses AWS as the example platform, but you can follow the same steps on any cloud provider that offers 64-bit Arm Ubuntu instances. + +### Instance Type ### +This guide uses an AWS `c8g.24xlarge` instance for demonstration. Any sufficiently large instance on your chosen provider will work, however, smaller instances may take longer or risk running out of memory during compilation. If you choose to use a different instance type, the minimal requirements are an Arm instance running at least 24 vCPUs with 200 GB of free storage. + +### Operating System ### +Ubuntu 24.04 LTS (64-bit Arm) is the recommended OS for this guide. Other distributions may work but are not officially supported (yet). If you find a different distro that works well, or you'd like to request support for a different setup, please open an issue or pull request. + +# Install and Clone + +With your build instance running and accessible via SSH, install the required dependencies: + +```bash +sudo apt update +sudo apt install -y git python3 python3-pip python3-venv build-essential bc rsync dwarves flex bison libssl-dev libelf-dev btop yq jq + +cd +git clone https://github.com/geremyCohen/arm_kernel_install_guide.git ~/arm_kernel_install_guide +cd ~/arm_kernel_install_guide +chmod +x scripts/*.sh +``` + +All commands in this guide assume you are inside this directory. The important script is `scripts/kernel_build_and_install.sh`, which orchestrates cloning the upstream kernel tree, configuring tuxmake, building artifacts, and optionally installing the kernel. The script now runs non-interactively—once invoked it proceeds without confirmation prompts, and any install operation automatically reboots the system when it finishes. + +## How do I build Arm kernels? + +The `kernel_build_and_install.sh` script is intentionally modular. Most users fall into two buckets: + +1. **General Usage (non-fastpath)** – build kernels for direct install or downstream packaging. +2. **Fastpath Usage** – build kernels that add the *fastpath* headers/perf configuration needed by the *fastpath* validation tool. Aside from those extra configs, the workflow mirrors the general case. + +The sections below start simple (demo flags) and progress toward advanced scenarios. Every flag referenced is shown in at least one example so you can mix and match confidently. + +### Flag overview + +#### General usage flags + +| Flag | Description | +| --- | --- | +| `--demo-default-build` | Shortcut: builds `v6.18.1` with default configs and leaves *fastpath* disabled. | +| `--tag ` / `--tags ` / `--tag-latest` | Select one or more kernel tags. Multiple tags build in parallel; the latest stable release can be added via `--tag-latest`. | +| `--install-from ` / `--install-format ` | Install an existing build (flat artifacts or `.deb` packages) without recompiling. | +| `--dry-run` | Generate a self-contained plan script (stored in `/tmp/kernel_plan_*.sh`) with the resolved arguments and exit without running the build. | +| `--kernel-install [tag\|bool]` | Install a kernel right after it finishes building. When multiple tags build, provide the specific tag to install. | +| `--change-to-64k ` | Generate a 64 KB page-size kernel. Often combined with the install flags to test high-page builds. | +| `--config-file ` | Reuse a captured stock config instead of `/boot/config-$(uname -r)`. | +| `--include-bindeb-pkg` | Adds the `bindeb-pkg` target so `.deb` packages are produced alongside `Image.gz` and `modules.tar.xz`. | +| `--kernel-command-line ` | Override GRUB’s `GRUB_CMDLINE_LINUX` when installing a kernel. | +| `--append-to-kernel-version ` | Attach custom suffixes to `EXTRAVERSION` (e.g., `--append "-lab"`). | +| `--kernel-dir ` / `--venv-path ` | Control where the kernel git checkout lives and which Python venv hosts tuxmake. Build artifacts always land under `~/kernels/`. | + +#### Fastpath usage flags + +| Flag | Description | +| --- | --- | +| `--demo-fastpath-build` | Shortcut: builds `v6.18.1` and `v6.19-rc1` with *fastpath* configs enabled. | +| `--fastpath ` | Manually enable/disable the *fastpath* configuration overlay (installs Docker as needed). | + +Run `./scripts/kernel_build_and_install.sh --help` anytime for the exhaustive list. + +--- + +## General Usage + +### Worked examples (general usage) + +#### 1. Quick sanity check (demo) +```bash +./scripts/kernel_build_and_install.sh --demo-default-build +``` +This demo builds `v6.18.1`, populates `~/kernels/6.18.1`, and leaves Docker as well as *fastpath* configs untouched. + +#### 2. Specify your own tag +```bash +./scripts/kernel_build_and_install.sh --tags v6.19-rc1 +``` +This behaves like the demo while targeting a release candidate instead of the pinned stable tag, and it still runs without any interactive prompts. + +#### 3. Produce both flat artifacts and Debian packages +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1 \ + --include-bindeb-pkg +``` +Running this command outputs `Image.gz`, `modules.tar.xz`, `perf.tar.xz`, and `.deb` files (headers, image, dbg) under `~/kernels/6.18.1`. + +#### 4. Build and immediately install (single tag) +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1 \ + --kernel-install true +``` +This command installs the freshly built kernel, regenerates initramfs, updates GRUB, and then reboots automatically. + +#### 5. Multi-tag build + targeted install +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1,v6.19-rc1 \ + --kernel-install v6.18.1 +``` +Both kernels build in parallel, but only `v6.18.1` is installed (followed by an automatic reboot), leaving the `v6.19-rc1` artifacts untouched under `~/kernels`. + +#### 6. 64K page-size build and install +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1 \ + --change-to-64k true \ + --kernel-install true \ + --append-to-kernel-version "-64k" +``` +This variation produces a 64 KB build, installs it, appends “-64k” to the reported kernel version, and reboots automatically so you can verify the new settings. + +#### 7. Install-only workflow (reusing flat artifacts) +```bash +./scripts/kernel_build_and_install.sh \ + --install-from ~/kernels/6.18.1 \ + --install-format flat +``` +Instead of compiling, the script installs the saved `Image.gz`, `modules.tar.xz`, and `config` from a prior run, which is ideal when the directory contains flat artifacts rather than `.deb` packages. + +#### 8. Install-only with Debian packages +```bash +./scripts/kernel_build_and_install.sh \ + --install-from ~/kernels/6.18.1 \ + --install-format deb +``` +Here the script installs the `.deb` artifacts produced earlier via `--include-bindeb-pkg`, expecting files such as `linux-image-*` and `linux-headers-*` to exist in the source directory. + +#### 9. Install-only with auto-detection +```bash +./scripts/kernel_build_and_install.sh \ + --install-from ~/kernels/6.18.1 +``` +This form lets the script auto-detect whether the directory contains flat artifacts or `.deb` files, which simplifies reuse when you are not sure which format is present. + +#### 10. Generate a runnable plan without executing +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1 \ + --dry-run +``` +Instead of performing the build, this command writes a self-contained plan such as `/tmp/kernel_plan_v6.18.1_.sh` that embeds the current script plus the resolved arguments (minus `--dry-run`). Running that plan file later—on the same host or another system with the required dependencies—replays the exact workflow. + +--- + +## Fastpath Usage + +*fastpath* builds use the same tuxmake pipelines but add a configuration fragment that exposes the interfaces needed by the *fastpath* testing framework (extra headers, perf tooling, and Docker so *fastpath* can drive the host). *fastpath* workflows are build-only: do not combine `--fastpath true` (or the demo shortcut) with `--kernel-install` or any `--install-from` commands. Instead, let the build finish, copy the flat artifacts (`Image.gz`, `modules.tar.xz`, and `config`) to the *fastpath* host, and let the *fastpath* tooling handle deployment to the SUT. Docker is still installed automatically whenever *fastpath* mode is enabled so the *fastpath* controller can manage the host. + +*fastpath* runs can still take advantage of tuning flags such as `--change-to-64k`, alternate configs, or custom output directories. Even if you specify packaging flags such as `--include-bindeb-pkg`, *fastpath* tests consume the flat artifacts. + +### Fastpath examples + +#### 1. Demo (dual-tag baseline) +```bash +./scripts/kernel_build_and_install.sh --demo-fastpath-build +``` +The demo builds `v6.18.1` and `v6.19-rc1` with *fastpath* configs enabled and installs Docker automatically if the host lacks it. + +#### 2. Custom tags with Fastpath enabled +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1,v6.19-rc1 \ + --fastpath true +``` +This explicit version mirrors the demo while making it easy to swap tag sets or add additional flags. + +#### 3. Fastpath build with additional tuning +```bash +./scripts/kernel_build_and_install.sh \ + --tags v6.18.1,v6.19-rc1 \ + --fastpath true \ + --change-to-64k true +``` +This variation still produces build-only artifacts, but it proves that you can layer other build-time options (like a 64 KB page size) on top of *fastpath* runs before exporting the results to the *fastpath* host. + +--- + +### Where are the artifacts stored? + +Each kernel tag produces a directory under `~/kernels/` containing: + +- `Image.gz` – compressed kernel image. +- `modules.tar.xz` – modules tree (untar to `/lib/modules/` when installing elsewhere). +- `perf.tar.xz`, `cpupower.tar.xz` – optional user-space tools. +- `config` – final merged configuration. +- `config.stock` – copy of the original base config used for the build. + +The script also writes a copy of the base config to `~/kernels/stock-configs/`, named after the running host kernel. Preserve this directory if you want to reuse the stock configuration later (for example, pass it via `--config-file`). + +### How do I verify the results? + +After every run: + +```bash +ls ~/kernels +ls ~/kernels/ +``` + +If you installed the kernel, the script reboots automatically when the install finishes. After it comes back up, confirm: + +```bash +uname -r +getconf PAGE_SIZE # expect 65536 for 64K builds, 4096 otherwise +``` + +When using build-only workflows, copy `Image.gz`, `modules.tar.xz`, and the corresponding `config` file to the downstream environment that will consume the kernel. diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/_index.md b/content/learning-paths/servers-and-cloud-computing/fastpath/_index.md new file mode 100644 index 0000000000..801658332f --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/_index.md @@ -0,0 +1,68 @@ +--- +title: Fastpath Kernel Build and Install Guide + +minutes_to_complete: 45 + +who_is_this_for: Software developers and performance engineers who want to explore benchmarking across different kernel versions with Fastpath on Arm. + +learning_objectives: + - Understand how Fastpath streamlines kernel experimentation workflows + - Provision an Arm-based build machine and compile Fastpath-enabled kernels on it + - Provision an Arm-based test system, also known as the System Under Test (SUT) + - Create a test plan consisting of kernel versions and benchmark suites + - Launch an Arm-based Fastpath host to orchestrate the kernel benchmarking process on the SUT + +prerequisites: + - An AWS account with permissions to create EC2 instances + - Familiarity with basic Linux administration and SSH + +author: Geremy Cohen + +### Tags +skilllevels: Intermediate +subjects: Operating Systems +armips: + - Neoverse +operatingsystems: + - Linux +tools_software_languages: + - Fastpath + - tuxmake + - Linux kernel + +further_reading: + - resource: + title: Fastpath documentation + link: https://fastpath.docs.arm.com/en/latest/index.html + type: documentation + - resource: + title: Kernel install guide + link: /install-guides/kernel-build/ + type: guide + - resource: + title: AWS Compute Service Provider learning path + link: /learning-paths/servers-and-cloud-computing/csp/ + type: guide + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 +layout: "learningpathall" +learning_path_main_page: "yes" +--- + +*fastpath* accelerates the cycle of building, deploying, and benchmarking Linux kernels on Arm-based infrastructure. + +Off-the-shelf distributions ship with general-purpose kernels, but when you want to maximize performance you often need to rebuild the kernel with custom configuration options, experimental patches, or prerelease code. Custom kernels let you validate questions like “does an RC fix my workload regression?” or “do these extra debug settings impose measurable overhead?” without waiting for distro updates. + +This learning path focuses on a concrete use case: run the Speedometer browser benchmark on two different kernel versions and determine which kernel delivers the best score. The workflow mirrors what kernel engineers do every day—build, deploy, and compare—while *fastpath* keeps the process reproducible. + +To make that manageable we split the work across three Arm-based nodes: + +1. **Build host** – compiles the kernels with *fastpath*-specific options. +2. **fastpath host** – orchestrates deployments, plan execution, and result collection. +3. **System Under Test (SUT)** – runs each kernel and executes the benchmark workloads. + +Arm’s `arm_kernel_install_guide` repository supplies wrapper scripts that streamline each step. You will use them to compile kernels on the build host, prepare the *fastpath* host and SUT, generate a plan, execute it, and then read the results without having to stitch together the workflow manually. + +> **Tip:** The complete *fastpath* reference documentation is available at [fastpath.docs.arm.com](https://fastpath.docs.arm.com/en/latest/index.html). diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/analyze_benchmark_results.md b/content/learning-paths/servers-and-cloud-computing/fastpath/analyze_benchmark_results.md new file mode 100644 index 0000000000..4a99c0d704 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/analyze_benchmark_results.md @@ -0,0 +1,111 @@ +--- +title: "Analyze the Benchmark Results" +weight: 14 + +layout: "learningpathall" +--- + +## Review benchmark results + + + +To inspect the outputs stored in the `results/` directory you can use the sample commands output when `generate_plan.sh` completed. Some examples are below: + +### List swprofiles involved from tests saved in the *results* folder: + +To see the swprofiles (kernels) tested and stored in the results directory, run: + +```commandline +source ~/venv/bin/activate +fastpath result list results-fastpath_test_010826-1837 --object swprofile +``` + +If you followed the tutorial exactly, you should see output similar to: + +```output ++------------------+----------------------+ +| ID | fp_6.18.1-ubuntu | ++------------------+----------------------+ +| Kernel Name | 6.18.1-ubuntu+ | ++------------------+----------------------+ +| Kernel Git SHA | | ++------------------+----------------------+ +| Userspace | Ubuntu 24.04.3 LTS | ++------------------+----------------------+ +| cmdline | | ++------------------+----------------------+ +| sysctl | | ++------------------+----------------------+ +| bootscript | | ++------------------+----------------------+ + ++------------------+----------------------+ +| ID | fp_6.19.0-rc1-ubuntu | ++------------------+----------------------+ +| Kernel Name | 6.19.0-rc1-ubuntu+ | ++------------------+----------------------+ +| Kernel Git SHA | | ++------------------+----------------------+ +| Userspace | Ubuntu 24.04.3 LTS | ++------------------+----------------------+ +| cmdline | | ++------------------+----------------------+ +| sysctl | | ++------------------+----------------------+ +| bootscript | | ++------------------+----------------------+ +``` + +### View relative results per kernel + +To see the relative results for each kernel, run the following commands: + +```commandline + fastpath result show results-fastpath_test_010826-1837 --swprofile fp_6.19.0-rc1-ubuntu --relative +``` +Relative in this case means that the statistics displayed are relative to the mean. In addition to the min/mean/max, you are also given the confidence interval bounds, the coefficient of variation and the number of samples, similar to: + +```output ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +| Benchmark | Result Class | min | ci95min | mean | ci95max | max | cv | count | ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +| speedometer/v2.1 | score (runs/min) | -1.81% | -2.42% | 221.00 | 2.42% | 1.81% | 1.52% | 4 | ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +``` + +You can run it again for the other kernel: + +```commandline +fastpath result show results-fastpath_test_010826-1837 --swprofile fp_6.18.1-ubuntu --relative +``` +with output similar to: + +```output ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +| Benchmark | Result Class | min | ci95min | mean | ci95max | max | cv | count | ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +| speedometer/v2.1 | score (runs/min) | -0.86% | -1.25% | 233.00 | 1.25% | 0.86% | 0.78% | 4 | ++------------------+--------------------+--------+----------+--------+----------+--------+--------+-------+ +``` + +### Compare results between kernels + +To compare the relative results between both kernels, run: + + + +```commandline +fastpath result show results-fastpath_test_010826-1837 --swprofile fp_6.19.0-rc1-ubuntu --swprofile fp_6.18.1-ubuntu --relative +``` +with output similar to: + +```output ++------------------+--------------------+-----------------------+---------------------+ +| Benchmark | Result Class | fp_6.19.0-rc1-ubuntu | fp_6.18.1-ubuntu | ++------------------+--------------------+-----------------------+---------------------+ +| speedometer/v2.1 | score (runs/min) | 221.00 | 3.39% | ++------------------+--------------------+-----------------------+---------------------+ +``` +We see 6.18.1 is performing slightly better than 6.19-rc1 in this benchmark. + +More examples of analyzing results can be found in the [*fastpath* Results User Guide](https://fastpath.docs.arm.com/en/latest/user-guide/resultshow.html). diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/build_setup.md b/content/learning-paths/servers-and-cloud-computing/fastpath/build_setup.md new file mode 100644 index 0000000000..31f2ea71b0 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/build_setup.md @@ -0,0 +1,280 @@ +--- +title: "Setup Build Instance" + +weight: 3 + +layout: "learningpathall" +--- + +## Provision the build host + +CPU-optimized instances compile kernels quickly, so in our example, an AWS Graviton4 `c8g.24xlarge` instance is used. It will be referred to as the *build* machine throughout the rest of the guide. + +{{% notice Note %}} +The following steps involve launching an EC2 instance. You can perform all EC2 instance creation steps via the AWS Management Console instead or AWS CLI. For step-by-step instructions to bring up an EC2 instance via the console, consult the [Compute Service Provider learning path](/learning-paths/servers-and-cloud-computing/csp/) for detailed instructions. A tutorial from AWS is also available via [Get started with Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html). +{{% /notice %}} + +Create build host with the following specifications: + +1. **Name** — *fastpath-build* +2. **Operating system** — *Ubuntu* +3. **AMI** — *Ubuntu 24.04 LTS (Arm)* +4. **Architecture** — *64-bit Arm* +5. **Instance type** — `c8g.24xlarge` +6. **Key pair** — *Select or create a key for SSH* +7. **Security group** — *allow SSH inbound from your IP and cluster peers* +8. **Storage** — *200 GB gp3* + +There are many different ways to create this instance, and a few different methods are demonstrated below. Choose the method which suits you best. + +{{< tabpane >}} + {{< tab header="AWS Console" img_src="/learning-paths/servers-and-cloud-computing/fastpath/images/ec2_setup.png">}} + {{< /tab >}} + + {{< tab header="AWS CLI" language="shell">}} +# Replace the placeholders with values from your account/environment +aws ec2 run-instances \ + --image-id resolve:ssm:/aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id \ + --instance-type c8g.24xlarge \ + --key-name \ + --subnet-id \ + --security-group-ids \ + --associate-public-ip-address \ + --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":200,"VolumeType":"gp3","DeleteOnTermination":true}}]' \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=fastpath-build},{Key=fastpath:role,Value=build}]' + {{< /tab >}} + + {{< tab header="CloudFormation" language="yaml" >}} + +cat <<'EOF' > build-host.yaml + +# README FIRST! +# +# 1. Click the copy icon to save this file to your clipboard. +# 2. Paste it into your terminal, it will be saved as `build-host.yaml` +# 3. Go to the CloudFormation console at https://us-east-1.console.aws.amazon.com/cloudformation/home +# 4. Click "Create stack -> With new resources (standard)" +# 5. Choose "Upload a template file" in the CloudFormation console. +# 6. Name the stack "fastpath-build", enter remaining required values, then click "Submit" to create the stack. + + +AWSTemplateFormatVersion: '2010-09-09' +Description: >- + Fastpath Learning Path - Build host for kernel compilation (Ubuntu 24.04 LTS on Graviton4 c8g.24xlarge). + +Parameters: + LatestUbuntuAmiId: + Type: 'AWS::SSM::Parameter::Value' + Default: /aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id + Description: SSM parameter for the latest Ubuntu 24.04 LTS (Arm) AMI. + InstanceType: + Type: String + Default: c8g.24xlarge + AllowedValues: + - c8g.24xlarge + - c8g.16xlarge + - c8g.12xlarge + - c8g.8xlarge + - c8g.4xlarge + Description: Instance size for the build host. + KeyPairName: + Type: AWS::EC2::KeyPair::KeyName + Description: Existing EC2 key pair to enable SSH access. + VpcId: + Type: AWS::EC2::VPC::Id + Description: VPC where the build host will run. + SubnetId: + Type: AWS::EC2::Subnet::Id + Description: Subnet (preferably public) for the build host. + SSHAllowedCidr: + Type: String + Default: 0.0.0.0/0 + Description: CIDR block allowed to SSH into the build host. + BuildSecurityGroupId: + Type: String + Default: '' + Description: Security group ID of the Fastpath host to allow peer-to-peer SSH. + RootVolumeSizeGiB: + Type: Number + Default: 200 + MinValue: 100 + MaxValue: 1024 + Description: Size (GiB) of the gp3 root volume. + +Conditions: + HasFastpathPeer: !Not [ !Equals [ !Ref BuildSecurityGroupId, '' ] ] + +Resources: + BuildSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Enable SSH access for Fastpath build host + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 22 + ToPort: 22 + CidrIp: !Ref SSHAllowedCidr + SecurityGroupEgress: + - IpProtocol: -1 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: fastpath-build-sg + + BuildHost: + Type: AWS::EC2::Instance + Properties: + ImageId: !Ref LatestUbuntuAmiId + InstanceType: !Ref InstanceType + KeyName: !Ref KeyPairName + SubnetId: !Ref SubnetId + SecurityGroupIds: + - !Ref BuildSecurityGroup + BlockDeviceMappings: + - DeviceName: /dev/sda1 + Ebs: + VolumeSize: !Ref RootVolumeSizeGiB + VolumeType: gp3 + Encrypted: true + DeleteOnTermination: true + Tags: + - Key: Name + Value: fastpath-build + - Key: fastpath:role + Value: build + +Outputs: + InstanceId: + Description: ID of the build host EC2 instance. + Value: !Ref BuildHost + PublicIp: + Description: Public IPv4 address (if assigned) to reference as BUILD_PUBLIC_IP. + Value: !GetAtt BuildHost.PublicIp + PrivateIp: + Description: Private IPv4 address to reference as BUILD_PRIVATE_IP. + Value: !GetAtt BuildHost.PrivateIp + SecurityGroupId: + Description: Security group attached to the build host. + Value: !Ref BuildSecurityGroup +EOF + + {{< /tab >}} +{{< /tabpane >}} + +When the instance reports a `running` state, note the public and private IP addresses as BUILD_PUBLIC_IP and BUILD_PRIVATE_IP. You'll need these values later. + +## Clone the Kernel Build repository on the build machine + +The Kernel Build repository contains build scripts and configuration files needed to easily compile kernels. To clone the repository: + +1. SSH into the `c8g.24xlarge` host using the configured key pair. + + ```output + ssh -i ~/.ssh/gcohen1.pem ubuntu@34.216.87.65 + + $ ssh -i ~/.ssh/gcohen1.pem ubuntu@54.174.185.226 + + Warning: Permanently added '54.174.185.226' (ED25519) to the list of known hosts. + Welcome to Ubuntu 24.04.3 LTS (GNU/Linux 6.14.0-1018-aws aarch64) + + System information as of Thu Jan 8 18:12:07 UTC 2026 + + System load: 0.35 Users logged in: 0 + Memory usage: 0% IPv4 address for ens66: 172.31.110.110 + + Last login: Thu Jan 8 18:12:14 2026 from 217.140.103.82 + ubuntu@ip-172-31-110-110:~$ + ``` + +2. Open the [Install and Clone section](https://localhost:1313/install-guides/kernel-build/#install-and-clone) of the install guide from your workstation. + +3. Run each command from that section on the build machine. It should be similar to the following (always refer to the above link for the latest command line): + + +```output +$ sudo apt update +$ sudo apt install -y git python3 python3-pip python3-venv build-essential bc rsync dwarves flex bison libssl-dev libelf-dev btop yq +$ cd +$ git clone https://github.com/geremyCohen/arm_kernel_install_guide.git ~/arm_kernel_install_guide +$ cd ~/arm_kernel_install_guide +$ chmod +x scripts/*.sh + +WARNING: apt does not have a stable CLI interface. Use with caution in scripts. +Hit:1 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble InRelease +Get:2 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble-updates InRelease [126 kB] +Get:3 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble-backports InRelease [126 kB] +Get:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease [126 kB] +... +0 upgraded, 104 newly installed, 0 to remove and 43 not upgraded. +Setting up build-essential (12.10ubuntu1) ... +Setting up libssl-dev:arm64 (3.0.13-0ubuntu3.6) ... +Setting up python3-pip (24.0+dfsg-1ubuntu1.3) ... +Cloning into 'arm_kernel_install_guide'... +``` + +## Building with the Kernel Build utility script + +With the repository cloned, you can now produce kernels with *fastpath* support. + +Normally, to manually build, you'd have to: + + - Update the host and install proper versions of every kernel build dependency + - Find and utilize the current stock kernel config + - Clone the upstream kernel tree to fetch the desired versions and clean the tree between builds + - For each kernel version, copy the base config into the workspace, and append all *fastpath*-specific options + - Run tuxmake for each kernel with the proper options + - Repeat the entire process for the second tag, ensuring the builds don’t collide + - Verify both kernel directories contain the required files + +But do not fear... Using ```scripts/kernel_build_and_install.sh``` bundles all those steps together in a single easy-to-use command. + +### Which kernels should you build and test against with Fastpath? +The answer to this question depends on what you are trying to accomplish. + +If you are running through the *fastpath* tutorial for the first time and getting used to how it works, its fine to use the arbitrary kernel versions given in *fastpath* Example 2, which are v6.18.1 and v6.19-rc1. + +Once you are familiar with the process and you wish to explore and test further, choose any specific kernel versions, based on your use case. + +## Compile and build Fastpath-enabled kernels 6.18.1 and 6.19-rc1 + +To run the script with *fastpath* options: + +1. On the build machine, ```cd``` into the `arm_kernel_install_guide` folder you just cloned. + +```command +cd ~/arm_kernel_install_guide +``` + +```output +ubuntu@ip-172-31-110-110:~/arm_kernel_install_guide$ +``` + +2. Open the [Custom tags with *fastpath* enabled](http://localhost:1313/install-guides/kernel-build/#2-custom-tags-with-fastpath-enabled) section from the install guide, and follow the instructions to run the build script. It should be similar to the following (always refer to the above link for the latest command line): + +```output +$ ./scripts/kernel_build_and_install.sh --tags v6.18.1,v6.19-rc1 --fastpath true + +[2026-01-08 18:13:14] Updating apt metadata +Kernel build settings: + Repo: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + Branch: linux-rolling-stable + Tags: v6.18.1,v6.19-rc1 + Config file: /boot/config-6.14.0-1018-aws + Kernel dir base: /home/ubuntu/kernels/linux + Output base: /home/ubuntu/kernels + Fastpath configs: true + 64K page size: false + Kernel install: false +... +I: build output in /home/ubuntu/kernels/6.18.1-ubuntu +[2026-01-08 18:32:03] [v6.18.1-1] Build artifacts are located in /home/ubuntu/kernels/6.18.1-ubuntu+ +I: build output in /home/ubuntu/kernels/6.19.0-rc1-ubuntu +[2026-01-08 18:32:06] [v6.19-rc1-2] Build artifacts are located in /home/ubuntu/kernels/6.19.0-rc1-ubuntu+ +``` + +The script will now build two kernel images. This process may take some time -- on a `c8g.24xlarge` instance, expect approximately 30 minutes for both kernel builds to complete. + +4. Monitor the console output for the `BUILD COMPLETE` message. + +Once finished, you will be ready to move on to the next step, where you prepare the *fastpath* host. diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/fastpath_setup.md b/content/learning-paths/servers-and-cloud-computing/fastpath/fastpath_setup.md new file mode 100644 index 0000000000..24b24658b7 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/fastpath_setup.md @@ -0,0 +1,274 @@ +--- +title: "Setup Fastpath Instance" + +weight: 6 + +layout: "learningpathall" +--- +With newly compiled kernels ready and waiting on the build instance, it's time to set up the *fastpath* host. + +The *fastpath* host will manage testing against the system under test (SUT) and coordinate benchmarking runs. + +## Provision the Fastpath host + +{{% notice Note %}} +The following steps involve launching an EC2 instance. You can perform all EC2 instance creation steps via the AWS Management Console instead or AWS CLI. For step-by-step instructions to bring up an EC2 instance via the console, consult the [Compute Service Provider learning path](/learning-paths/servers-and-cloud-computing/csp/) for detailed instructions. A tutorial from AWS is also available via [Get started with Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html). +{{% /notice %}} + +Create build host with the following specifications: + +1. **Name** — *fastpath-host* +2. **Operating system** — *Ubuntu* +3. **AMI** — *Ubuntu 24.04 LTS (Arm)* +4. **Architecture** — *64-bit Arm* +5. **Instance type** — `c8g.24xlarge` +6. **Key pair** — *Select or create a key for SSH* +7. **Security group** — *allow SSH inbound from your IP and cluster peers* +8. **Storage** — *200 GB gp3* + +{{< tabpane >}} + {{< tab header="AWS Console" img_src="/learning-paths/servers-and-cloud-computing/fastpath/images/ec2_setup.png">}} + {{< /tab >}} + + {{< tab header="AWS CLI" language="shell">}} +# Replace the placeholders with values from your account/environment +aws ec2 run-instances \ + --image-id resolve:ssm:/aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id \ + --instance-type c8g.4xlarge \ + --key-name \ + --subnet-id \ + --security-group-ids \ + --associate-public-ip-address \ + --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":200,"VolumeType":"gp3","DeleteOnTermination":true}}]' \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=fastpath-host},{Key=fastpath:role,Value=controller}]' + {{< /tab >}} + + {{< tab header="CloudFormation" language="yaml">}} + +cat <<'EOF' > fastpath-host.yaml + +# README FIRST! +# +# 1. Click the copy icon to save this file to your clipboard. +# 2. Paste it into your terminal, it will be saved as `fastpath-host.yaml` +# 3. Go to the CloudFormation console at https://us-east-1.console.aws.amazon.com/cloudformation/home +# 4. Click "Create stack -> With new resources (standard)" +# 5. Choose "Upload a template file" in the CloudFormation console. +# 6. Name the stack "fastpath-host", enter remaining required values, then click "Submit" to create the stack. + +AWSTemplateFormatVersion: '2010-09-09' +Description: >- + Fastpath Learning Path - Fastpath host (controller) instance for benchmarking orchestration. + +Parameters: + LatestUbuntuAmiId: + Type: 'AWS::SSM::Parameter::Value' + Default: /aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id + Description: SSM parameter for the latest Ubuntu 24.04 LTS (Arm) AMI. + InstanceType: + Type: String + Default: c8g.4xlarge + AllowedValues: + - c8g.4xlarge + - c8g.8xlarge + - c8g.12xlarge + Description: Instance size for the Fastpath host. + KeyPairName: + Type: AWS::EC2::KeyPair::KeyName + Description: Existing EC2 key pair to enable SSH access. + VpcId: + Type: AWS::EC2::VPC::Id + Description: VPC where the Fastpath host will run. + SubnetId: + Type: AWS::EC2::Subnet::Id + Description: Subnet (public or private with outbound access) for the Fastpath host. + SSHAllowedCidr: + Type: String + Default: 0.0.0.0/0 + Description: CIDR block allowed to SSH into the Fastpath host. + FastpathSecurityGroupId: + Type: String + Default: '' + Description: Security group ID of the Fastpath host to allow peer-to-peer SSH. + RootVolumeSizeGiB: + Type: Number + Default: 200 + MinValue: 100 + MaxValue: 1024 + Description: Size (GiB) of the gp3 root volume. + + +Conditions: + HasFastpathPeer: !Not [ !Equals [ !Ref FastpathSecurityGroupId, '' ] ] + +Resources: + FastpathSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Enable SSH access for the Fastpath host + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 22 + ToPort: 22 + CidrIp: !Ref SSHAllowedCidr + SecurityGroupEgress: + - IpProtocol: -1 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: fastpath-host-sg + + FastpathHost: + Type: AWS::EC2::Instance + Properties: + ImageId: !Ref LatestUbuntuAmiId + InstanceType: !Ref InstanceType + KeyName: !Ref KeyPairName + SubnetId: !Ref SubnetId + SecurityGroupIds: + - !Ref FastpathSecurityGroup + BlockDeviceMappings: + - DeviceName: /dev/sda1 + Ebs: + VolumeSize: !Ref RootVolumeSizeGiB + VolumeType: gp3 + Encrypted: true + DeleteOnTermination: true + Tags: + - Key: Name + Value: fastpath-host + - Key: fastpath:role + Value: controller + +Outputs: + InstanceId: + Description: ID of the Fastpath host EC2 instance. + Value: !Ref FastpathHost + PublicIp: + Description: Public IPv4 address for the Fastpath host (if applicable). + Value: !GetAtt FastpathHost.PublicIp + PrivateIp: + Description: Private IPv4 address for intra-cluster communication. + Value: !GetAtt FastpathHost.PrivateIp + SecurityGroupId: + Description: Security group attached to the Fastpath host. + Value: !Ref FastpathSecurityGroup +EOF + + {{< /tab >}} +{{< /tabpane >}} + +When the instance reports a `running` state, note the public and private IP addresses as FASTPATH_PUBLIC_IP and FASTPATH_PRIVATE_IP. You'll need these values later. + +## Install Fastpath Dependencies + +Repeat the dependency installation process so the *fastpath* host has the same toolchain and helper scripts as the build machine. + +1. SSH into the `c8g.4xlarge` *fastpath* host using the configured key pair. + +2. Open the [Install and Clone section](https://localhost:1313/install-guides/kernel-build/#install-and-clone) of the install guide from your workstation. + +3. Run each command from that section on the *fastpath* machine. It should be similar to the following (always refer to the above link for the latest command line): + + ```output + $ sudo apt update && sudo apt install -y git python3 python3-pip python3-venv build-essential bc rsync dwarves flex bison libssl-dev libelf-dev btop yq + + WARNING: apt does not have a stable CLI interface. Use with caution in scripts. + Hit:1 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble InRelease + Get:2 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble-updates InRelease [126 kB] + ... + 0 upgraded, 104 newly installed, 0 to remove and 43 not upgraded. + Setting up build-essential (12.10ubuntu1) ... + Setting up libssl-dev:arm64 (3.0.13-0ubuntu3.6) ... + Setting up python3-pip (24.0+dfsg-1ubuntu1.3) ... + + $ cd + $ git clone https://github.com/geremyCohen/arm_kernel_install_guide.git ~/arm_kernel_install_guide + $ cd ~/arm_kernel_install_guide && chmod +x scripts/*.sh + + Cloning into 'arm_kernel_install_guide'... + ``` + +## Copy kernels between build and Fastpath instances + +When we begin testing, the *fastpath* instance will push the compiled kernels to the SUT for testing. But as of now, the kernels are still on the build instance. This next step copies the kernels from the build instance to the new *fastpath* instance. + +1. Locate the value you recorded earlier for BUILD_PRIVATE_IP. + +2. On the *fastpath* instance, ```cd``` into the `arm_kernel_install_guide` folder you just cloned. + +3. Run the `pull_kernel_artifacts.sh` script, substituting BUILD_PRIVATE_IP with the private IP of the build instance: + + ```command + cd ~/arm_kernel_install_guide + ./scripts/pull_kernel_artifacts.sh --host 172.31.110.110 + ``` + + ```output + [2026-01-08 18:35:14] Pulling kernel artifacts: + [2026-01-08 18:35:14] Host : 172.31.110.110 + [2026-01-08 18:35:14] SSH user : ubuntu + [2026-01-08 18:35:14] Remote dir : /home/ubuntu/kernels + [2026-01-08 18:35:14] Local dir : /home/ubuntu/kernels + [2026-01-08 18:35:14] Versions : auto-detected + [2026-01-08 18:35:15] Copying 6.18.1-ubuntu+/Image.gz + [2026-01-08 18:35:16] Copying 6.18.1-ubuntu+/modules.tar.xz + [2026-01-08 18:35:18] Copying optional 6.18.1-ubuntu+/config.stock + [2026-01-08 18:35:18] Copying 6.19.0-rc1-ubuntu+/Image.gz + [2026-01-08 18:35:19] Copying 6.19.0-rc1-ubuntu+/modules.tar.xz + [2026-01-08 18:35:21] Copying optional 6.19.0-rc1-ubuntu+/config.stock + [2026-01-08 18:35:21] Artifact pull complete. + ``` + + +When the script completes, the *fastpath* host is ready with the kernels it needs for testing. + +## Power down the build machine + +After copying the artifacts from the build machine, stop (or terminate it) to avoid incurring additional costs. If you wish to keep it around for future kernel builds, stopping it is sufficient. + + +{{% notice Note %}} +If you do decide to keep the machine around as a kernel copy host, you can modify it to a smaller instance type such as `c8g.4xlarge` to save on costs when its running. The larger 24xlarge instance is only needed during kernel compilation. +{{% /notice %}} + +## Configure the Fastpath host + +With kernels copied over, the final step is to install and configure the *fastpath* software onto the *fastpath* host. From the same folder, run the host configuration script targeting localhost: + +1. Stay on the *fastpath* host and ensure you are in the cloned repository. If needed, you can easily navigate there again: + + ```command + cd ~/arm_kernel_install_guide + ``` + +2. Run the *fastpath* host setup script, targeting localhost (the current machine): + + ```command + ./scripts/configure_fastpath_host.sh --host localhost + ``` + + ```output + [2026-01-08 18:35:27] Configuring fastpath host localhost (non-interactive mode) + [2026-01-08 18:35:27] Installing prerequisites + ... + [2026-01-08 18:36:10] Fastpath host setup complete. + ``` + +Note that the script creates a Python virtual environment at `~/venv` and installs the *fastpath* CLI alongside its dependencies: + +```command +source ~/venv/bin/activate +which python +``` + +```output +/home/ubuntu/venv/bin/python +``` + +{{% notice Note %}} +Whenever you log back into the machine, make sure to activate the virtual environment (the above command line) before running any *fastpath* commands. +{{% /notice %}} + +With the *fastpath* host configured, you're now ready to provision the system under test (SUT) and verify connectivity between them. diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/generate_and_execute_plan.md b/content/learning-paths/servers-and-cloud-computing/fastpath/generate_and_execute_plan.md new file mode 100644 index 0000000000..7f59a4c1b0 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/generate_and_execute_plan.md @@ -0,0 +1,128 @@ +--- +title: "Generate and Execute the Benchmark Plan" +weight: 11 + +layout: "learningpathall" +--- + +With all required components in place, it's time to generate a *fastpath* benchmark plan, execute it, and analyze the results. + +## Generate the Fastpath plan + +*fastpath* uses a YAML-formatted [plan file](https://fastpath.docs.arm.com/en/latest/user-guide/planexec.html#introduction) to define the system under test (SUT), the kernels to deploy, and the benchmark workloads to run. + +This YAML file can be manually authored, but to simplify the process for this LP, a helper script is provided that gathers the required information and produces a valid `plan.yaml` file output. + + +1. Run the following on the *fastpath* instance to activate the *fastpath* virtual environment, and run the plan generator script. Have your SUT's private IP address handy, as you'll be prompted to enter it: + +```command +source ~/venv/bin/activate +cd ~/arm_kernel_install_guide +./scripts/generate_plan.sh +``` + +When complete, the script creates the plan, and provides a list of frequently performed tasks relative to the plan: + +```output +Enter SUT private IP: 172.31.100.19 +Plan name: + fastpath_test_010826-1837 + +Plan written to: + /home/ubuntu/arm_kernel_install_guide/plans/fastpath_test_010826-1837.yaml + +Run Fastpath with: + fastpath plan exec --output results/ /home/ubuntu/arm_kernel_install_guide/plans/fastpath_test_010826-1837.yaml + +After Fastpath run completes, gather results with: + fastpath result list results/ --object swprofile + +Relative results per kernel: + fastpath result show results/ --swprofile fp_6.19.0-rc1-ubuntu --relative + fastpath result show results/ --swprofile fp_6.18.1-ubuntu --relative + +Comparison between kernels: + fastpath result show results/ --swprofile fp_6.19.0-rc1-ubuntu --swprofile fp_6.18.1-ubuntu --relative +``` + + + +## Understand the Fastpath benchmark plan + +With the plan generated, you can now run *fastpath* to execute the benchmark workloads defined in the plan. + +Before you do, take a quick look at the generated plan file to see what kernels and workloads are defined: + +```console +cat plans/fastpath_test_010826-1837.yaml # your plan filename will be different, replace accordingly +``` + +```output +sut: + name: fastpath_test_010826-1837 + connection: + method: SSH + params: + user: fpuser + host: 172.31.100.19 +swprofiles: +- name: fp_6.19.0-rc1-ubuntu + kernel: /home/ubuntu/kernels/6.19.0-rc1-ubuntu+/Image.gz + modules: /home/ubuntu/kernels/6.19.0-rc1-ubuntu+/modules.tar.xz +- name: fp_6.18.1-ubuntu + kernel: /home/ubuntu/kernels/6.18.1-ubuntu+/Image.gz + modules: /home/ubuntu/kernels/6.18.1-ubuntu+/modules.tar.xz +benchmarks: +- include: speedometer/v2.1.yaml +defaults: + benchmark: + warmups: 1 + repeats: 2 + sessions: 2 + timeout: 1h +``` + +Within the YAML file, you can see the plan name (sut.name), and connection info (sut.connection) for the SUT. + +Under swprofiles, the two kernel variants built earlier are defined, along with their paths on the *fastpath* host (which will be pushed to the SUT during test runtime). + +Under benchmarks, the Speedometer v2.1 benchmark workload is specified to run against each kernel. + +The yaml also shows the two kernel variants to be tested (swprofiles), and the benchmark workload to run (benchmarks). + +For more information on the plan file format and options, see the [*fastpath* PlanExec User Guide](https://fastpath.docs.arm.com/en/latest/user-guide/planexec.html). + +For a list of current benchmarks supported by *fastpath*, see the [*fastpath* Benchmark User Guide](https://fastpath.docs.arm.com/en/latest/user-guide/planschema.html#benchmark-library). + + + +## Execute the Fastpath benchmark plan + +Time to run the benchmark! An example plan execution command line is given under the "Run Fastpath with:" section of the plan generator's output, similar to: + +```command +source ~/venv/bin/activate +~/fastpath/fastpath/fastpath plan exec --output results-fastpath_test_010826-1837 /home/ubuntu/arm_kernel_install_guide/plans/fastpath_test_010826-1837.yaml +``` + +Copy and paste that line, to begin the testing: + +```output +Executing fastpath_test_010826-1837.yaml... + 0%| | 0/12 [00:00 + EC2 setup +

+ +### Build Host + +The kernel build host is responsible for producing kernel artifacts. Using kernel‑guide scripts, it installs the required development environment and calls into [Tuxmake](https://tuxmake.org/) to build kernels in a consistent and repeatable way. + +### System Under Test (SUT) + +The SUT host is a blank slate where benchmark workloads actually run -- its the system you are testing performance for. + +The provided utility scripts make it easy to prepare the SUT by installing prerequisites such as Docker and the *fastpath* system account. + +### Fastpath Host + +The *fastpath* host brings it all together, acting as the control plane for benchmarking tasks. Provided utility scripts help you copy kernels from the build host to the *fastpath* host, enabling it to execute benchmarks on the SUT, and aggregate benchmark run results. + + +## Learning Path Structure + +Each chapter in this learning path corresponds to a step in the flow above. For each chapter, the same high‑level pattern applies: + +- Bring up the required cloud instance +- Run utility-script setup for that machine +- Use the machine for its intended role + +You do not need to understand every tool or dependency used in this LP. The goal is to understand how the pieces fit together and how each machine contributes to the overall benchmarking workflow. + +With that background, you're ready to begin setting up the build machine! diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/manage-resources.md b/content/learning-paths/servers-and-cloud-computing/fastpath/manage-resources.md new file mode 100644 index 0000000000..518e32ad66 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/manage-resources.md @@ -0,0 +1,15 @@ +--- +title: "Manage your AWS resources" + +weight: 17 + +layout: "learningpathall" +--- + +Benchmarks can run for hours, so its easy to forget about the AWS resources you are using for your *fastpath* lab. + +To avoid unexpected charges, stop the *fastpath* and SUT instances when you are not actively testing. + +Terminate instances once you no longer need them -- you may wish to archive the `results/` directory on your fastpath host to save results for later. + +Following these practices keeps your *fastpath* lab ready for future experiments without surprise expenses. diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/next-steps.md b/content/learning-paths/servers-and-cloud-computing/fastpath/next-steps.md new file mode 100644 index 0000000000..2e41e269a0 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/next-steps.md @@ -0,0 +1,22 @@ +--- +title: "Where to go next" + +weight: 18 + +layout: "learningpathall" +--- + +You now have a working *fastpath* benchmarking environment consisting of: + +- A build host that can compile custom Linux kernels with all *fastpath* dependencies. +- A reusable *fastpath* host which can generate and orchestrate benchmark plans, as well as analyze results. +- An SUT that can run the benchmark plan. + + +Continue exploring by revisiting the source documentation: + +- [*fastpath* documentation](https://fastpath.docs.arm.com/en/latest/index.html) for advanced deployment models, additional benchmarks, and troubleshooting tips. +- [Kernel Install Guide](/install-guides/kernel-build/) for deeper dives into kernel customization, *fastpath* tagging, and artifact management. +- [Tuxmake documentation](https://docs.tuxmake.org/en/latest/) to learn how to tailor build matrices, cross-compilation targets, and caching strategies that complement the *fastpath* workflow. + +Extend this learning path by experimenting with new kernel patches, expanding the benchmark suite, or automating the instance lifecycle with Terraform. diff --git a/content/learning-paths/servers-and-cloud-computing/fastpath/sut_setup.md b/content/learning-paths/servers-and-cloud-computing/fastpath/sut_setup.md new file mode 100644 index 0000000000..57d64ac6b5 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fastpath/sut_setup.md @@ -0,0 +1,241 @@ +--- +title: "Setup System Under Test Instance" + +weight: 10 + +layout: "learningpathall" +--- +Now that kernels are built and the *fastpath* host is ready, it's time to set up the system under test (SUT). + +## Provision the SUT + +The System Under Test (SUT) is the target machine where *fastpath* installs your kernels, runs benchmarks on each kernel (one at a time) and when complete, compares and displays the results via *fastpath*. + +Just like choosing the kernels to test, the instance type of the SUT depends on your use case. For this *fastpath* LP, we recommend a Graviton4 `c8g.12xlarge` instance with Ubuntu 24.04 LTS. This instance type provides a good balance of CPU and memory for a test benchmark. + +{{% notice Note %}} +The following steps involve launching an EC2 instance. You can perform all EC2 instance creation steps via the AWS Management Console instead or AWS CLI. For step-by-step instructions to bring up an EC2 instance via the console, consult the [Compute Service Provider learning path](/learning-paths/servers-and-cloud-computing/csp/) for detailed instructions. A tutorial from AWS is also available via [Get started with Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html). +{{% /notice %}} + +Create build host with the following specifications: + +1. **Name** — *fastpath-sut* +2. **Operating system** — *Ubuntu* +3. **AMI** — *Ubuntu 24.04 LTS (Arm)* +4. **Architecture** — *64-bit Arm* +5. **Instance type** — `c8g.12xlarge` +6. **Key pair** — *Select or create a key for SSH* +7. **Security group** — *allow SSH inbound from your IP and cluster peers* +8. **Storage** — *200 GB gp3* + + +Choose whichever provisioning method is most convenient—the tabs below provide the console settings, a CLI example, and the CloudFormation template (including the optional Fastpath security-group peer parameter). + +{{< tabpane >}} + {{< tab header="AWS Console" img_src="/learning-paths/servers-and-cloud-computing/fastpath/images/ec2_setup.png">}} + {{< /tab >}} + + {{< tab header="AWS CLI" language="shell">}} + +# Replace the placeholders with values from your account/environment +aws ec2 run-instances \ + --image-id resolve:ssm:/aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id \ + --instance-type c8g.12xlarge \ + --key-name \ + --subnet-id \ + --security-group-ids \ + --associate-public-ip-address \ + --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":200,"VolumeType":"gp3","DeleteOnTermination":true}}]' \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=fastpath-sut},{Key=fastpath:role,Value=sut}]' + {{< /tab >}} + + {{< tab header="CloudFormation" language="yaml">}} + +cat <<'EOF' > sut-host.yaml + +# README FIRST! +# +# 1. Click the copy icon to save this file to your clipboard. +# 2. Paste it into your terminal, it will be saved as `build-host.yaml` +# 3. Go to the CloudFormation console at https://us-east-1.console.aws.amazon.com/cloudformation/home +# 4. Click "Create stack -> With new resources (standard)" +# 5. Choose "Upload a template file" in the CloudFormation console. +# 6. Name the stack "fastpath-sut", enter remaining required values, then click "Submit" to create the stack. + +AWSTemplateFormatVersion: '2010-09-09' +Description: >- + Fastpath Learning Path - System Under Test (SUT) instance for benchmark execution. + +Parameters: + LatestUbuntuAmiId: + Type: 'AWS::SSM::Parameter::Value' + Default: /aws/service/canonical/ubuntu/server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id + Description: SSM parameter for the latest Ubuntu 24.04 LTS (Arm) AMI. + InstanceType: + Type: String + Default: c8g.12xlarge + AllowedValues: + - c8g.8xlarge + - c8g.12xlarge + - c8g.16xlarge + Description: Instance size for the SUT. + KeyPairName: + Type: AWS::EC2::KeyPair::KeyName + Description: Existing EC2 key pair to enable SSH access. + VpcId: + Type: AWS::EC2::VPC::Id + Description: VPC where the SUT will run. + SubnetId: + Type: AWS::EC2::Subnet::Id + Description: Subnet for the SUT (needs network reachability from the Fastpath host). + SSHAllowedCidr: + Type: String + Default: 0.0.0.0/0 + Description: CIDR block allowed to SSH into the SUT (replace with your IP/CIDR). + SutSecurityGroupId: + Type: String + Default: '' + Description: Security group ID of the Fastpath host to allow peer-to-peer SSH. + RootVolumeSizeGiB: + Type: Number + Default: 200 + MinValue: 100 + MaxValue: 1024 + Description: Size (GiB) of the gp3 root volume. + +Conditions: + HasFastpathPeer: !Not [ !Equals [ !Ref SutSecurityGroupId, '' ] ] + +Resources: + SutSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Enable SSH access for the SUT + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 22 + ToPort: 22 + CidrIp: !Ref SSHAllowedCidr + SecurityGroupEgress: + - IpProtocol: -1 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: fastpath-sut-sg + + SutPeerIngress: + Type: AWS::EC2::SecurityGroupIngress + Condition: HasFastpathPeer + Properties: + IpProtocol: tcp + FromPort: 22 + ToPort: 22 + SourceSecurityGroupId: !Ref SutSecurityGroupId + GroupId: !Ref SutSecurityGroup + + SutHost: + Type: AWS::EC2::Instance + Properties: + ImageId: !Ref LatestUbuntuAmiId + InstanceType: !Ref InstanceType + KeyName: !Ref KeyPairName + SubnetId: !Ref SubnetId + SecurityGroupIds: + - !Ref SutSecurityGroup + BlockDeviceMappings: + - DeviceName: /dev/sda1 + Ebs: + VolumeSize: !Ref RootVolumeSizeGiB + VolumeType: gp3 + Encrypted: true + DeleteOnTermination: true + Tags: + - Key: Name + Value: fastpath-sut + - Key: fastpath:role + Value: sut + +Outputs: + InstanceId: + Description: ID of the SUT EC2 instance. + Value: !Ref SutHost + PublicIp: + Description: Public IPv4 address for the SUT (if applicable). + Value: !GetAtt SutHost.PublicIp + PrivateIp: + Description: Private IPv4 address for intra-cluster communication. + Value: !GetAtt SutHost.PrivateIp + SecurityGroupId: + Description: Security group attached to the SUT. + Value: !Ref SutSecurityGroup +EOF + + {{< /tab >}} +{{< /tabpane >}} + +When the instance reports a `running` state, note the public and private IP addresses as SUT_PUBLIC_IP and SUT_PRIVATE_IP. You'll need these values later. + + +## Configure the SUT via the Fastpath instance + +You'll next run a script that remotely installs the *fastpath* software, and the required `fpuser` system account. It also sets up SSH access for the new `fpuser` account by copying over ubuntu@SUT's `~/.ssh/authorized_keys` file. + +{{% notice Note %}} +When communicating from the *fastpath* host to the SUT, use the SUT's private IP address. This will allow for much faster communication and file transfer. +{{% /notice %}} + +In this example, `44.201.174.17` is used as the *fastpath* host public IP, and `100.119.0.19` is used as the SUT's private IP. Replace with your own values: + +```command +# ssh into the fastpath host if you aren't already there. Note the use of agent forwarding -A +ssh -A -i ~/.ssh/gcohen1.pem ubuntu@44.201.174.17 # Replace with YOUR fastpath host public IP + +# Run the configuration script from within the fastpath virtual environment +source ~/venv/bin/activate # Activate the fastpath virtual environment + +# Make sure you are in the arm_kernel_install_guide repository +cd ~/arm_kernel_install_guide # Enter the helper scripts repository + +# Configure the SUT - replace the IP with your SUT's private IP +./scripts/configure_fastpath_sut.sh --host 172.31.100.19 # Replace with YOUR SUT private IP +``` + +```output +[2026-01-08 18:36:23] Configuring 172.31.100.19 as fastpath SUT (non-interactive mode) +[2026-01-08 18:36:23] Ensuring docker.io, btop, and yq are installed +Warning: Permanently added '172.31.100.19' (ED25519) to the list of known hosts. +Hit:1 http://us-east-1.ec2.ports.ubuntu.com/ubuntu-ports noble InRelease +... +[2026-01-08 18:36:47] Creating/updating fpuser +[2026-01-08 18:36:53] Testing SSH connectivity for fpuser +fpuser +[2026-01-08 18:36:54] Fastpath SUT configuration complete. +[2026-01-08 18:36:54] Note: ubuntu may need to re-login for docker group membership to take effect. +``` + + + +## Validate Fastpath connectivity +Once complete, ensure the *fastpath* host can properly ping the SUT with the following command: + +```command +source ~/venv/bin/activate +~/fastpath/fastpath/fastpath sut fingerprint --user fpuser 172.31.100.19 +``` + +```output +HW: + host_name: ip-172-31-100-19 + architecture: aarch64 + cpu_count: 48 + ... + product_name: c8g.12xlarge +SW: + kernel_name: 6.14.0-1018-aws + userspace_name: Ubuntu 24.04.3 LTS +``` + +A successful run prints hardware details for the SUT. If the command fails, verify security group rules and rerun the configuration script. If you are able to ssh into the SUT as `fpuser`, but the fingerprint command still fails, ensure that `docker.io` is installed on the SUT. + +With the SUT now configured, you're ready to move on to the next step: setting up and running a *fastpath* benchmark! Remember to stop (but not terminate) the build instance so that kernel artifacts remain available, and stop any *fastpath*/SUT instances when you are finished testing to avoid unnecessary spend.