add builder README.md and cleanup script

.github/workflows/build-test.yml

@@ -23,7 +23,7 @@ jobs:
         uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0
         with:
           path: tools-cache
-          key: perfspect-tools-binaries-${{ hashFiles('tools/Makefile', 'tools/**/*.Dockerfile', 'tools/**/*.patch') }}
+          key: perfspect-tools-binaries-${{ hashFiles('tools/**') }}
       - name: Set cache hit flag
         if: steps.cache-tools.outputs.cache-hit == 'true'
         run: |

builder/README.md

@@ -0,0 +1,167 @@
+# PerfSpect Build System
+
+This directory contains the Docker-based build system for PerfSpect, which creates reproducible builds of the PerfSpect distribution packages for both x86_64 and aarch64 architectures.
+
+## Overview
+
+The build system uses a multi-stage approach:
+
+1. **Tools Image** (`perfspect-tools:v1`): Contains pre-compiled external tools (perf, fio, etc.)
+2. **Builder Image** (`perfspect-builder:v1`): Contains Go build environment with pre-installed dependencies
+3. **Build Container**: Executes the actual build using the builder image
+
+## Quick Start
+
+```bash
+# Build from the repository root
+./builder/build.sh
+
+# Force rebuild of tools (skip cache)
+SKIP_TOOLS_CACHE=1 ./builder/build.sh
+```
+
+## Build Artifacts
+
+The build produces the following artifacts in the `dist/` directory:
+
+- `perfspect.tgz` - x86_64 distribution package
+- `perfspect.tgz.md5.txt` - MD5 checksum for x86_64
+- `perfspect-aarch64.tgz` - aarch64 distribution package
+- `perfspect-aarch64.tgz.md5.txt` - MD5 checksum for aarch64
+- `manifest.json` - Build metadata (version, commit, date)
+- `oss_source.tgz` - Open source packages used in tools
+
+## Tools Binary Caching
+The build system uses caching to optimize build times.
+
+**Purpose:** Skip expensive C compilation of external tools (perf, fio, etc.)
+
+**Cache Location:**
+- **Local:** `tools-cache/` directory (gitignored)
+- **GitHub Actions:** GitHub's remote cache storage
+
+**How It Works:**
+
+### Local Development
+
+**First Build:**
+```
+1. tools-cache/ doesn't exist
+2. Build tools from source using tools/build.Dockerfile (~5-10 minutes)
+   - Compiles perf from Linux kernel source
+   - Compiles fio benchmark tool
+   - Cross-compiles for both x86_64 and aarch64
+3. Extract compiled binaries to tools-cache/
+4. Build continues with perfspect-builder image
+```
+
+**Subsequent Builds:**
+```
+1. tools-cache/ exists and contains required files
+2. Create minimal perfspect-tools:v1 image by copying from tools-cache/ (~5 seconds)
+3. Skip building from source entirely
+4. Build continues with perfspect-builder image
+```
+
+**Cache Invalidation:**
+- Manual: `SKIP_TOOLS_CACHE=1 ./builder/build.sh`
+- Manual: `make clean-tools-cache`
+- Automatic: If required files are missing, cache is invalidated
+
+### GitHub Actions
+
+**First Run (cache miss):**
+```
+1. actions/cache attempts to restore tools-cache/ → MISS
+2. Build tools from source using docker buildx with GHA layer cache
+3. Extract binaries to tools-cache/
+4. actions/cache automatically saves tools-cache/ to GitHub's cache
+```
+
+**Subsequent Runs (cache hit):**
+```
+1. actions/cache restores tools-cache/ from GitHub's cache → HIT
+2. Workflow sets TOOLS_CACHE_HIT=true
+3. Create minimal perfspect-tools:v1 image from cache
+4. Skip building from source entirely
+```
+
+**Cache Invalidation:**
+- Automatic: Cache key is based on hash of all files in the tools directory:
+  ```
+  key: perfspect-tools-binaries-${{ hashFiles('tools/**') }}
+  ```
+- When any of these files change, cache key changes and cache misses
+
+## Build Flow Diagram
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    builder/build.sh                         │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+        ┌───────────────────────────────────┐
+        │  Check tools-cache/ exists?       │
+        └───────────────┬───────────────────┘
+                        │
+           ┌────────────┴────────────┐
+           │                         │
+          YES                       NO
+           │                         │
+           ▼                         ▼
+    ┌─────────────┐          ┌──────────────┐
+    │ Use cached  │          │ Build tools  │
+    │ binaries    │          │ from source  │
+    │             │          │              │
+    │ COPY from   │          │ Docker build │
+    │ tools-cache/│          │ (5-10 min)   │
+    │ (~5 sec)    │          │              │
+    └─────┬───────┘          └──────┬───────┘
+          │                         │
+          │                         ▼
+          │                  ┌──────────────┐
+          │                  │ Extract bins │
+          │                  │ to tools-    │
+          │                  │ cache/       │
+          │                  └──────┬───────┘
+          │                         │
+          └─────────┬───────────────┘
+                    ▼
+         ┌─────────────────────┐
+         │ perfspect-tools:v1  │
+         │ (Docker image)      │
+         └──────────┬──────────┘
+                    ▼
+         ┌─────────────────────┐
+         │ Build perfspect-    │
+         │ builder:v1          │
+         │                     │
+         │ - Copy tools from   │
+         │   perfspect-tools   │
+         └──────────┬──────────┘
+                    ▼
+         ┌─────────────────────┐
+         │ Run build container │
+         │                     │
+         │ docker run          │
+         │   perfspect-builder │
+         │   make dist         │
+         └──────────┬──────────┘
+                    ▼
+              ┌──────────┐
+              │ dist/    │
+              │ artifacts│
+              └──────────┘
+```
+
+## Files
+
+- `build.sh` - Main build orchestration script
+- `build.Dockerfile` - Defines the perfspect-builder image with Go environment
+
+## Environment Variables
+
+- `TOOLS_CACHE_HIT=true` - Set by GitHub Actions when tools cache is restored
+- `SKIP_TOOLS_CACHE=1` - Force rebuild tools from source, skip cache
+- `GITHUB_ACTIONS` - Automatically set in GitHub Actions, enables buildx with GHA cache

builder/build.sh

@@ -40,38 +40,25 @@ invalidate_cache() {
     fi
 }
 
-# Determine if we're in GitHub Actions
-if [ -n "$GITHUB_ACTIONS" ]; then
-    # Use buildx with GitHub Actions cache
-    CACHE_FROM="--cache-from type=gha,scope=perfspect-tools"
-    CACHE_TO="--cache-to type=gha,mode=max,scope=perfspect-tools"
-    BUILD_CMD="docker buildx build --load"
-else
-    # Local build without cache export
-    CACHE_FROM=""
-    CACHE_TO=""
-    BUILD_CMD="docker build"
-fi
-
-# Check if we can use cached binaries
-USE_CACHE=""
-if [ "$TOOLS_CACHE_HIT" = "true" ]; then
+# Check if we can use cached tools binaries
+USE_TOOLS_CACHE=""
+if [ "$TOOLS_CACHE_HIT" = "true" ]; then # GitHub Actions will set this env var on cache hit
     # GitHub Actions cache hit
-    USE_CACHE="true"
+    USE_TOOLS_CACHE="true"
 elif [ -z "$SKIP_TOOLS_CACHE" ] && [ -d "$CACHE_DIR/bin" ]; then
     # Local cache exists and not disabled
     echo "Found local tools cache in $CACHE_DIR/"
     echo "To force rebuild, run: SKIP_TOOLS_CACHE=1 builder/build.sh"
     if cache_ready; then
-        USE_CACHE="true"
+        USE_TOOLS_CACHE="true"
     else
         echo "Local tools cache is incomplete; it will be discarded and rebuilt."
         invalidate_cache
     fi
 fi
 
-# build tools image (or use cached binaries)
-if [ "$USE_CACHE" = "true" ]; then
+# build tools image (or use cached tools binaries)
+if [ "$USE_TOOLS_CACHE" = "true" ]; then
     echo "Using cached tool binaries, creating minimal tools image"
     # Create a minimal Dockerfile that packages the cached binaries
     cat > /tmp/cached-tools.Dockerfile << EOF
@@ -84,11 +71,14 @@ EOF
     rm /tmp/cached-tools.Dockerfile
 else
     echo "Building tools from source"
-    $BUILD_CMD -f tools/build.Dockerfile \
-        $CACHE_FROM $CACHE_TO \
-        --tag perfspect-tools:$TAG ./tools
+    if [ -n "$GITHUB_ACTIONS" ]; then
+        # Use buildx with GitHub Actions cache
+        docker buildx build --load -f tools/build.Dockerfile --cache-from type=gha,scope=perfspect-tools --cache-to type=gha,mode=max,scope=perfspect-tools --tag perfspect-tools:$TAG ./tools
+    else
+        docker build -f tools/build.Dockerfile --tag perfspect-tools:$TAG ./tools
+    fi
 
-    # Extract binaries for caching (both GitHub Actions and local)
+    # Extract binaries for caching
     if [ -z "$SKIP_TOOLS_CACHE" ]; then
         echo "Extracting tool binaries to tools-cache/ for future builds"
         rm -rf "$CACHE_DIR"