diff --git a/tools/pgo/README.md b/tools/pgo/README.md index 9277315508ca..234f20e638ac 100644 --- a/tools/pgo/README.md +++ b/tools/pgo/README.md @@ -1,7 +1,6 @@ # Node.js PGO Training Scripts -Training workloads for Profile-Guided Optimization (PGO) builds using -Clang/LLVM (including Clang-CL on Windows). +Training workloads for Profile-Guided Optimization (PGO) builds. ## What is PGO? @@ -10,21 +9,36 @@ branch prediction, code layout), typically improving throughput by 5-20%. The process has three phases: -1. **Instrument** — Build with `-fprofile-generate` (produces `.profraw` files) +1. **Instrument** — Build with `-fprofile-generate` 2. **Train** — Run representative workloads to collect profile data -3. **Optimize** — Merge `.profraw` → `node.profdata` via `llvm-profdata`, - then rebuild with `-fprofile-use` +3. **Optimize** — Rebuild with `-fprofile-use` -## Quick Start +## Platform Support -From a VS Developer Command Prompt: +| Platform | Supported toolchains | Driver | +| -------- | -------------------- | ------------------------- | +| Windows | Clang-CL | `vcbuild.bat` + `pgo.ps1` | +| Linux | GCC | `configure` + `make` | +| macOS | — | — | + +The two supported flows differ in how profile data is collected. Clang writes +one `.profraw` file per process, which must be merged into a single +`.profdata` before the optimize phase. GCC's libgcov instead merges counters +into `.gcda` files next to each object file as each process exits, so there is +no merge step. + +Clang on Linux and macOS are not supported yet. + +## Quick Start: Windows + +From a VS Developer Command Prompt, at the repo root: ```powershell # Step 1: Build the instrumented binary vcbuild.bat pgo-generate -# Step 2: Run workloads and merge profile data -.\pgo.ps1 +# Step 2: Run workloads to collect profile data +powershell -ExecutionPolicy Bypass -File .\tools\pgo\pgo.ps1 # Step 3: Build the optimized binary vcbuild.bat pgo-use @@ -33,16 +47,51 @@ vcbuild.bat pgo-use `pgo.ps1` expects the instrumented binary at `Release\node.exe` (produced by step 1) and writes `node.profdata` to the repo root (consumed by step 3). +The script is unsigned, so the default execution policy refuses to run it +without `-ExecutionPolicy Bypass`. Use `pwsh` in place of `powershell` on +PowerShell 7. + ```powershell # Optionally set a longer training duration (default: 15s per script) -.\pgo.ps1 -Duration 30 +powershell -ExecutionPolicy Bypass -File .\tools\pgo\pgo.ps1 -Duration 30 +``` + +## Quick Start: Linux + +```bash +# Step 1: Build the instrumented binary +./configure --enable-pgo-generate +make + +# Step 2: Run workloads to collect profile data +./out/Release/node tools/pgo/pgo-run-all.js --duration=15 --verbose + +# Step 3: Build the optimized binary +./configure --enable-pgo-use +make ``` +Step 2 needs no driver script. Each object file gets one counter file beside +it, with the same basename and a `.gcda` extension: + +```text +out/Release/obj/src/node_base.node_binding.o # from step 1 +out/Release/obj/src/node_base.node_binding.gcda # from step 2 +``` + +Keep `out/` intact between steps 1 and 3. GCC records the `.gcda` path into +each object at compile time, so `make clean` or `make distclean` discards the +training data and step 3 silently produces an ordinary build. + +The build passes `-fprofile-correction`, which is required here. Counter +updates from the worker threads and the libuv thread pool race with each +other, and GCC treats the resulting inconsistent profile as an error unless +told to smooth it out. + ## Training Scripts All scripts use only Node.js built-in modules (no npm dependencies). -Each script is run as a separate process via `fork()`, producing its own -`.profraw` file. +Each script is run as a separate process via `fork()`. | Script | What it exercises | | ------------------------ | ------------------------------------------------------------- | @@ -65,13 +114,13 @@ workloads). When used with `pgo.ps1`, this is handled automatically. ```bash # Run all scripts -node tools/pgo/pgo-run-all.js --duration=15 --verbose +./out/Release/node tools/pgo/pgo-run-all.js --duration=15 --verbose # Run specific scripts -node tools/pgo/pgo-run-all.js --scripts=http-server,json,crypto --duration=30 +./out/Release/node tools/pgo/pgo-run-all.js --scripts=http-server,json,crypto --duration=30 # Show help -node tools/pgo/pgo-run-all.js --help +./out/Release/node tools/pgo/pgo-run-all.js --help ``` Each script reads the `PGO_TRAINING_DURATION` environment variable (in @@ -82,6 +131,7 @@ automatically from the `--duration` flag (in seconds). ``` tools/pgo/ +├── pgo.ps1 # Windows training driver (collect + merge) ├── pgo-run-all.js # Training orchestrator ├── pgo-http-server.js # HTTP server + client workload ├── pgo-json.js # JSON parse/stringify workload diff --git a/pgo.ps1 b/tools/pgo/pgo.ps1 similarity index 88% rename from pgo.ps1 rename to tools/pgo/pgo.ps1 index 39f5352463e3..1cfedddd9e69 100644 --- a/pgo.ps1 +++ b/tools/pgo/pgo.ps1 @@ -4,9 +4,12 @@ # (Release\node.exe) and merges the resulting .profraw files into # node.profdata for use with -fprofile-use. # -# Usage (from a VS Developer Command Prompt): -# .\pgo.ps1 # Run workloads (15s each) and merge -# .\pgo.ps1 -Duration 30 # Run workloads (30s each) and merge +# Usage (from a VS Developer Command Prompt, at the repo root): +# powershell -ExecutionPolicy Bypass -File .\tools\pgo\pgo.ps1 +# powershell -ExecutionPolicy Bypass -File .\tools\pgo\pgo.ps1 -Duration 30 +# +# The script is unsigned, so the default execution policy blocks it without +# -ExecutionPolicy Bypass. Default duration is 15s per workload. # # Prerequisites: # - Release\node.exe must be an instrumented build (built with pgo-generate) @@ -22,6 +25,10 @@ param( Set-StrictMode -Version Latest $ErrorActionPreference = 'Stop' +# The instrumented binary and the merged profile both live at the repo root, +# two levels up from tools\pgo. common.gypi reads node.profdata from there. +$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..")).Path + # --------------------------------------------------------------------------- # Locate llvm-profdata shipped with Visual Studio's LLVM toolset # --------------------------------------------------------------------------- @@ -64,13 +71,13 @@ function Find-LlvmProfdata { # Validate prerequisites # --------------------------------------------------------------------------- -$instrumentedNode = Join-Path $PSScriptRoot "Release\node.exe" +$instrumentedNode = Join-Path $repoRoot "Release\node.exe" if (-not (Test-Path $instrumentedNode)) { Write-Error "Instrumented binary not found: $instrumentedNode`nBuild with: vcbuild.bat pgo-generate" exit 1 } -$pgoRunAll = Join-Path $PSScriptRoot "tools\pgo\pgo-run-all.js" +$pgoRunAll = Join-Path $PSScriptRoot "pgo-run-all.js" if (-not (Test-Path $pgoRunAll)) { Write-Error "PGO training script not found: $pgoRunAll" exit 1 @@ -90,7 +97,7 @@ Write-Host "`n=== STEP 1: Collect PGO profiles ===" -ForegroundColor Cyan # Directory that will receive .profraw files from the instrumented binary. # %p (PID) and %m (module hash) keep concurrent/fork'd processes from colliding. -$profileDir = Join-Path $PSScriptRoot "pgo-profiles" +$profileDir = Join-Path $repoRoot "pgo-profiles" if (Test-Path $profileDir) { Remove-Item -Recurse -Force $profileDir @@ -137,7 +144,7 @@ $totalSize = ($profrawFiles | Measure-Object -Property Length -Sum).Sum $totalSizeMB = [math]::Round($totalSize / 1MB, 1) Write-Host "Found $($profrawFiles.Count) .profraw file(s), ${totalSizeMB} MB total" -$profdata = Join-Path $PSScriptRoot "node.profdata" +$profdata = Join-Path $repoRoot "node.profdata" $mergeArgs = @("merge", "--output=$profdata") + ($profrawFiles | Select-Object -ExpandProperty FullName) $mergeStopwatch = [System.Diagnostics.Stopwatch]::StartNew()