Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# CUDA Compatibility Tests

The compatibility adapter compiles and runs CUDA Samples on NVIDIA and
CUDA-compatible accelerator toolchains. It reports compilation, execution,
failure, and waived-sample counts without treating a partial pass rate as an
adapter execution error.

## Test Input

```json
{
"run_id": "cuda_samples.nvidia.quick",
"testcase": "compatibility.CudaSamples.PassRate",
"config": {
"platform": "nvidia",
"cuda_samples_dir": "/workspace/cuda-samples",
"build_system": "make",
"sample_filter": ["vectorAdd", "matrixMul", "clock"],
"timeout_per_sample": 180,
"jobs": 4
}
}
```

`cuda_samples_dir` must contain a `Samples` directory. The adapter recursively
discovers `Samples/<category>/<sample>` directories containing a Makefile or a
standalone CMake project. CMake grouping manifests that only aggregate child
directories are excluded. A requested sample name that is not found, or an
empty `sample_filter`, is a configuration error instead of a successful
zero-sample result.

The InfiniPerf cuda-samples submodule pins the CMake-based `master` revision.
Its `batch_test` branch provides the Makefiles used by the original InfiniPerf
compatibility workflow. The adapter supports both layouts; use `build_system`
to select `cmake`, `make`, or the default `auto` detection.

CMake samples are configured through a temporary wrapper project. The wrapper
sets `CUDA_ARCHITECTURES` on every generated target after the sample manifest
has been evaluated, so manifests that set their own default architecture list
cannot override the requested `sms` value.

## Platform Toolchains

Platform aliases and compiler candidates are shared with the hardware adapter
through `infinimetrics.hardware.constants`. Compatibility-only architecture
values and Make arguments remain in `infinimetrics.compatibility.constants`.
This keeps hardware detection and compiler discovery consistent without
coupling CUDA Samples to the hardware benchmark build script.

| Platform | Default compiler | Default architecture |
| --- | --- | --- |
| NVIDIA | `nvcc` | `80` |
| MetaX | `cucc` (falls back to `mxcc`) | `70` |
| Iluvatar CoreX | `/usr/local/corex/bin/clang++` | `ivcore20` |

The supported canonical platform names are `cuda`, `metax`, and `corex`.
The existing aliases `nvidia` and `iluvatar` are also accepted.

Set `compiler`, `sms`, or `make_args` in the input when the installed vendor
SDK uses a wrapper or different target. Arguments are passed directly as an
argument list; shell expansion is not performed. The resolved default compiler
and architecture are added to the result config when they were not explicit in
the input. For MetaX, the adapter also infers `MACA_PATH` from the resolved
`cucc` or `mxcc` location when the variable is unset. An explicit `MACA_PATH`
is preserved.

For non-NVIDIA Makefile builds, the default arguments remove NVIDIA-only
`--threads`, `-gencode`, and `-m64` flags. Platform support is declared only
after its compile and runtime workflow has been validated on target hardware.

## Metrics

- `compile_passed` and `compile_failed` cover all discovered samples.
- `run_passed` and `run_failed` cover samples that produced an executable.
- `run_skipped` counts CUDA Samples that explicitly return a waived result.
- A sample that does not run because compilation failed has `run_result:
"not_run"` and is not included in `run_skipped`.
- `run_pass_rate` keeps the original end-to-end definition: run passes divided
by all selected samples.
- `details` records each sample path and the final compiler or runtime error.

`result_code: 0` means the compatibility test completed and produced valid
measurements. It does not mean every sample passed; use the pass-rate metrics
for that decision.

Generate a readable report and optional CSV with:

```bash
python scripts/generate_compat_report.py result.json --csv result.csv
```
1 change: 1 addition & 0 deletions infinimetrics/common/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ class TestCategory(str, Enum):
INFER = "infer"
COMM = "comm"
TRAIN = "train"
COMPATIBILITY = "compatibility"


# Valid test categories (derived from TestCategory enum)
Expand Down
2 changes: 2 additions & 0 deletions infinimetrics/compatibility/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/usr/bin/env python3
"""Compatibility testing module for CUDA Samples compilation and execution."""
Loading
Loading