perf!: inline TensorView metadata storage

[voltjia]

Summary

• Replace the public TensorView::Shape and TensorView::Strides std::vector aliases with bounded SmallVector inputs and store both arrays in one ShapeStridesStorage owner. Ranks 0 through 8 stay inline; rank 9 and above use owned heap fallback.
• Return typed borrowed contiguous views from lvalue shape() / strides() accessors while preserving owning results for rvalues and common vector-like construction, including (count, value) metadata construction.
• Add unit, allocation, installed-consumer, sanitizer, footprint, and cache-key benchmarks, plus API, ABI, compatibility, and downstream migration documentation.

Motivation

PR #33 removed redundant TensorView metadata copies, but each ordinary non-empty tensor still required one heap allocation for shape and another for strides. TensorView is passed through framework-neutral operator and cache-key paths, so those allocations are significant relative to the work performed by this thin metadata object.

The selected capacity-8 combined representation removes metadata allocations for the rank range used by most deep-learning tensors while retaining owned fallback for larger ranks. The benchmark section below records the explicit object-footprint and rank-4 trade-off.

No issue is closed by this PR; this is the next optimization stage after #33.

Type of Change

• feat - new feature / new backend capability / new public API
• fix - bug fix
• perf - performance improvement without behavior change
• refactor - code restructuring without behavior change
• test - adding or fixing tests only
• docs - documentation only
• build / ci - build system or CI configuration
• chore - tooling, formatting, or other non-code changes
• Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

• CPU (WITH_CPU)
• NVIDIA (WITH_NVIDIA)
• Iluvatar (WITH_ILUVATAR)
• Hygon (WITH_HYGON)
• MetaX (WITH_METAX)
• Moore (WITH_MOORE)
• Cambricon (WITH_CAMBRICON)
• Ascend (WITH_ASCEND)
• Build system / CMake / generated headers
• Public headers / installed consumer API
• Documentation only

Smoke Build and Test Result

Final commit: 3a4bb7a91fa291876c5886b119ff894f42e96c30. Full validation commit: 425c770457ce590917a30549a2284a8889ad47bd. The final review follow-up only moves the shared noinline macro into the existing performance-test helper.

Environment: accelerator-dev/nvidia:latest (sha256:dd94fce2f83a180e2271f02f69713de5360aaf916b01e1f7f54173519fd54efd), GCC 13.3.0, CMake 3.31.6, Python 3.12.3. NVIDIA validation used an 8 x A100-SXM4-80GB host, driver 580.105.08, and CUDA 13.1.80.

# CPU
cmake -S . -B build-cpu \
  -DCMAKE_BUILD_TYPE=Release \
  -DAUTO_DETECT_DEVICES=OFF \
  -DWITH_CPU=ON \
  -DWITH_NVIDIA=OFF \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DINFINI_RT_BUILD_PERFORMANCE_TESTING=ON
cmake --build build-cpu --parallel 8
ctest --test-dir build-cpu --output-on-failure

100% tests passed, 0 tests failed out of 14
Total Test time (real) = 4.82 sec

# NVIDIA
cmake -S . -B build-nvidia \
  -DCMAKE_BUILD_TYPE=Release \
  -DAUTO_DETECT_DEVICES=OFF \
  -DWITH_CPU=ON \
  -DWITH_NVIDIA=ON \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DINFINI_RT_BUILD_PERFORMANCE_TESTING=ON
cmake --build build-nvidia --parallel 8
ctest --test-dir build-nvidia --output-on-failure

100% tests passed, 0 tests failed out of 16
Total Test time (real) = 212.05 sec

Both suites include test_install_consumer; the build directories were reconfigured at the validation SHA so generated public headers were not reused from an earlier checkout.

Test Results on Supported Platforms

Platform	Affected	Build / Smoke Result	Full Result / Notes
CPU	Yes	passed	full passed, 14/14 at validation SHA
NVIDIA	Yes	passed	full passed, 16/16 at validation SHA, including runtime and graph tests
Iluvatar	Yes	not run	SDK and runner unavailable; backend-owner validation requested
Hygon	Yes	not run	SDK and runner unavailable; backend-owner validation requested
MetaX	Yes	not run	SDK and runner unavailable; backend-owner validation requested
Moore	Yes	not run	SDK and runner unavailable; backend-owner validation requested
Cambricon	Yes	not run	SDK and runner unavailable; backend-owner validation requested
Ascend	Yes	not run	SDK and runner unavailable; backend-owner validation requested

The repository has no CODEOWNERS mapping from which to identify individual backend reviewers. T-Head is also affected through the shared public type but is not represented in the current PR template; no T-Head SDK/runner was available. Please assign maintainers with access to the untested backends.

Full `ctest` summaries

# CPU
14/14 Test #14: perf_tensor_view_footprint ....... Passed 0.66 sec
100% tests passed, 0 tests failed out of 14
Label Time Summary:
performance = 3.36 sec*proc (4 tests)
Total Test time (real) = 4.82 sec

# NVIDIA
14/16 Test #14: perf_memory ...................... Passed 197.50 sec
15/16 Test #15: perf_tensor_view ................. Passed   1.37 sec
16/16 Test #16: perf_tensor_view_footprint ....... Passed   0.67 sec
100% tests passed, 0 tests failed out of 16
Label Time Summary:
performance = 205.83 sec*proc (4 tests)
Total Test time (real) = 212.05 sec

Additional checks:

• clang-format 21.1.2 --dry-run --Werror: passed in GitHub Actions for both the branch push and the pull-request synthetic merge at the final SHA.
• git diff --check origin/master...HEAD: passed at the final SHA.
• Review follow-up at 3a4bb7a: remote CPU and NVIDIA builds of perf_tensor_view and perf_tensor_view_footprint passed 2/2 in each configuration.
• Fresh generated and installed trees contain detail/common/shape_strides_storage.h, do not contain the obsolete detail/common/tensor_metadata.h, and generate tensor_view.h with the new include path; test_install_consumer passed on CPU and NVIDIA.
• ASan + UBSan with leak detection: test_core, the storage test now named test_shape_strides_storage, and the fill-construction regression passed on patch-equivalent pre-rename commits. The later rename, accessor-inlining, and style-only commits do not change storage ownership or destruction logic.

Benchmark / Performance Impact

Harness: perf_tensor_view_footprint.cache_key_build_hit. Each sample hashes each input, copies it into a temporary non-reserved vector, compares against a prebuilt key, and destroys it. Each key processes 262,144 TensorView visits. The results are medians from 15 paired round-robin process groups with seven in-process samples per key, pinned to CPU 0 on an AMD Ryzen 9 8945HS under WSL2, GCC 13.3.0, C++17, -O3 -DNDEBUG.

Rank / tensor count	cap8 vs. vector baseline	cap8 vs. cap4
4 / 8	-36.887% (-180.459 ns)	+7.275% (+21.826 ns)
4 / 256	-60.816% (-12,733.150 ns)	-1.867% (-163.378 ns)
8 / 8	-26.603% (-139.257 ns)	-17.234% (-76.338 ns)
8 / 256	-52.062% (-12,494.684 ns)	-33.757% (-5,621.646 ns)

Layout changes from sizeof(TensorView) == 72 bytes with std::vector to 160 bytes with combined capacity 8 (Shape / Strides: 24 to 80 bytes). Capacity 8 wins every measured key against the shipping vector baseline and both rank-8 keys against capacity 4. The known trade-off is the small rank-4 / count-8 key, where cap8 is 7.275% slower than cap4 while still 36.887% faster than baseline.

Measurements were captured at pre-rebase commit 7eb07233d9767e127f8614a7a22174607212dc2b. git range-diff maps that patch unchanged to 8ba2501 in the rebased branch; d113995 adds the downstream-required (count, value) constructor and regression coverage, b5f5154 applies CI-equivalent formatting, 2d94f2e removes process documents, 4cdfb51 renames the internal shape/strides owner and its files, 181474c inlines metadata accessors, 425c770 aligns the code with repository style, and 3a4bb7a moves the duplicated noinline macro into perf_common.h. These follow-up commits do not change the selected capacity or object layout.

Notes for Reviewers

• This is an intentional source and ABI compatibility break. Shape and Strides are no longer exact std::vector aliases, TensorView layout changes, and lvalue accessors return borrowed views by value. Consumers must rebuild against matching InfiniRT headers and libinfinirt.so.
• Borrowed accessor results must not outlive their TensorView. Rvalue accessors return owning metadata to avoid dangling from temporaries.
• Rank 0 through 8 is allocation-free. Rank 9 and above remains supported via owned heap fallback; it is a correctness path, not an inline-performance target.
• No project version or SOVERSION changes are included because InfiniRT has not had a formal release.
• InfiniOps current master 01f63f4beaa1ca0e178c9b728c1706a4065657cf requires explicit owning metadata at five exact-std::vector / pybind boundaries. With that separate downstream patch, InfiniRT 1824269fc72a69d9a6276a28af41034661960588 built and installed into a self-contained consumer prefix, and the official CPU smoke command reports 51 passed, 7 skipped, 3991 deselected, including both C++ installed-consumer tests. The smoke run uses a recorded compiler wrapper only to demote existing deprecated generated-header warnings from -Werror.
• torch-infini current master 17743133d674fdb1c2569d96d0f2af3be6517d6f requires a test-only borrowed-view-to-vector conversion. Current InfiniRT master also added T-Head after that torch-infini pin, so a separate T-Head capability-policy update is required before the downstream wheel can build. With those two independent downstream adaptations, the wheel built against InfiniRT 1824269fc72a69d9a6276a28af41034661960588 (SHA-256 44b08cf0731221bfdb5d72cb7259753add25d22f525d0398d521245a2bf53dc5), imported outside the source tree, and its focused suite reported 29 passed, 2 skipped. readelf -d shows libinfiniops.so and libinfinirt.so as DT_NEEDED entries and no source/build RPATH.
• The downstream patches are intentionally not part of this repository. They should land in their owning repositories before those projects update their InfiniRT revision.