Skip to content

refactor: minimize Python C API usage in tensor conversion - #892

Merged
voltjia merged 3 commits into
perf/optimize-generic-tensor-conversionfrom
refactor/minimize-python-c-api
Aug 7, 2026
Merged

refactor: minimize Python C API usage in tensor conversion#892
voltjia merged 3 commits into
perf/optimize-generic-tensor-conversionfrom
refactor/minimize-python-c-api

Conversation

@voltjia

@voltjia voltjia commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • represent interned attribute names as pybind11::handle instead of exposing raw PyObject * values throughout the conversion helpers
  • replace the custom PyObject_GetAttr wrapper with public pybind11::getattr
  • keep the Python C API fast paths that do not have a zero-overhead public pybind11 equivalent

Motivation

PR #832 gets most of its speedup from cached attribute names, fused no-argument method calls, and fast metadata conversion. This follow-up narrows direct Python C API usage without giving those gains back.

A broader prototype also replaced PySequence_Fast_GET_* and direct UTF-8 access with public pybind11 iteration/casters. It regressed the 16x16 GEMM B -> C comparison by 4.25% (95% CI: +1.43% to +6.56%), so this revision deliberately retains those measured hot paths. The remaining direct calls are limited to name interning, fused no-argument calls, integer extraction, fast sequence access, UTF-8 views, and their error handling.

Validation

Environment:

  • NVIDIA A100-SXM4-80GB, physical GPU 3 exposed as cuda:0
  • process pinned to CPU 24, with a 1 second CPU ramp before each timed case
  • Python 3.12.3, pybind11 2.13.6, Torch 2.10.0a0+nv25.12
  • InfiniRT 31b2a32c3e0b8eb2ef9da634c3ab934ce87e6af3
  • 5,000 warmup calls, 20,000 calls per round, 9 rounds per process
  • 6 balanced A/B/C process-order cycles; process median is the statistical unit
  • 200,000 paired bootstrap resamples for 95% confidence intervals

A is PR #832 base 2cfb526, B is PR #832 head 50f979df, and C is this branch.

Case A -> B B -> C A -> C
tensor conversion, 16x16 FP32 -35.94% [-42.00%, -31.57%] -0.42% [-2.77%, +1.98%] -36.21% [-42.23%, -31.92%]
Add, 13x4 FP32 -24.14% [-31.24%, -19.14%] +0.57% [-1.16%, +1.97%] -23.70% [-30.25%, -19.80%]
GEMM, 16x16 FP32 -18.69% [-26.30%, -13.99%] -0.77% [-2.21%, +0.82%] -19.31% [-27.31%, -14.00%]

The upper bound of the B -> C regression interval stays below the preselected +2% non-inferiority margin for all three cases.

Checks:

  • focused CPU/NVIDIA wheel build with abs,add,gemm
  • pytest tests/test_gemm.py -m smoke -q -n 1: 6 passed
  • ruff check tests/conftest.py
  • clang-format 21.1.8 --dry-run --Werror src/pybind11_utils.h
  • git diff --check

@voltjia
voltjia marked this pull request as ready for review August 7, 2026 03:37
@voltjia
voltjia requested a review from baominghelly August 7, 2026 03:37
@voltjia
voltjia merged commit af5d19e into perf/optimize-generic-tensor-conversion Aug 7, 2026
6 checks passed
@voltjia
voltjia deleted the refactor/minimize-python-c-api branch August 7, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants