Skip to content

Use stable API python 310 - #4453

Open
aymanhab wants to merge 14 commits into
mainfrom
limited_python_310
Open

aymanhab wants to merge 14 commits into
mainfrom
limited_python_310

Conversation

@aymanhab

@aymanhab aymanhab commented Sep 15, 2026

Copy link
Copy Markdown
Member

Fixes issue #0

Brief summary of changes

This PR tells swig to build against the 3.10 python and use only the stable API for 3.10 so that the bindings can be used in any version 3.10+

Testing I've completed

Looking for feedback on...

CHANGELOG.md (choose one)

  • no need to update because...
  • updated.

This change is Reviewable

…n python3 rather than version specific library need to test wheel packaging and cross-platform
@adamkewley

Copy link
Copy Markdown
Contributor

@aymanhab if you are looking for some additional tests/checks for the stable ABI stuff then it might be useful to know what I ended up doing for OPynSim, which has strict dependency/symbol/stableABI checks. I have some scripts that specifically check the binaries that pop out of the build right before they are packaged into wheels.

E.g. anything with check_ in this directory is a post-build binary check, rather than a unit test:

The pertinent thing for stable ABI is to make sure the executable links to an unversioned python3.dll on windows (not 3.12 or something):

UNIXes don't necessarily need to link to Python iirc. It just needs to ensured that they expose an appropriate PyInit symbol that the Python interpreter can pick up at import time. In my case, i shake out all symbols in the Python binaries apart from the Init one so that there's zero chance of a symbol collision when loading OPynSim:

There's also checks in there for macOS framework version and libstdc++ symbol versions. They're less about stable ABI and more about ensuring the binaries install on older versions of Linux etc.

@aymanhab

Copy link
Copy Markdown
Member Author

Thanks for the pointers @adamkewley much appreciated 👍

The linking to python3 was done in CMake by linking to PYTHON3_STABLE_LIB and seems to work/link to the correct library on Windows, Mac, so all looking promising. Once I get ManyLinux to cooperate will have wheels on all platforms to test with and to run scripts similar to yours before packaging.

@aymanhab

Copy link
Copy Markdown
Member Author

All platforms now build and publish the abi3 wheels. Tested on windows with 3.13 and 3.14 using the same wheel worked out of the box like a charm 😃 Any volunteers to give the wheels a spin on a version other than python 3.11 (used on ci) on Mac or linux? @nickbianco @AlbertoCasasOrtiz @carmichaelong or @adamkewley Thank you

@adamkewley

Copy link
Copy Markdown
Contributor

@aymanhab Ubuntu 24 seems fine for me. Detailed report below


For Ubuntu 24, I tried this one-liner with uv that tests many versions of Python against the wheel by importing OpenSim and then making a blank Model:

for v in "3.10" "3.11" "3.12" "3.13" "3.14"; do mkdir -p $v && cd $v && uv venv --clear --python $v && uv pip install ../opensim-4.6-1-cp310-abi3-linux_x86_64.whl && uv run python -c "import opensim ; opensim.Model() ; print('$v works')" && cd - || break ; done

First run had an issue with openblas:

Using CPython 3.10.21
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 117ms
Prepared 2 packages in 704ms
Installed 2 packages in 6ms
 + numpy==2.2.6
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/adam/Desktop/opensim-stableabi/3.10/.venv/lib/python3.10/site-packages/opensim/__init__.py", line 50, in <module>
    from .simbody import *
  File "/home/adam/Desktop/opensim-stableabi/3.10/.venv/lib/python3.10/site-packages/opensim/simbody.py", line 10, in <module>
    from . import _simbody
ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory

Fixed by installing it system-wide (this is something almost any Linux user will automatically LLM/google and run):

sudo apt-get install libopenblas0

Re-running passed for all versions:

Using CPython 3.10.21
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 2ms
Installed 2 packages in 6ms
 + numpy==2.2.6
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
3.10 works
/home/adam/Desktop/opensim-stableabi
Using CPython 3.11.16
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 1ms
Installed 2 packages in 5ms
 + numpy==2.4.6
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
3.11 works
/home/adam/Desktop/opensim-stableabi
Using CPython 3.12.3 interpreter at: /usr/bin/python3.12
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 1ms
Installed 2 packages in 5ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
3.12 works
/home/adam/Desktop/opensim-stableabi
Using CPython 3.13.15
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 1ms
Installed 2 packages in 5ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
3.13 works
/home/adam/Desktop/opensim-stableabi
Using CPython 3.14.7
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 1ms
Installed 2 packages in 5ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///home/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp310-abi3-linux_x86_64.whl)
3.14 works

@adamkewley

adamkewley commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

@aymanhab for macOS, the wheel in CI doesn't look like a stableABI one? It's cp311-cp311, rather than cp311-abi3 (or even cp310-abi3, if we're going for 3.10).


Just to give it a whack anyway, I renamed the file to cp311-abi3 and ran it. It superficially ran for 3.11, 3.12, and 3.13 but it segfaulted in 3.14, which might be because it's a not-SABI wheel?

For macOS, i ran this one-liner (after modding the filename):

for v in "3.11" "3.12" "3.13" "3.14"; do mkdir -p $v && cd $v && uv venv --clear --python $v && uv pip install ../opensim-4.6-1-cp311-abi3-macosx_15_0_universal2.whl && uv run python -c "import opensim ; opensim.Model() ; print('$v works')" && cd - || break ; done

Output:

Using CPython 3.11.15
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 4ms
Installed 2 packages in 17ms
 + numpy==2.4.6
 + opensim==4.6 (from file:///Users/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp311-abi3-macosx_15_0_universal2.whl)
3.11 works
~/Desktop/opensim-stableabi
Using CPython 3.12.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 3ms
Installed 2 packages in 16ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp311-abi3-macosx_15_0_universal2.whl)
3.12 works
~/Desktop/opensim-stableabi
Using CPython 3.13.14
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 3ms
Installed 2 packages in 14ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp311-abi3-macosx_15_0_universal2.whl)
3.13 works
~/Desktop/opensim-stableabi
Using CPython 3.14.0 interpreter at: /Library/Frameworks/Python.framework/Versions/3.14/bin/python3.14
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 3ms
Installed 2 packages in 15ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/adam/Desktop/opensim-stableabi/opensim-4.6-1-cp311-abi3-macosx_15_0_universal2.whl)
(macOS CRASH: segfault)

Segfault excerpt:

Exception Type:    EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000000000007
Exception Codes:   0x0000000000000001, 0x0000000000000007

Termination Reason:  Namespace SIGNAL, Code 11, Segmentation fault: 11
Terminating Process: exc handler [68533]


VM Region Info: 0x7 is not in any region.  Bytes before following region: 4297080825
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      100204000-100208000    [   16K] r-x/r-x SM=COW  /Library/Frameworks/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   _simulation.so                	       0x10d129b88 <deduplicated_symbol> + 12
1   _simulation.so                	       0x10d129a24 SwigPyObject_Check(_object*) + 32
2   _simulation.so                	       0x10d129518 SWIG_Python_GetSwigThis(_object*) + 24
3   _simulation.so                	       0x10d154ed0 SWIG_Python_InitShadowInstance(_object*) + 84
4   _simulation.so                	       0x10ce19a44 Model_swiginit(_object*, _object*) + 28
5   Python                        	       0x1014401a0 cfunction_call + 108
6   Python                        	       0x101558678 _PyEval_EvalFrameDefault + 74220
7   Python                        	       0x101543de0 _PyEval_Vector + 632
8   Python                        	       0x1013c8688 _PyObject_Call_Prepend + 160
9   Python                        	       0x1014876bc slot_tp_init + 860
10  Python                        	       0x10147b308 type_call + 148
11  Python                        	       0x101558678 _PyEval_EvalFrameDefault + 74220
12  Python                        	       0x101543de0 _PyEval_Vector + 632
13  Python                        	       0x101543ac4 PyEval_EvalCode + 160
14  Python                        	       0x1016516f8 run_mod + 308
15  Python                        	       0x1016532c8 _PyRun_SimpleStringFlagsWithName + 280
16  Python                        	       0x101680dc0 Py_RunMain + 1272
17  Python                        	       0x101682144 pymain_main + 444
18  Python                        	       0x1016822c4 Py_BytesMain + 40
19  dyld                          	       0x181df44e4 start + 6992

@nickbianco

Copy link
Copy Markdown
Member

@adamkewley the wheel file I'm seeing attached to the CI runs is opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl (not the cp311-cp311 one you found).

I had to modify the one-liner slightly, but everything works fine on my Mac:

$ for v in "3.11" "3.12" "3.13" "3.14"; do mkdir -p $v && cd $v && uv venv && uv pip install ../opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl && uv run python -c "import opensim ; opensim.Model() ; print('$v works')" && cd - || break ; done

Using CPython 3.13.3 interpreter at: /opt/homebrew/opt/python@3.13/bin/python3.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 3ms
Installed 2 packages in 14ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/nbianco/Desktop/test_wheels/opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl)
3.11 works
~/Desktop/test_wheels
Using CPython 3.13.3 interpreter at: /opt/homebrew/opt/python@3.13/bin/python3.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 2ms
Installed 2 packages in 12ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/nbianco/Desktop/test_wheels/opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl)
3.12 works
~/Desktop/test_wheels
Using CPython 3.13.3 interpreter at: /opt/homebrew/opt/python@3.13/bin/python3.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 2ms
Installed 2 packages in 10ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/nbianco/Desktop/test_wheels/opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl)
3.13 works
~/Desktop/test_wheels
Using CPython 3.13.3 interpreter at: /opt/homebrew/opt/python@3.13/bin/python3.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 2 packages in 2ms
Installed 2 packages in 11ms
 + numpy==2.5.3
 + opensim==4.6 (from file:///Users/nbianco/Desktop/test_wheels/opensim-4.6-1-cp310-abi3-macosx_15_0_universal2.whl)
3.14 works
~/Desktop/test_wheels

@aymanhab

Copy link
Copy Markdown
Member Author

Awesome feedback @adamkewley and @nickbianco 🥇 will try to automate more testing in ci and investigate reported issue(s)
Now 3.10 reaches EOL next month, should we continue supporting or up to 3.11 based on your experience with existing python users? Thanks again

@adamkewley

Copy link
Copy Markdown
Contributor

@aymanhab I'd say that if we look hard enough there's certainly going to be end-users that refuse to move from 3.8, even, because (e.g.) they have some ancient ROS integration and don't want to upgrade.

In my opinion, in the case of legacy users using beyond-EOL Pythons, they should be advised to pin to an earlier OpenSim or build the project from source against the Python they need. Expecting the OpenSim project to maintain and ship 3.8, 3.9, 3.10, abi3 wheels on 3 platforms (i.e. 12 builds) is quite a bit of work. Doing one stable-ABI build reduces maintenance work to a point where the wheels can be part of the main build and the special "wheels build" stuff can be dropped, meaning OpenSim can test+ship new Python wheels to end-users before a release, even.

However, for legacy source builds against older Pythons, builders might need some kind of SABI-toggle in CMake. This is because systems like SWIG might disallow SABI against older Pythons, and because OS maintainers "know" the OS has particular version of Python and therefore would prefer to build an optimized Python-version-specific target. It might be that it's easiest to add that toggle now, because this PR now has a changeset that shows every singe location that needs to be controlled by the toggle. Otherwise, I'd PR the toggle separately when the need pops up (imo).

For context, OPynSim provides a toggle via OPYN_STABLE_ABI, which has to do something similar (turn off/on compiler flags, #defines, etc.). It uses 3.12 purely because nanobind requires that for SABI builds and because OPynSim's development timeline is 1/2 years, during which more users will upgrade to >= 3.12. OpenSim's got a much larger userbase with more legacy concerns, though, so of course it might be the case that the project needs to ship an SABI build that's as low as possible, plus perhaps even a legacy 3.8-specific build to support Stanford's robotics labs etc. for a cycle or two until users upgrade.

@aymanhab

Copy link
Copy Markdown
Member Author

Great feedback @adamkewley Indeed I have updated the ci script to publish the abi wheel on all platforms including windows so that regular ci artifacts can be used without waiting for the wheels build (which may stay around for tagged versions+authentication on pypi when needed).
I'll go ahead and add the toggle now with this change since it logically belongs here. Thanks again

@nickbianco

Copy link
Copy Markdown
Member

@aymanhab maybe I'm misunderstanding your previous comment, but I assume that the wheels builds will eventually be updated to only build one wheel per platform via the stable ABI? I think I would prefer to have a separate workflow for building and publishing wheels to PyPI.

@aymanhab

aymanhab commented Sep 17, 2026

Copy link
Copy Markdown
Member Author

@nickbianco Sorry for the confusion, the current plan is to include the stable abi wheels on all platforms with every ci build, then when we have a release/tag build and publish those stable abi wheels to PyPi using the separate build/publish workflows. And the build_wheels workflow will be updated accordingly.

@nickbianco

Copy link
Copy Markdown
Member

@aymanhab no worries and sounds good -- that was my understanding of the plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants