Building CMSIS-NN Python extension

[christophe0606]

Building the CMSIS-NN Python extension on all platforms (Mac, Linux, Windows ...) with different compilers is not easy.

But, it looks like, the only use of this extension in the executorch Cortex-M backend is to have access to some constants:
executorch\backends\cortex_m\target_config.py

There are some use of the API but it looks like it is for tests:
backends\cortex_m\test\misc\test_cmsis_pybind.py

So perhaps it would be simpler to have two cmsis-nn Python extensions:

• One pure Python based that expose the constant needed by Cortex-M backend
• Another one that gives access to the functions for testing purposes ?

It would simplify the installation for people who only want to use executorch.

[AdrianLundell]

Hi, the API is used for determining buffer sizes in the Executorch flow so splitting the package will probably not help here (https://github.com/pytorch/executorch/blob/main/backends/cortex_m/passes/scratch_buffer_sizes.py).

We are looking into making this installation as smooth as possible however, what issues have you been having and have you tested it since pytorch/executorch#20531 was merged?

/Adrian

[christophe0606]

@AdrianLundell Hi,
I am on windows so I am not using the executorch scripts directly.
I have attempted to follow the CMSIS-NN README to rebuild the package. And I got a few issues.
But I think the main issue may be documentation : I do not need to build the cmsis-nn shared library ? Right ? It is only for tests.

So I built the shared library and got a few minor issues:

• I used MSVC. I was not able to use the clang compiler coming with Visual Studio
• Unsupported compiler warning : not an issue.
• Some compilations options not recognized by MSVC:
  • cl : command line warning D9025: overriding '/Os' with '/Ot'
  • cl : command line warning D9002: ignoring unknown option '-Of'
  • cl : command line warning D9002: ignoring unknown option '-Oa'
• An error when building the python package. So I edited the file to use cmake.version

  error: subprocess-exited-with-error
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 7
  ╰─> [1 lines of output]
  ERROR: Use cmake.version instead of cmake.minimum-version with scikit-build-core >= 0.8
  [end of output]

• Dynamic linking issue when running binding tests:
  • RuntimeError: Missing symbol arm_avgpool_s8_get_buffer_size_mve in CMSIS-NN\build\Debug\cmsis-nn.dll
  • It looks like Windows build needs WINDOWS_EXPORT_ALL_SYMBOLS ON

    set_target_properties(cmsis-nn-shared PROPERTIES OUTPUT_NAME cmsis-nn WINDOWS_EXPORT_ALL_SYMBOLS ON)

• Other issue on windows : CMSIS_NN_SHARED_LIB must be set since library uses a different extension

After correcting those issues, I was able to run the binding tests.

But as I understand it is not needed if I just want to use executorch ? Right ?

[AdrianLundell]

The Cortex-M backend in executorch does use the bindings but it is not expected that you should need to manually build it, it should be included with the regular Executorch installation. If you are not using the Cortex-M backend or if you are using CMSIS-NN separately there is no need to build the bindings.

[christophe0606]

If by regular executorch installation you mean cloning the repository, then for windows it is not very practical. And most documentation explains to use examples/arm/setup.sh which is not ok on windows.

If you mean, doing pip install of torch, executorch, numpy etc .. then loading cortex_m_pass_manager in python gives the error:

ModuleNotFoundError: Cortex-M backend dependencies are not installed (missing: cmsis_nn). Install them with pip install --no-dependencies -r backends/cortex_m/requirements-cortex-m.txt, or run examples/arm/setup.sh --i-agree-to-the-contained-eula.

And using this command will give:

x Failed to build cmsis-nn @ | git+https://github.com/ARM-software/CMSIS-NN.git@d933672e7ca97eec70ef43230baee7b20c2a28ae
|-> The build backend returned an error
-> Call to scikit_build_core.build.build_wheel` failed (exit code: 7)

 [stderr]
ERROR: Use cmake.version instead of cmake.minimum-version with scikit-build-core >= 0.8

which is one of the error I reported above.

[AdrianLundell]

It will come with the pre-built python wheel when there is one available including the commit I linked above, so likely when 1.4 is released. Building executorch locally is done through the ./install_executorch.sh script, have you tried using that on the main branch as a solution in the meanwhile?

The Cortex-M backend is still in beta so I apologize if it is still a bit rough, but we are working on making it more stable. However, the complete CMSIS-NN python library was created for the purpose of memory planning in Executorch so splitting it is likely not going to help here.

[christophe0606]

I have not yet tested the ./install_executorch.sh on Windows. I know there are some bash shell for windows so I'll test.
But I am not blocked. I was able to rebuild the extension.
I just wanted to share my findings in case it may help to improve the developer experience on windows.