This repository was archived by the owner on Jul 17, 2024. It is now read-only.
Repository navigation
This repository was archived by the owner on Jul 17, 2024. It is now read-only.
44 - ell-python-interface-test (Failed) #254
Description
I'm trying to build ELL on windows 10. However, on running the following command:
ctest . --build-config release -R ell-python-interface-test
I get this error:
Test project C:/ELL/ELL/build
Start 44: ell-python-interface-test
1/1 Test #44: ell-python-interface-test ........***Failed 0.12 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.25 sec
The following tests FAILED:
44 - ell-python-interface-test (Failed)
Errors while running CTest
Activity
lovettchris commentedon Dec 2, 2020
If you run
ctest . --build-config release -R ell-python-interface-test -VVyou will get more details on why it fails. It could fail if you have not setup your python environment correctly, or you have some missing python packages -- see the "requirements.txt" file for a full list.A-KL commentedon Dec 5, 2020
Hi guys, I'm also trying to tun python tests for ELL on windows + Python 3.8. Same error.
The content of ell package looks like:
And installed packages:
lovettchris commentedon Dec 6, 2020
Thanks, so the compiled ELL module named "_ell_py" is not loading. This can happen if this path is not in your PATH environment:
It could require this path or the sandybridge path depending on your CPU type. The actual path needed is output when you run "cmake" to build ELL.
A-KL commentedon Dec 14, 2020
Thanks, Chris Lovett (@lovettchris). I've checked and verified my path set up:
also
However, during these tests - I didn't use Miniconda and Python 3.6. Just 3.8.5. Is it essential to run under 3.6?
n-gineer commentedon Dec 19, 2020
You mean this command, right?
cmake --build . --config Releasenot this one?
cmake --build . --target _ELL_python --config ReleaseI don't have a folder "C:\Sources"; should I have made this directory and cloned there? I didn't see anything to this effect in the ReadMe. I'm not seeing any path mentioning OpenBLASLibs when building ELL or ELL_python.
The haswell and sandy bridge seem to be rather old CPU's. Mine is an Intel i7-8650U, which seems to be Kaby Lake; does this mean it isn't supported by ELL?
lovettchris commentedon Dec 20, 2020
A-KL, yes please try python 3.6, I just tried python 3.8 and it didn't work... so we might have a bug there to investigate.
n-gineer, when you run cmake it prints out something like
-- Using BLAS include library ...take that path replace the lib at the end with "bin" and add that to your PATH environment, so that ELL can find the openblas.dll that lives there.If you are using AMD chip, it will not use openblas since we have not shipped a version of openblas that works on AMD chips. In that case you would have to build openblas yourself if you want that. You can use ELL without openblas of course.
It also prints out a bunch of
-- PYTHON_*information showing you what version of python it found, this is the version of python that _ELL_Python will target.You can also download this [Dependency Walker tool][https://dependencywalker.com/) and check the _ell_py.pyd to see what it thinks is missing. I had the best luck running "depends.exe" then using file open to load the _ell_py.pyd. PS: the depends.exe UI is very slow on this binary for some reason, so just be patient. I find if I use View/collapseAll then the UI works better. You can safely ignore all the missing modules that start with the name API-MS-WIN-CORE... You'll be looking for missing dependencies in the main view, mine is ok, and looks like this. Note, I'm using AMD so you can see there is no openblas listed. But if you see a warning or error icon in this view then that's your missing dependency.
lovettchris commentedon Dec 20, 2020
n-gineer, also check this file in the ELL repo: "cmake\OpenBLASSetup.cmake" as you may need to edit that file to add the processor_model of your new chip like we did with model numbers 60, 63, 69, 70, 79, 78, 142, 158 binding them all toe the haswell version of openblas with "set(processor_generation "haswell")".
You can get your CPU family and model number by running this:
lovettchris commentedon Dec 20, 2020
A-KL - my python 3.8 problem was this issue on the latest version of numpy. The solution is:
Now my build of ELL is working fine on python 3.8.6.
n-gineer commentedon Dec 22, 2020
Chris Lovett (@lovettchris) thanks for the tips!
It seems since I already built, it doesn't output the lines I'm looking for. Should I delete the build folder contents and rebuild?
I did found this file in the ELL build directory: ell_build_tools.json, within which it gave:
"blas": "C:/.../Autocharger/Computer Vision/ELL/external/OpenBLASLibs.0.2.19.3/build/native/x64/haswell/lib/libopenblas.dll.a"I had already added the haswell/bin version of this to my Path and verified it is being seen by Powershell with an echo $ENV:PATH
Here is the entirety of the output from the build command (apologies that I can't get code tag to capture properly):
`$: build> cmake --build . --config Release
Microsoft (R) Build Engine version 16.8.2+25e4d540b for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
utilities.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\utilities\Release\utilities.lib
math.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\math\Release\math.lib
data.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\data\Release\data.lib
dsp.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\dsp\Release\dsp.lib
emitters.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\emitters\Release\emitters.lib
value.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\value\Release\value.lib
emittable_functions.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\emittable_functions\Release\emittable_functions.lib
evaluators.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\evaluators\Release\evaluators.lib
functions.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\functions\Release\functions.lib
model.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\model\Release\model.lib
predictors.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\predictors\Release\predictors.lib
nodes.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\nodes\Release\nodes.lib
trainers.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\trainers\Release\trainers.lib
common.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\common\Release\common.lib
makeExamples.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\makeExamples.exe
Generating example models
optimization.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\optimization\Release\optimization.lib
passes.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\passes\Release\passes.lib
_ELL_python.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\interfaces\python\Release_ell_py.pyd
compile.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\compile.exe
pythonPlugins.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\tools\utilities\pythonPlugins\Release\pythonPlugins.lib
profile.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\profile.exe
apply.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\apply.exe
testing.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\testing\Release\testing.lib
common_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\common\Release\common_test.exe
data_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\data\Release\data_test.exe
debugCompiler.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\debugCompiler.exe
model_testing.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\model_testing\Release\model_testing.lib
dsp_nodes_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\nodes\Release\dsp_nodes_test.exe
dsp_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\dsp\Release\dsp_test.exe
dsp_timing.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\dsp\Release\dsp_timing.exe
emittable_functions_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\emittable_functions\Release\emittable_functions_test.exe
emitters_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\emitters\Release\emitters_test.exe
evaluators_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\evaluators\Release\evaluators_test.exe
finetune.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\finetune.exe
finetune_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\finetune_test.exe
forestTrainer.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\forestTrainer.exe
functions_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\functions\Release\functions_test.exe
global_optimizer_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\model\Release\global_optimizer_test.exe
linearTrainer.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\linearTrainer.exe
makeProfileModels.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\makeProfileModels.exe
math_profile.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\math\Release\math_profile.exe
math_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\math\Release\math_test.exe
model_compiler_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\model\Release\model_compiler_test.exe
model_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\model\Release\model_test.exe
nn_nodes_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\nodes\Release\nn_nodes_test.exe
nodes_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\nodes\Release\nodes_test.exe
nodes_timing.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\nodes\Release\nodes_timing.exe
optimization_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\optimization\Release\optimization_test.exe
passes_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\passes\Release\passes_test.exe
predictors_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\predictors\Release\predictors_test.exe
print.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\print.exe
protoNNTrainer.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\protoNNTrainer.exe
retargetTrainer.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\retargetTrainer.exe
sweepingSGDTrainer.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\bin\Release\sweepingSGDTrainer.exe
trainers_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\trainers\Release\trainers_test.exe
utilities_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\utilities\Release\utilities_test.exe
value_test.vcxproj -> C:...\Autocharger\Computer Vision\ELL\build\libraries\value\Release\value_test.exe`
Here is what dependency walker shows:
There are a number of API-WIN-MS-CRT and some others hidden in the tree. The LIBOPENBLAS tree has no warnings or errors showing when I expanded fully.
returns
Identifier REG_SZ Intel64 Family 6 Model 142 Stepping 10which appears to be already supported in "cmake\OpenBLASSetup.cmake"
lovettchris commentedon Dec 30, 2020
Ok, great, so your openblas setup looks great, the cmake setup is recommending haswell version of openblas and depends.exe is finding it, so now it is a complete mystery why the compiled ELL model is not loading properly in your python 3.8 environment.
n-gineer, since you are not the one that opened this bug, can you remind me what is the exact issue you are facing? Is it still that the "ell-python-interface-test" unit test is failing for you, namely you cannot get your python environment to load _ell_py module? If so can you try this:
This is how you can test the ell package loads into your python environment independent of the rest of the unit test.
MoonSunBeen commentedon Dec 31, 2020
Chris Lovett (@lovettchris)
Hello, I have the same problem.
When I enter the cmd
ctest . --build-config release -R ell-python-interface-test -VVIt is part of my error code
44: DoubleVector test ... Passed44: FloatVector test ... Passed44: vector_test passed44: model_test failed44: exception:44: Loading file ../../../examples/models/model_1.model44: common_test failed44: trainers_test.test -- TBD44: trainers_test passed44: test_tensor ... Passed44: test_predictor ... Passedmodel_test and common_test are failed
So, I've check the requirements.txt
(py36) C:\Program Files\ELL\interfaces\python\package\ell>python -m pip freezeappdirs==1.4.4audioread==2.1.9bcrypt==3.2.0certifi==2020.12.5cffi==1.14.4chardet==4.0.0cntk==2.7cntk-gpu==2.7configparser==5.0.1cryptography==3.3.1cycler==0.10.0dask==2020.12.0dataclasses==0.7decorator==4.4.2flake8==3.8.4future==0.18.2idna==2.10importlib-metadata==3.3.0iso8601==0.1.13joblib==1.0.0kiwisolver==1.3.1librosa==0.8.0llvmlite==0.35.0matplotlib==3.3.3mccabe==0.6.1numba==0.52.0numpy==1.19.3olefile==0.46onnx==1.8.0opencv-python==4.4.0.46packaging==20.8paramiko==2.7.2parse==1.18.0Pillow @ file:///C:/ci/pillow_1603822370986/workpooch==1.3.0protobuf==3.4.1psutil==5.8.0PyAudio==0.2.11pycodestyle==2.6.0pycparser==2.20pyflakes==2.2.0PyNaCl==1.4.0pyparsing==2.4.7python-dateutil==2.8.1python-speech-features==0.6PyYAML==5.3.1requests==2.25.1resampy==0.2.2scikit-learn==0.24.0scipy==1.5.4serial==0.0.97six @ file:///home/conda/feedstock_root/build_artifacts/six_1590081179328/workSoundFile==0.10.3.post1threadpoolctl==2.1.0torch==1.7.0torchvision==0.8.1typing-extensions @ file:///tmp/build/80754af9/typing_extensions_1598376058250/workurllib3==1.26.2wincertstore==0.2zipp==3.4.0I've checked all the package are installed
I have no idea about this problem. Can you suggest me the solution of this problem?
lovettchris commentedon Dec 31, 2020
MoonSunBeen, this looks like a different issue, can you open a new issue for it? Thanks.
n-gineer commentedon Jan 1, 2021
I'm having the same error as OP after attempting this step of the install instructions:
ctest . --build-config release -R ell-python-interface-testAnd it is failing on the same test, "ell-python-interface-test".
I attempted to import ELL and it returned:
Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'ELL'Chris Lovett (@lovettchris), thanks for the help!
lovettchris commentedon Jan 1, 2021
n-gineer, thanks can you run this as this is the lowest level test you can do to see if you can import the ELL module.
From your ELL repo:
You should see these files:
Now run "python" interactively in the same conda environment you used to build ELL:
python
then run this python statement
And let me know what you see? If this works then try this python command:
and you should see this:
n-gineer commentedon Feb 6, 2021
Hi Chris Lovett (@lovettchris), thanks for the help on this. I'm sorry I did a poor job of reading instructions and thought the use of conda was optional, but upon re-reading I realized that python as a whole was optional.
results of dir:
I had installed ell without using a python environment, so I suspect that may be causing my problems. I went back and installed miniconda and made a python environment and tried import ell there (not expecting it to work due to backwards order of operations) and got back:
When I tried doing it from conda prompt , it worked okay (probably because I didn't properly set up the environment beforehand), and I got back:
I'm guessing this won't actually work and I should completely uninstall ELL and start over?
lovettchris commentedon Feb 6, 2021
Ok great, using miniconda is fine, and if "import ell" works then you should be good to go. You can run this test from your build folder to be sure it is working properly:
shalaka188 commentedon Mar 19, 2021
Hiii ,
even I am facing the same problem
I used anaconda prompt of version 3.18 & as directed i have created an environment py36 & done as directed but still facing issue in
ctest . --build-config release -R ell-python-interface-test__
ouput is :
Test project C:/Users/Shalaka.DESKTOP-9R8N143/ELL/build
Start 44: ell-python-interface-test
1/1 Test #44: ell-python-interface-test ........***Failed 0.06 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.12 sec
The following tests FAILED:
44 - ell-python-interface-test (Failed)
Errors while running CTest_
but I did,'t install python 3.6
was that neccessary?
plz help me out
Thank you..