Error installing graphkernels/GKextCPy: Unable to locate python.h header file.

[git-tea]

Hello!

I want to install graphkernels, and am receiving the output at the following file when using the command: pip --no-cache-dir install graphkernels:
graphkernels_install_err.txt

The main error from this seems to be "Python.h: No such file or directory". My understanding is that this usually means that pythonX.Y-dev must be installed according to the python version used.

I have confirmed that I do have python3.9-dev installed corresponding to my version of python using the command sudo apt-get install python3.9-dev, which gave the output "python3.9-dev is already the newest version (3.9.13-1+jammy1)." And the Python.h file exists on my system at /usr/include/python3.9/Python.h.

Is there something else I should do to install the package?

[Pseudomanifold]

Sorry to hear this—not aware of this problem, but it's been a while since I was working on this project. Can you try installing everything from source (i.e. from the src directory of this repo)?

[git-tea]

Thank you for the quick response!

I have tried to install via src, and I get the output in the following file:
graphkernels_src_install.txt

I do not see any obvious errors during the installation. However, I seem to come across a second error later on.

When I try to call CalculateWLKernel from my script, I get the following error:
kernel_mat = gk.CalculateWLKernel( relabeled_graphs, n_iters )
File "/opt/GraphKernels/src/graphkernels/graphkernels/kernels.py", line 160, in CalculateWLKernel
E, V_label, V_count, E_count, D_max = GetGKInput(G)
File "/opt/GraphKernels/src/graphkernels/graphkernels/utilities.py", line 71, in GetGKInput
E = gkCpy.VecMatrixXi()
AttributeError: module 'GKextCPy' has no attribute 'VecMatrixXi'

Is there a separate path that I need to link to be able to locate these attributes?

[Pseudomanifold]

That's weird—can you load the module and tell me the output of the following command, please:

import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent

[git-tea]

The following is what is returned from running that command in the python interpreter:

">>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_GKextCPy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 17, in
_GKextCPy = swig_import_helper()
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 16, in swig_import_helper
return importlib.import_module('_GKextCPy')
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_GKextCPy'"

[Pseudomanifold]

This looks to me like the extension module was not installed. Can you try to install this manually as well (from the GKextCPy folder)?

[git-tea]

I believe that I have now installed GKextCPy from src using the corresponding setup.py file.

I ran python setup.py build and then python setup.py install --user like with graphkernels. I then repeated the build and installation of graphkernels.

I get the same error when I repeat the test from above (import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent)

[Pseudomanifold]

I don't understand this fully. Is the module maybe already installed (if somewhat partially)? The setup script might refuse to overwrite it.

[git-tea]

I have cleared all the references I can find to the module and tried to install it from source from scratch.

After clearing everything, the entire local GraphKernels directory was gone, and there was no listing of either GKextCPy or graphkernels in pip list.

I re-cloned the repository and both built and installed GKextCPy and then graphkernels. I also built and installed with sudo permissions to ensure no access problems. The following files correspond to each command, such that the setup.py files are the ones inside their corresponding package.
GKextCPy - sudo ./setup.py build: gkcpy_build.txt
GKextCPy - sudo ./setup.py install --user: gkcpy_install.txt
graphkernels - sudo ./setup.py build: gk_build.txt
graphkernels - sudo ./setup.py install --user: gk_install.txt

When I then run the command above import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent from a python interpreter in my home directory, I get the following error:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'GKextCPy'

[Pseudomanifold]

Sorry for the hassle here...

It seems that the packages are being installed for the root user only. If you use sudo, you can drop the --user flag. This should install the packages globally (might cause other problems, though; maybe a virtual environment would be best).

[git-tea]

It's no problem. I appreciate you taking the time to help with this.

I've deleted and re-installed it again to make sure there's no conflict, and I installed it without the --user flag. I seem to be getting the same ModuleNotFoundError when running the test command.

Do I need to run the command from within a specific directory to ensure that all packages are locate-able?

When I run the test from somewhere outside of the GraphKernels directories, I get the ModuleNotFoundError above:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'GKextCPy'

When I run within GraphKernels/src, I get the following error:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "", line 1, in
File "/opt/anaconda3/lib/python3.9/pathlib.py", line 1082, in new
self = cls._from_parts(args, init=False)
File "/opt/anaconda3/lib/python3.9/pathlib.py", line 707, in _from_parts
drv, root, parts = self._parse_args(args)
File "/opt/anaconda3/lib/python3.9/pathlib.py", line 691, in _parse_args
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

And when I run within the GraphKernels/src/GKextCPy directory, I get the following errors:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_GKextCPy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 17, in
_GKextCPy = swig_import_helper()
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 16, in swig_import_helper
return importlib.import_module('_GKextCPy')
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_GKextCPy'

[Pseudomanifold]

Still looks like there's something wrong with the installation. I see that you have Anaconda active as well. Can you try installing it in the Anaconda environment using --user and no use of sudo?

[Pseudomanifold]

(I just pushed a simplified module installation fix, but that should not apply to your issue, unfortunately. If we can't get anywhere, the main developers @mahito-sugiyama or @eghisu might want to weigh in.)

[git-tea]

When you say to install it in the Anaconda environment, do you mean using pip to install it? Or do you mean to install it into the directories for the conda packages?

[Pseudomanifold]

I mean installing it using setup.py while the Anaconda environment is active, but without using sudo. From what I saw in the logs, your previous installation used a different Python version than the one your Anaconda environment uses.

[git-tea]

The package is within my /opt directory, so it has limited permissions by default.

To account for that, I have now added write permissions (using chmod +w) to each of the directories of GraphKernels so that the build process can create what it needs to.

Afterward, I ran python3 setup.py build on the setup.py file in GKextCPy, and it returned the following error. It appears to be the same error that I saw when installing via pip:

Requirement already satisfied: numpy in /opt/anaconda3/lib/python3.9/site-packages (1.21.5)
Requirement already satisfied: pkgconfig in /opt/anaconda3/lib/python3.9/site-packages (1.5.5)
running build
running build_py
creating build/lib.linux-x86_64-3.9
copying GKextCPy.py -> build/lib.linux-x86_64-3.9
running build_ext
building '_GKextCPy' extension
creating build/temp.linux-x86_64-3.9
gcc -pthread -B /opt/anaconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/anaconda3/include -I/opt/anaconda3/include -fPIC -O2 -isystem /opt/anaconda3/include -fPIC -I/opt/anaconda3/lib/python3.9/site-packages/numpy/core/include -I -I/opt/anaconda3/include/python3.9 -c GKextCPy.cpp -o build/temp.linux-x86_64-3.9/GKextCPy.o -std=c++11 -O3
gcc -pthread -B /opt/anaconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/anaconda3/include -I/opt/anaconda3/include -fPIC -O2 -isystem /opt/anaconda3/include -fPIC -I/opt/anaconda3/lib/python3.9/site-packages/numpy/core/include -I -I/opt/anaconda3/include/python3.9 -c GKextCPy_wrap.cxx -o build/temp.linux-x86_64-3.9/GKextCPy_wrap.o -std=c++11 -O3
GKextCPy_wrap.cxx:173:11: fatal error: Python.h: No such file or directory
173 | # include <Python.h>
| ^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1