PyOpenCL and PyCUDA in Debian Wheezy

Latest versions of PyOpenCL and PyCUDA packages (2012.1-1 in both cases) have reached testing; thanks to Piotr Ożarowski for sponsoring. Wheezy is frozen now, so there will be no new versions of PyOpenCL nor PyCUDA in the new stable; the only allowed changes now are bug fixes. There has been no serious errors detected in my packages so it looks like there is no need for 2012.1-2.

All my packages have been going through some changes recently. Both PyCUDA and PyOpenCL were removed from testing during Boost 1.49 transition as they were depending on old Boost 1.46. Neither PyOpenCL nor PyCUDA could be rebuilt automatically as they depend on non-free packages but it might change in case of PyOpenCL.

I have created Python 3 package for pytools. This allowed for creation of PyOpenCL Python 3 package; it required extracting headers to separate package with headers to allow for installation of both PYthon 2 and Python 3 versions. Unfortunately I forgot to put some metadata (Replaces and Conflicts fields) which was noticed by Andreas Beckman in bug #674000. I have fixed it. At the same time Patrick Matthai uploaded AMD 12-4 drivers with OpenCL 1.2 support and Andreas Beckman uploaded new OpenCL 1.2 headers. This allowed for compiling PyOpenCL with OpenCL 1.2 support.  Unfortunately NVIDIA does not support OpenCL 1.2, so I had to force dependency on AMD libraries.

Bug #673992 was initially created by Vedran Miletic as the request for clarification of this situation, but soon it was clear that using AMD ICD loader with NVIDIA libraries fails in some cases. Using mixed libraries worked on all machines I had access to, but failed on hardware of other PyOpenCL users. I do not know whether this was caused by new
NVIDIA hardware (GF114) or by new AMD APU, but PyOpenCL was crashing during initialisation. Fortunately Vincent Danjean uploaded ocl-icd-libopencl1 package just
about that time. ocl-icd-libopencl1 is free ICD loader which seems to be giving better results for heterogeneous libraries.  Now PyOpenCL can be build with only free software. It can also run with only free software, but it’ll not be useful, as it’ll not find any ICDs. But there is work on free ICD so let’s hope that it will be soon solved. When it happens I’ll move PyOpenCL to the main section of Debian. If you want to know more, Vincent Danjean wrote email describing situation of OpenCL in Debian.

Currently PyOpenCL depends on any package providing libopencl1:

  • ocl-icd-libpencl1
  • amd-libopencl1
  • nvidia-libopencl1.

It’ll not work reliably with current nvidia-libopencl1 though, as it requires OpenCL 1.2 – and nvidia-libopencl1 provides only OpenCL 1.1, as was notices in bug #682435.  I have decided to leave relaxed dependency (instead of forcing PyOpenCL to require ocl-icd-libopencl1) to allow for experimentation with different hardware and software configurations and to be ready for new OpenCL implementations which might be uploaded to Debian in the future. But if you have some problems please try free ocl-icd-libopencl1 before filling the new bug report.

There is still small problem with PyOpenCL though. When using ocl-icd as ICD loader and running tests on NVIDIA hardware it crashes on Image testing. I was not able to test it throughly and I am not sure what is the root cause of the problem.

PyCUDA is Python 3 ready, but I have not created Python 3 package because there are still problems with compiling kernels on Python 3. Current package has already been split into python-pycuda and python-pycuda-headers so now adding Python 3 support is only matter of enabling python3-pycuda package.

As for Ubuntu, it looks like PyOpenCL 2012.1-1 has already been uploaded for Quantal Quetzal. It looks like changes I made for Debian also were beneficial for Ubuntu. PyCUDA is seen by Ubuntu but only as foreign (Debian) package, not as a part of Ubuntu. I think it will be problematic to push PyCUDA into Ubuntu as it depends on NVIDIA CUDA toolkit which is not packaged for Ubuntu.  If someone reading this knows how Ubuntu process works, or how to contact people responsible for release, or maintainers of NVIDIA or AMD drivers, please let me know. I am open for proposals of changes which will not break Debian package and will make life of Ubuntu maintainers easier.