Table of Contents
I have a newer blog post about doing CUDA setups on Ubuntu that you might find more useful than this post –dbk
"Install Ubuntu 16.04 or 14.04 and CUDA 8 and 7.5 for NVIDIA Pascal GPU"
I was configuring a laptop to run GPU accelerated molecular dynamics simulations as a demo at the Spring 2016 ACS meeting. This laptop is to be used for running simulations directly and as a visualization front end for some larger machines. The laptop is "bleeding edge", a monster, with an Intel Skylake i7 6700K, 32GB mem, m.2 storage, and a 4K display! That hardware can be a challenge for Linux so I decided to go "bleeding edge" with the OS install too. I was not optimistic but miraculously it all worked including the needed NVIDIA CUDA setup!
Disclaimer!
First things first. No one endorses, recommends, suggests or condones doing this setup. Including myself, Puget Systems, NVIDIA and all other sane persons. If that is completely understood then read on to see what I had to do to get this to work. Note: also, that this is not a guide! If you don't understand what I did then don't try it!
Seeing is believing …That's CUDA nbody running on Ubuntu 16.04
Hardware used
I've done this "Ubuntu 16.04 beta plus NVIDIA CUDA" setup on two systems so far, a laptop and a Peak Mini.
Puget Systems Traverse Pro
- 15 inch Notebook w/ TPM and 4K Matte IPS Screen
- Intel Core i7 6700 (3.4GHz)
- 2 x 16GB DDR4 SODIMM
- NVIDIA GTX 980M 8GB
- Samsung 500GB 950 Pro M.2 SSD
Puget Systems Peak Mini
- Intel Xeon E5-1650 (3.5GHz) 6-core
- 64GB DDR4 Reg ECC
- 2 x NVIDIA GTX 980
- Samsung 500GB 850 Pro SSD
Software used
Ubuntu MATE 16.04 beta 1
Note: I used the MATE desktop spin because Unity and Gnome3 failed to work on the laptop (as usual, the 3D compositors are very problematic with the NVIDIA drivers). Also, MATE works very well with remote desktops like X2go. … and MATE is my favorite Linux desktop at the moment!
NVIDIA CUDA 7.5.18
Note: The most recent Ubuntu supported for CUDA is 15.04 so installing on 16.04 beta is stretching things a bit! The main limitation is that version 5 gcc compilers are not supported yet.
NVIDIA Display Driver 361.28
NVIDIA driver from the Ubuntu "Software & Updates" –> "Additional Drivers" utility.
gcc version 5.3.1
You have to modify a cuda header file to enable this gcc version to work.
VMD and NAMD
These are wonderful molecular dynamics tools from the great folks at the Theoretical and Computational Biophysics Group at UIUC.
OS Install
The basic OS install was trouble free on both systems. I used the live install image ubuntu-mate-16.04-beta1-desktop-amd64.iso. You could also try using the ISO image from Ubuntu MATE 16.04 (Xenial Xerus) Daily Build. The Lubuntu and Xubuntu spins may also work but I would recommend to stay away from Unity and Gnome 3 unless you are feeling adventurous.
After the basic install do an "apt-get update" and "upgrade", reboot, and then go to "Software & Updates" –> "Additional Drivers" and install the latest tested NVIDIA driver. I didn't have any trouble with this using the MATE spin. After a reboot open the "nvidia-settings" app and check that everything is as expected. Configure you system to taste as you would for any install.
I tried to use the normal NVIDIA CUDA Ubuntu 15.04 repo deb files to do the CUDA install but failed. We will be doing the CUDA install manually from the ".run" file. Dependencies that would normally get pulled in from doing a CUDA install from the NVIDIA CUDA repo will have to be added.
There are lots of dependencies for a CUDA install. To make life easier I generated a dependency list from another system that was running 15.04.
The "-s" flag for apt-get will show the details of what would be done for an install including dumping the dependency list.
kinghorn@i7:~$ apt-get -s install cuda
In the output below I have removed all of the deb files that would be in the CUDA repo itself and just left dependencies that would get pulled in from the standard Ubuntu repos.
ca-certificates-java default-jre default-jre-headless fonts-dejavu-extra freeglut3 freeglut3- dev java-common libatk-wrapper-java libatk-wrapper-java-jni libdrm-dev libgl1-mesa-dev libglu 1-mesa-dev libgnomevfs2-0 libgnomevfs2-common libice-dev libpthread-stubs0-dev libsctp1 libsm- dev libx11-dev libx11-doc libx11-xcb-dev libxau-dev libxcb-dri2-0-dev libxcb-dri3-dev libxcb-g lx0-dev libxcb-present-dev libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync- dev libxcb-xfixes0-dev libxcb1-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxi -dev libxmu-dev libxmu-headers libxshmfence-dev libxt-dev libxxf86vm-dev lksctp-tools mesa-com mon-dev openjdk-7-jre openjdk-7-jre-headless tzdata-java x11proto-core-dev x11proto-damage-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev x11pr oto-xext-dev x11proto-xf86vidmode-dev xorg-sgml-doctools xtrans-dev libgles2-mesa-dev nvidia-modprobe build-essential
You can "sudo apt-get install" the above list and that should get most or all of the dependencies. There is a possibility that there will still be missing debs. For example I added the last three entries to the list when I discovered that they were missing on my new 16.04 install after the others were installed. That's is probably because they were already installed on the 15.04 system it ran "apt-get -s install cuda" on i.e. they didn't come up as needed dependencies because they were already installed.
CUDA Install
Download the CUDA .run file from the NVIDIA download site. I used this, http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run
kinghorn@skylap:~/Downloads$ chmod 755 cuda_7.5.18_linux.run kinghorn@skylap:~/Downloads$ sudo ./cuda_7.5.18_linux.run --override
The "–override" is needed so you don't get the error, "Toolkit: Installation Failed. Using unsupported Compiler."
Be sure to not install the NVIDIA driver that is in the .run file since you already have a more up to date version installed. Here's the command history with the long EULA text cut out.
kinghorn@skylap:~/Downloads$ sudo ./cuda_7.5.18_linux.run --override ------------------------------------------------------------- Do you accept the previously read EULA? (accept/decline/quit): accept You are attempting to install on an unsupported configuration. Do you wish to continue? ((y)es/(n)o) [ default is no ]: y Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): n Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]: Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y Enter CUDA Samples Location [ default is /home/kinghorn ]: /usr/local/cuda-7.5 Installing the CUDA Toolkit in /usr/local/cuda-7.5 ... Finished copying samples. =========== = Summary = =========== Driver: Not Selected Toolkit: Installed in /usr/local/cuda-7.5 Samples: Installed in /usr/local/cuda-7.5
CUDA Environment
Set up the environment and libs for the install
kinghorn@skylap:~/projects$ sudo emacs -nw /etc/profile.d/cuda.sh export PATH=$PATH:/usr/local/cuda/bin kinghorn@skylap:~/projects$ sudo emacs -nw /etc/ld.so.conf.d/cuda.conf /usr/local/cuda/lib64 kinghorn@skylap:~/projects$ source /etc/profile.d/cuda.conf kinghorn@skylap:~/projects$ sudo ldconfig
Force it to work with gcc 5
Fix/break the header file that doesn't want to let us use gcc > 4.8. All we are going to do is comment out (//) the error line that drops you out of a build.
kinghorn@skylap:~/projects$ sudo emacs -nw /usr/local/cuda/include/host_config.h line: 115 comment out error //#error -- unsupported GNU version! gcc versions later than 4.9 are not supported!
Compiling the CUDA Samples
You can now copy the CUDA sample to your testing directory and build some code.
kinghorn@skylap:~/projects$ rsync -av /usr/local/cuda/samples .
The "Makefile" in the samples directory does not pick up all of the paths for this install and I haven't fix all of the problems, just hacked around a few of them. One change I made was in the file "common/findgllib.mk
kinghorn@skylap:~/projects/samples/common$ emacs -nw findgllib.mk
line : 61 changed nvidia-352 to nvidia-361
UBUNTU_PKG_NAME = "nvidia-361"
Another common Ubuntu problem is needing to specify GLPATH=/usr/lib on the make line.
GLPATH=/usr/lib make
That will build a lot of the sample code. I only got one error in the output,
/usr/bin/ld: cannot find -lnvcuvid collect2: error: ld returned 1 exit status Makefile:346: recipe for target 'cudaDecodeGL' failed
That one was strange since the lib is there and other libs around it link OK?? … still some bugs.
Here's the output from the nbody benchmark run on the laptop (the openGL rendering works fine too),
kinghorn@skylap:~/projects/samples/bin/x86_64/linux/release$ ./nbody -benchmark -numbodies=256000 > Windowed mode > Simulation data stored in video memory > Single precision floating point simulation > 1 Devices used for simulation GPU Device 0: "GeForce GTX 980M" with compute capability 5.2 > Compute 5.2 CUDA device: [GeForce GTX 980M] number of bodies = 256000 256000 bodies, total time for 10 iterations: 7207.025 ms = 90.934 billion interactions per second = 1818.670 single-precision GFLOP/s at 20 flops per interaction
To finish off here's a snapshot of the laptop running a molecular dynamics simulation on the "Satellite Tobacco Mosaic Virus" with NAMD and displaying the running simulation to a 4K monitor in real time using VMD. Awesome!
Happy computing! –dbk