The Best Way To Install Ubuntu 16.04 with NVIDIA Drivers and CUDA

Posted on by

In this post I’ll be going over details of Installing Ubuntu 16.04 including the NVIDIA display driver and, optionally, NVIDIA CUDA. I have found the method presented here to be the most likely to succeed no matter what hardware configuration you are installing onto.

I usually don’t like to start a blog post with the words “The Best”. However, I have to confess that I often do searches myself that start with those words. Installing Linux can sometimes be frustrating especially if you are installing onto new “bleeding edge” hardware or something with an unusual configuration. I have been installing Linux since it has existed. The early days were very challenging but great fun. Modern Linux distributions, unfortunately, can sometimes still be challenging but with out the fun part. Linux is usually very easy to install and configure but when it’s not it can be very frustrating. The installation method I present here is, in general, my preferred method for installing Linux. In particular this post details how I nearly always install Ubuntu. I’m doing Ubuntu 16.04 because it is a very solid base platform for a lot of applications and is widely used. I will be using this base install for the docker plus NVIDIA-docker and NGC docker registry series of posts I will be doing soon.

If you have run into difficulties doing an Ubuntu 16.04 (or any Linux install) you might want to try what I outline in this post. I’ve used this to install on a large variety of hardware configurations and it is nearly always successful, — quick and trouble free. Note, that I said “nearly always” 🙂 I never cease to be amazed at how troublesome a seemingly simple task can become when computers are involved.

Installing a Desktop with Ubuntu 16.04 from the “Server” base image

Install Ubuntu from the server base. You might be asking yourself — if we are going to do a desktop setup why are we installing from the server media which will give us a text based console? Here are some reasons for doing it this way,

Why install from server?

  • Server install image is small and downloads easily. (826MB vs 1.7GB for a typical desktop image).

  • A minimal server install is fast and easy and it’s simple to add any desktop environment you want on top of that.

  • A small server install is fast and easy to update to current application versions. The software you install after that will be up-to-date rather than the old versions that would be on the desktop install media that would need to be updated.

  • It’s fairly simple to script a complete install that starts from a simple server base. [I will do this in the first post on doing a docker setup to use the NVIDIA NCG registry.]

  • It’s a reasonable base for docker or VM images that you may create later so it’s a good idea to be familiar with it.

  • A server install is “usually” much more likely to work if you are using “bleeding edge” hardware or unusual configurations.

That last reason is actually why I do it this way! I often have a pile of steaming fresh … hardware … to test. That hardware may be pre-production samples etc.. I’ve installed Linux thousands of times, and that’s not counting automated provisioning. I usually have best luck with installs by keeping things simple and then build on that. [ I had an Ubuntu 18.04 pre alpha 1 install running Tensorflow test jobs on pre release hardware with a full docker and NVIDIA-docker v2 configuration in about an hour … and none of that is supported yet! (“don’t try this at home” 🙂 ]

Note! Installing from the full desktop live media can be very simple “when it works” and it often does work just fine. It can sometimes be frustrating trying to get proprietary graphics drivers installed and working right etc.. Don’t hesitate to try the install that way, it may work great but, if it doesn’t, then knowing how to fall back to a simple base install can save you some grief and frustration.

Step 0) Prepare your hardware and make backups if needed.

I’m assuming that you are doing a clean install. That could be installing on new hardware or reinstalling over a systems that you don’t need to save any data from. If you have anything important on the disk you are going to use for the install please back it up! If you have secondary drives or data drives that you want to preserve and use in the new install it is a good idea to disconnect them during your fresh install so that you don’t accidentally overwrite them. You can reattach and mount them in your new install later.

Step 1) Get Ubuntu

You can get the Ubuntu server image from Canonical’s site. Use the 16.04.3 LTS download. I do not recommend 17.10. I have used it but it has caused trouble for some people and it will go away soon. 16.04 LTS is solid and well supported and will remain that way for several more years. If you use the HWE kernel it will work fine with recent hardware. Ubuntu is released as an LTS (long term support) version every 2 years. The next LTS will be 18.04 due to be released at the end of March. It will be a while before that is fully supported after it’s released. I will do a post on how to update or reinstall after it’s out.

Step 2) Boot the install image in UEFI mode

I recommend you use UEFI for your install. If you have been doing this for a while you may be comfortable with legacy MBR installs but it’s probably time to try UEFI. If you are using new hardware some of it may not even work correctly if you don’t boot to UEFI. Be sure you BIOS has “secure boot” disabled. You might be able to get your install to work with it enabled but you will have mysterious errors when you try to install the NVIDIA driver kernel modules.

To start your install you will need to know the magic key to press at startup to get a boot device select screen. Unfortunately that’s not standard among different motherboard manufacturers so you will need to look it up if you don’t know what it is. When you get to that boot selection screen you may have a choice of boot partitions on the install device. One of them should be labeled UEFI.

Step 3) Install with the HWE kernel

I recommend using the HWE kernel (Hardware Enhanced). This will be a more recent stable kernel with better support for newer hardware (remember the 16 in 16.04 stands for the year 2016). Currently the kernel from the install will be a 4.10 version but will update after the install to an LTS 4.13 kernel. That has the latest patches for various Intel flaws and security issues. [I’m picking on Intel obviously but, this applies to AMD and ARM CPU’s too.] The 4.13 kernel seems to be very good. I’ve observed that it gives better performance for most systems than the 4.10 or 4.4 kernels (at least on current hardware that I’ve tested). The server install will be an HWE 4.10 kernel that will update to 4.13.

HWE Kernel boot

Step 4.1) Proceed with the install

Note: if you start the install and have corrupted video then you may need to do what I suggest in Step 5) below to get the install started correctly.

Start install
Once you have started the install you will be guided though several question that will be pretty obvious … until, you get to “Partition disks”

Step 4.2) Disk partitions

I recommend that you do manual disk partitioning. It’s not hard and and if you let the installer do it manually you may get some surprises if you are not careful! One insane thing that the Ubuntu installer will do with automatic partitioning is to create a swap partition equal to the size of your physical memory. Years ago when a system may have 4-8GB of memory that was a recommend practice. I recently did a quick install on a system that had 128GB of memory and in my rush I selected automatic partitioning and didn’t check what it had configured. It created a 128GB swap file on my 256GB SSD!

Disk partitioning is an area where people seem to often disagree. You used to be able to tell a systems that I had installed by the partition layout, it was almost a signature. That was the old days! On a modern system I highly recommend that you use an SSD device for your OS install and keep the partitioning dirt simple. You can add, format and mount data disks and such later if needed. Just keep thinks simple. You will need a small EFI partition, a small swap partition is good, and the rest of the disk can be your system root partition. This is in my opinion the best way to layout a modern SSD.

Manual partitioning

The partitioner on the server install is pretty good and easy to use. Select the disk you are using for the install and, delete any old partitions if needed.

  • If it’s a new device selecting it will ask if you want to make a new partition table — do that.

  • Then select “Free space” and create your EFI partition (for the the UEFI boot information). Select partition type EFI. Add it to the beginning of the disk and I recommend a size of 256MB.

  • Next select the “Free space” again and add a partition of type swap to the end of the disk space. I used to normally add a 16GB swap partition regardless of how much physical memory is in the system but now I think it is fine to just create a 2GB swap partition just for those occasions that the systems expects to have one. [Note: It’s not required to have a swap partition but I think it is still good practice. I noticed that Ubuntu 18.04 adds a 2GB swap file by default with automatic partitioning. That’s a big improvement over the old installer! A swap file can be created and mounted after the install if you think you need it for some reason.]

  • Next, select the “Free space” again and use the remaining storage space for your OS root partition (/). For this partition you can use the default EXT4 type and mount point /

  • Then “Finish partitioning and write changes to disk”

Partitions

At this point the installation will will proceed with just a couple more simple questions. When you get to the “Software selection” screen I recommend that you add the OpenSSH server.

Add ssh server

After this the install will finish and you will be ready to boot into the OS.

Step 5) Potential problem number 1!

You might need “nomodeset” until you get GPU drivers installed! You might have this problem during the install too. What I describe here will “usually” take care of this.

When you reboot the system may come up into a corrupted screen or it may hang! This is because the proprietary driver is not installed for the NVIDIA card (which I presume you have in the system). This is most often caused by the system seeing your great video card and then trying to bring up a console in a fancy “frame-buffer”. RANT-ON There is no reason for that on a server install! … RANT-OFF

Here’s how you get around this if you encounter it,

When you get to the Grub selection screen type e That will let you edit the kernel boot parameter line (for this boot only). You need to find the line that starts “Linux” go to the end of that line and add nomodeset Then press F10 to boot. That should get you to a login screen. We will have everything installed including the NVIDIA driver before the next boot so this should be the only time you have to do this and there should be no reason to have to add nomodeset permanently.

nomodeset

nomodeset end

You should now have a login prompt waiting for you!

Step 6) Run updates, add your desktop of choice

Login to your system. Now we’ll add a bunch of the good stuff!

First run updates,

sudo apt-get update
sudo apt-get dist-upgrade

You will probably get a kernel update but don’t reboot yet!

Select and install your desktop of choice

We will use a very handy command that is is installed by default with a server install.

sudo tasksel

With tasksel you you have a lot of options available. It’s a really nice tool! The image below show some of the tasksel options. I have selected “Ubuntu MATE desktop”. That’s my current favorite Linux desktop.
tasksel image

After you make your selection hit return. In the screen-shot below you will see that it is installing 1420 new packages!
1420 packages

When those packages finish downloading and installing you will be able to reboot into your new desktop, but …

Don’t reboot yet! We want to get the NVIDIA display driver installed BEFORE we boot to a GUI desktop.

Step 7) Install the NVIDIA display driver

There are a few methods for installing the proprietary NVIDIA display driver. For Ubuntu installs the one I recommend is to install the latest driver by adding the the graphics driver ppa repository. This is well maintained and you will be able to get the latest driver (or older ones if needed). I recommend you use this method even if you want to do a NVIDIA CUDA install as I describe in a section near the end of this post.

The command to add graphics-drivers ppa is,

sudo add-apt-repository ppa:graphics-drivers/ppa

Before you install the driver is good to be certain dkms (dynamic kernel module support) is installed. I also install a few extra packages that I know I will want now too.

sudo apt-get install dkms synaptic emacs build-essential

I think Synaptic is a great package management interface, build-essential will install a good base of development packages and yes, I like emacs! 🙂

Now to install the driver,

sudo apt-get update
sudo apt-get install nvidia-387

That would install the latest driver version 387 (as of this writing). That will work with NVIDIA Volta and lower cards.

This install should be robust. New kernels or drivers should automatically rebuild with dkms.

Now you can reboot!

sudo shutdown -r Now

When your system comes up you should be greeted with a graphical login manager. You will notice that you get a lot of messages on the screen during boot. That’s because we installed from server. A typical desktop install form live media will be configured to hide all of that information from you. Personally I like to see it! You will sometimes see some warnings or errors. Those are almost always artifacts of startup ordering or some such no concerning events. However, if there is a real problem on the system those messages can give you useful information for diagnoses.

Step 8) Optional: enable NetworkManager on install interface

One artifact of doing an install from server is that the network interface that is configured during the install will not be using NetworkManager. This is because the server install doesn’t have NetworkManager! That’s a good thing for a server because you most likely want to manually configure your network. NetworkManager can be useful, especially for systems like laptops where there may be frequent interface switching. NetworkManager is a good system tool and it can be used to give a user other than root the ability to turn on and off the interfaces and it handles wifi well. You can switch your system to NetworkManager control with the following commands (I’m using sed to edit the files but you could do this with a “normal” editor but you do have be root i.e. use sudo)

Move manual network config that happened from server install to NetworkManager control.
This sed line comments out the primary nic interface.

sudo sed -i '/The primary network interface/,/^$/ s/^/#/' /etc/network/interfaces

This line enables NetworkManager for everything

sudo sed -i 's/managed=false/managed=true/' /etc/NetworkManager/NetworkManager.conf

You can reboot after this to check that your network comes up OK.

Step 9) Optional: NVIDIA CUDA install

The NVIDIA display driver in the CUDA 9.1 install repository is nvidia-387 which is the current driver as of this writing. It is the same major version as the driver we installed in Step 7) above. This means that we can do an easy CUDA install from the NVIDIA CUDA repositories even though it will reinstall the display driver. This will make the CUDA install very simple. [Note: that I tried using the CUDA install instead of the ppa driver method in Step 7) and the driver failed to install correctly. I recommend you do your base install as I outlined above including the NVIDIA driver install from the ppa before you do the CUDA install.]

Note: When I am scripting an automated system install I add CUDA manually i.e. using the “run” files. That is a convenient way to install multiple version of CUDA on the same system too. However, going forward I will be recommending using docker to support multiple CUDA versions!

Install and configure NVIDIA CUDA

If you have followed the Ubuntu install method in this post you will have a GUI desktop on your system that is working correctly with the NVIDIA display driver. You can do the install easily from your desktop.

  • Get the latest CUDA repo install deb for Ubuntu 16.04 . I recommend using the “deb(network)” package. If you download this file with firefox it should offer to open the file with “GDebi Package Installer”. That is a great utility, go ahead and use it. [If you don’t want to use GDebi or don’t have the option, then you can just download the deb file and install with dpkg i.e. 

    sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb

    ]

  • Next you will want to download and install the signing key for the repo.

On the NVIDIA CUDA download page there are instructions that show this 

sudo apt-key adv --fetch-keys http://developer...

Uggg! that is using http not https for downloading the keys. That kind of bugs me! That command uses curl by default and curl is a little out of date on Ubuntu 16.04 I recommend that you install gnupg-curl. This will allow that key download command to work with https.

sudo apt-get install gnupg-curl

then do

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub

  • Now install CUDA

sudo apt-get update
sudo apt-get install cuda

  • Configure CUDA for use on your system

    • I like to configure CUDA for all users. To do this create the file (use sudo and a text editor of your choice)

    /etc/profile.d/cuda.sh

    with the following content,

    export PATH=$PATH:/usr/local/cuda/bin
    export CUDADIR=/usr/local/cuda
    export GLPATH=/usr/lib/

    Also create the file,

    /etc/ld.so.conf.d/cuda.conf

    and add the line,

    /usr/local/cuda/lib64

    Then run

    sudo ldconfig

The next time you login CUDA will be on your path and ready to use. If you want to load that environment right now without logging out the just do, 

source /etc/profile.d/cuda.sh

.

Happy computing –dbk

Tags: , , ,