The Best Way to Install TensorFlow with GPU Support on Windows 10 (Without Installing CUDA)

Posted on by

A NEW VERSION OF THIS POST IS NOW AVAILABLE

How to Install TensorFlow with GPU Support on Windows 10 (Without Installing CUDA) UPDATED!


PLEASE USE THE NEW GUIDE

A couple of weeks ago I wrote a post titled Install TensorFlow with GPU Support on Windows 10 (without a full CUDA install). What you are reading now is a replacement for that post.

In that older post I couldn’t find a way around installing at least some of CUDA. I tried to minimize it by installing only the essential DLL’s to make things work. It did require making changes to the “User PATH” which I would rather not have done. I have now found a way around this using only Anaconda Python packages.

I was doing a pytorch install on Linux and Windows 10 and I noticed that they were adding the needed CUDA and cuDNN libraries with a separate, single, Anaconda package (pytorch/cuda90 or 91). I tried that package with TensorFlow but it didn’t work. However, I searched Anaconda Cloud and found Anaconda supported packages for CUDA 9.0 and cuDNN 7 that did work! All I had to do was install these two packages in the conda virtual environment for TensorFlow. [ The Linux TensorFlow Anaconda package includes CUDA and cuDNN internally in the same package. ]

The focus here is to get a good GPU accelerated work environment for TensorFlow (with Keras and Jupyter notebook) up and running for Windows 10. You will not need to install CUDA for this!

In this post I’ll walk you through the best way I have found so far to get a good TensorFlow work environment on Windows 10 including GPU acceleration. I’ll also go through setting up Anaconda Python and create an environment for TensorFlow and how to make that available for use with Jupyter notebook. As a “non-trivial” example of using this setup we’ll go through training LeNet-5 with Keras using TensorFlow with GPU acceleration. We’ll get a setup that is 18 times faster than using the CPU alone.

Python Environment Setup with Anaconda Python

I highly recommend you use Anaconda Python. If you need some arguments for using Python take a look at my post Should You Learn to Program with Python. For arguments on why you should use the Anaconda Python distribution see, How to Install Anaconda Python and First Steps for Linux and Windows. The best reason for using Anaconda Python in the context of installing GPU accelerated TensorFlow is that by doing so you will not have to do a CUDA install on the system.

Anaconda is focused toward data-science and machine learning. It installs cleanly on your system in a single directory so it doesn’t make a mess in your systems application and library directories. It is also performance optimized and links important numerical packages like numpy to Intel’s MKL.

Install Anaconda Python

1) Download and check the installer

  • Go to the Anaconda downloads page https://www.anaconda.com/downloads and get the Python 3.6 version.

  • It’s good practice to check the file hash to be sure you got a good copy. [ I have to admit that I don’t always do this myself. ]

    • Open Powershell and cd to the directory where you downloaded the Anaconda installer exe file. In my case that is the Downloads directory.
cd Downloads

Then run

 Get-FileHash .\Anaconda3-5.2.0-Windows-x86_64.exe -Algorithm SHA256

Run the installer

Since you have Powershell open in the directory with the Anaconda installer exe file you can start it by just typing it’s name (type A and hit tab to expand the name) and hitting return. [ You could alternatively just double click on the download install exe. from your file browser. ]

.\Anaconda3-5.1.0-Windows-x86_64.exe

The installer GUI should now be running.

  • You will be asked to accept a license agreement …
  • “Select Install Type” I recommend you chose “Just Me” since this is part of your personal development environment.
  • “Chose Install Location” I recommend you keep the default which is at the top level of you user directory.
  • “Advanced Installation Options”
    Advanced install opts
    My recommendation is to check both boxes. Make Anaconda Python 3 your default Python. And, as a developer you really should be aware of your PATH environment variable. So yes, go ahead and let the installer add the Anaconda bin directory to your PATH. If you haven’t looked at your environment variables in awhile you should have a look. Do a search from the Windows menu for “environment variables”. You should find a settings panel that will show your account environment and system wide environment. After the Anaconda install you will see that its application and library directories have been prepended to your user PATH. We’ll look at it, and modify it, after installing the CUDA libraries.

VSCode?

  • Next you will be asked if you want to install Microsoft VSCode. VSCode is a really good editor and it is available for free on Windows, Linux and MacOS. However, if you are interested in trying it out I would recommend that you go to the VSCode website and check it out first. If you think you want to try it, then go ahead and download it and install it yourself. I like VSCode but I usually use the Atom editor which also runs on Windows, Linux and MacOS. If you are checking out editors I recommend you try both of these as well as Sublime Text. They are all great editors!

Check your install

If you still have Powershell open you will need to close it and restart it so that it will re-read your environment variables and pick up your PATH which now includes the Anaconda Python directories. With Powershell reopened you can check that you now have Anaconda Python 3 as your default Python.

python --version

Python 3.6.5 :: Anaconda custom (64-bit)

Update your base Anaconda packages

conda is a powerful package and environment management tool for Anaconda. We’ll use conda from Powershell to update our base Python install. Run the following commands. It may take some time to do this since there are a lot of modules to update.

conda update conda
conda update anaconda
conda update python
conda update --all

That should bring your entire base Anaconda install up to the latest packages. (Anaconda 5.2 had just been released when I wrote this and nearly everything was fully up-to-date.)

Anaconda Navigator

There is a GUI for Anaconda called anaconda-navigator. I personally find it distracting/confusing/annoying and prefer using conda from the command-line. Your taste may differ! … and my opinion is subject to change if they keep improving it. If you are new to Anaconda then I recommend you read up on conda even (or especially!) if you are thinking about using the “navigator” GUI.

Create a Python “virtual environment” for TensorFlow using conda

You should set up an environment for TensorFlow separate from your base Anaconda Python environment. This keeps your base clean and will give TensorFlow a space for all of it’s dependencies. It is in general good practice to keep separate environments for projects especially when they have special package dependencies. Think of it as a separate “name-space” for your project.

There are many possible options when creating an environment with conda including adding packages with specific version numbers and specific Python base versions. This is sometimes useful if you want fine control and it also helps with version dependencies resolution. Here we will keep it simple and just create a named environment and then activate that environment and install the packages we want inside of that.

From a command line do,

conda create --name tf-gpu

I named the environment ‘tf-gpu’ but you can use any name you want.

“activate” the environment

Now activate the environment, (I’ll show my full terminal prompt and output instead of just the commands)

Note: for some reason Powershell will not run the “activate” script! You will need to start “CMD” shell to do this. You can start CMD shell from Powershell (notice how the “PS” that was at the beginning of the Powershell prompt disappears). Having to switch to CMD is an annoyance but you can easily switch back and forth in a Powershell window

PS C:\Users\don> cmd
Microsoft Windows [Version 10.0.16299.461]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\don> conda activate tf-gpu

(tf-gpu) C:\Users\don>

You can see that my CMD shell prompt is now preceded by the the name of the environment (tf-gpu). Any conda package (or pip) installs will now be local to this environment.

Install TensorFlow-GPU from the Anaconda Cloud Repositories

THIS SECTION IS OUT OF DATE!!!

Just do the following to install, the now officially supported, TF and Keras versions Do not install aaronzs build or the cudatoolkit and cudnn 

conda install tensorflow-gpu keras-gpu

That’s it! now go to the next section and do the first test…

My preference would be to install the “official” Anaconda maintained TensorFlow-GPU package like I did for Ubuntu 18.04, unfortunately the Anaconda maintained Windows version of TensorFlow is way out-of-date (version 1.1). There is a current CPU-only version 1.8 for Windows but we want GPU acceleration.

A search for “tensorflow” on the Anaconda Cloud will list the available packages from Anaconda and the community. There is a package “aaronzs / tensorflow-gpu 1.8.0” listed near the top that has builds for Linux and Windows. This is the only up-to-date package I know of that is working correctly with Windows 10. This package was built by, and is being nicely maintained by, Aaron Sun. You can check out his GitHub page for the project.

Lets install TensorFlow with GPU acceleration in this conda environment.

(tf-gpu) C:\Users\don> conda install -c aaronzs tensorflow-gpu

Now, we can do the CUDA and cuDNN dependencies,

(tf-gpu) C:\Users\don> conda install -c anaconda cudatoolkit
(tf-gpu) C:\Users\don> conda install -c anaconda cudnn

Note that I explicitly use the -c flag to specify the “anaconda” “channel”. That would be default if you leave out the channel name but in this case I wanted to be explicit about where the packages came from. The links are, cudatoolkit current is 9.0 and cudnn current is 7.1.4. You should check version numbers when you install.

That’s it! You do not need to do a CUDA install on your system.

Check That TensorFlow is working with your GPU

Close any Powershell or CMD shells you had open and reopen one. You need to do that so that your new PATH settings get read in. You can use a CMD shell to activate your tf-gpu environment start Python and run the following lines,

>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))

My session including the output looked like this, (there was a long delay during this “first run” session startup )

PS C:\Users\don> cmd
Microsoft Windows [Version 10.0.16299.461]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\don> activate tf-gpu

(tf-gpu) C:\Users\don>python
Python 3.6.5 |Anaconda custom (64-bit)| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow')
>>> sess = tf.Session()
2018-06-01 16:37:57.666250: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2018-06-01 16:37:57.967130: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.62GiB
2018-06-01 16:37:57.975868: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-01 16:40:10.162112: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-01 16:40:10.168554: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929]      0
2018-06-01 16:40:10.171214: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0:   N
2018-06-01 16:40:10.174162: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6400 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
>>> print(sess.run(hello))
b'Hello, TensorFlow'
>>>

Yea! PATH’s are correct and everything is working. You can see that it is has GPU support.

Next we will do something a little more useful and fun with Keras, after we configure Jupyter notebook to use our ‘tf-gpu’ environment.

Create a Jupyter Notebook Kernel for the TensorFlow Environment

You can work with an editor and the command line and you often want to do that, but, Jupyter notebooks are great for doing machine learning development work. In order to get Jupyter notebook to work the way you want with this new TensorFlow environment you will need to add a “kernel” for it.

With your tf-gpu environment activated do,

(tf-gpu) C:\Users\don>conda install ipykernel

Now create the Jupyter kernel,

(tf-gpu) C:\Users\don>python -m ipykernel install --user --name tf-gpu --display-name "TensorFlow-GPU"

With this “tf-gpu” kernel installed, when you start Jupyter notebook you will now have an option to to open a new notebook using this kernel.

Jupyter kernel for TF

An Example using Keras with TensorFlow Backend

In order to check everything out lets setup LeNet-5 using Keras (with our TensorFlow backend) using a Jupyter notebook with our “TensorFlow-GPU” kernel. We’ll train the model on the MNIST digits data-set and then open TensorBoard to look at some plots of the job run.

Install Keras

With the tf-gpu environment activated do,

(tf-gpu) C:\Users\don\projects>conda install keras-gpu

You now have Keras installed utilizing your GPU accelerated TensorFlow.

Launch a Jupyter Notebook

With the tf-gpu environment activated start Jupyter,

(tf-gpu) C:\Users\don>jupyter notebook

From the ‘New’ drop-down menu select the ‘TensorFlow-GPU’ kernel that you added (as seen in the image in the last section). You can now start writing code!

MNIST example

Following are Python snippets you can copy into cells in your Jupyter notebook to setup and train LeNet-5 with MNIST digits data.

Import dependencies

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.layers import Flatten,  MaxPooling2D, Conv2D
from keras.callbacks import TensorBoard

Load and process the MNIST data

(X_train,y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(60000,28,28,1).astype('float32')
X_test = X_test.reshape(10000,28,28,1).astype('float32')

X_train /= 255
X_test /= 255

n_classes = 10
y_train = keras.utils.to_categorical(y_train, n_classes)
y_test = keras.utils.to_categorical(y_test, n_classes)

Create the LeNet-5 neural network architecture

model = Sequential()
model.add(Conv2D(32, kernel_size=(3,3), activation='relu', input_shape=(28,28,1)) )
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Flatten())          
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(n_classes, activation='softmax'))

Compile the model

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

Set log data to feed to TensorBoard for visual analysis

tensor_board = TensorBoard('./logs/LeNet-MNIST-1')

Train the model

model.fit(X_train, y_train, batch_size=128, epochs=15, verbose=1,
          validation_data=(X_test,y_test), callbacks=[tensor_board])

The results

After running that training for 15 epochs the last epoch gave,

Epoch 15/15
60000/60000 [==============================] - 6s 102us/step - loss: 0.0192 - acc: 0.9936 - val_loss: 0.0290 - val_acc: 0.9914

Not bad! Training accuracy 99.36% and Validation accuracy 99.14%

Look at the job run with TensorBoard

You will need “bleach” for TensorBoard so install it first,

(tf-gpu) C:\Users\don>conda install bleach

Start TensorBoard

 (tf-gpu) C:\Users\don\projects>tensorboard --logdir=./logs --port 6006

It will give you an address similar to http://stratw:6006 Open that in your browser and you will be greeted with (the wonderful) TensorBoard. These are the plots it had for that job run,
TensorBoard output

That was a model with 1.2 million training parameters and a dataset with 60,000 images. It took 1 minute and 26 seconds utilizing the NVIDIA GeForce 1070 in my laptop system! For reference it took 26 minutes using all cores at 100% of the Intel 6700HQ CPU in that system. That’s an 18 fold speedup on the GPU!

Happy computing! –dbk

Tags: , , , ,