Install CUDA ToolKit
The first step in our process is to install the CUDA ToolKit, which is what gives us the ability to run against the the GPU CUDA cores. Because TensorFlow is very version specific, you'll have to go to the CUDA ToolKit Archive to download the version that works with TF, instead of the standard downloads page which only has the current version.
At the time of this article, the correct version of the CUDA ToolKit is 8.0 GA2 (Feb. 2017 Release Date). When in doubt, check the TensorFlow Documentation Page for additional version information.
Once you've got the CUDA ToolKit, begin the installation. Be aware that the ToolKit contains more software than just the CUDA drivers. For example, it also contains the standard NVIDIA, PhysX, and 3D drivers, plus the GeForce Experience software, etc. Personally, I maintain all of that software separately, so I deselect it during the installation process.
Next, we'll need to grab the cuDNN (CUDA Deep Neural Network) Library) from the NVIDIA site. This software provides the GPU-accelerated library functions needed by TensorFlow for building Neural Networks. As a side note, you'll have to create an account to download this software.
Again, TensorFlow is very version specific sensitive, so at the time of this article, the correct version is cuDNN 6.0 for CUDA 8.0. When you click the
Download button on the cuDNN page, select that version from the list. Do not select the most recent version!
This software just comes in a zip file, which makes the installation both simpler and complex. Simpler, because you can just pick wherever you want the software to go, but more complex because if it goes somewhere outside the
PATH environment variable, you'll need to include it in the
Typically, I place the cuDNN directory adjacent to the CUDA directory inside the NVIDIA GPU Computing Toolkit directory (
C:\Program Files\NVIDIA GPU Computing Toolkit\cudnn_8.0_6.0). This makes it easy to swap out the cuDNN software or the CUDA software as needed, but it does require you to add the cuDNN directory to the
PATH environment variable.
Optionally, you can also unzip the cuDNN software directly into the CUDA directory (default location:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0). This has the advantage of not having to change the
PATH variable, but it does co-mingle the CUDA and cuDNN software into a single environment, making upgrades more difficult in the future.
Once you've got the CUDA and cuDNN software installed, you'll want to check the environment variables to make everything is in order. During the installation of the CUDA ToolKit, it should have added several new environment variables (such as
CUDA_PATH), as well as adding the CUDA path to the
PATH variable. For the cuDNN software, if you've placed it in the same directory as the CUDA software directory, then no additional changes are required. If you've placed it adjacent to the CUDA directory, then you'll need to add the cuDNN path to the
There are plenty of videos out there on how to edit environment variables, so I won't explain it here. Please check out some videos on YouTube for directions.
Install Anaconda Python
Since TensorFlow requires Python, it seems wise to use a distribution like Anaconda, which includes a package management system called
Download and install the Python 3.6 version from Anaconda.
Create Conda Environment
Now comes the fun part. We have an installation of Anaconda Python, but we'll want to create an environment that is specific to TensorFlow. Open up a terminal window (personally, I quite like ConEmu) and enter the following command:
conda create -n tensorflow python=3.6
Next up, we'll want to activate that
tensorflow Python environment so we can add the TensorFlow software using the
activate tensorflow pip install --ignore-installed --upgrade tensorflow-gpu
Congratulations! You've got TensorFlow installed! Probably! Wait, what?!
TensorFlow can be a bit tricky, so we're going to have to test the installation to see if everything went fine. Fortunately, someone has created an easy to use Python script that does just that!
Head on over to mrry's GitHub account and download the tensorflow_self_check.py. Once it's downloaded, run the following command in the terminal window where you've activated the
If everything goes well and your installation was successful, you'll see this message:
TensorFlow successfully installed. The installed version of TensorFlow includes GPU support.
Huzzah! Okay, now let's get down to business and run some code. Take the code snippet below and copy it into a file named
import sys import numpy as np import tensorflow as tf from datetime import datetime ### argv = type of device and which one ### argv = size of the matrix to operate on device_name = sys.argv shape = (int(sys.argv), int(sys.argv)) if device_name == "gpu": device_name = "/gpu:0" else: device_name = "/cpu:0" with tf.device(device_name): random_matrix = tf.random_uniform(shape=shape, minval=0, maxval=1) dot_operation = tf.matmul(random_matrix, tf.transpose(random_matrix)) sum_operation = tf.reduce_sum(dot_operation) startTime = datetime.now() with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as session: result = session.run(sum_operation) print(result) ### Print the shape, device name and timing print("\n" * 3) print("Shape:", shape, "Device:", device_name) print("Time taken:", datetime.now() - startTime) print("\n" * 3)
Now run that code (in the terminal window where you previously activated the
tensorflow Anaconda Python environment):
python tensorflow_test.py gpu 10000
You'll get a lot of output, but at the bottom, if everything went well, you should have some lines that look like this:
Shape: (10000, 10000) Device: /gpu:0 Time taken: 0:00:01.933932
For comparison, you can change the execution of the
tensorflow_test.py script to use your CPU, which should be several times slower:
python tensorflow_test.py cpu 10000 ... Shape: (10000, 10000) Device: /cpu:0 Time taken: 0:00:12.856383
The finish line is almost in sight! Next up is to install Jupyter. You might say 'This is unnecessary! I've already got Jupyter in my Anaconda installation!' ... well, you'd be wrong. You need an installation of Jupyter inside your Anaconda TensorFlow environment in order to work properly. If you don't perform this step and attempt to use the veresion of Jupyter that was installed with base Anaconda, you'll get an error when you attempt to import TensorFlow in the Python script.
In the same terminal window in which you activated the
tensorflow Python environment, run the following command:
pip install --ignore-installed --upgrade jupyter
Test TensorFlow-GPU on Jupyter
And finally, we test using the Jupyter Notebook ... In the same terminal window in which you activated the
tensorflow Python environment, run the following command:
A browser window should now have opened up. Click the
New button on the right hand side of the screen and select
Python 3 from the drop down. You have just created a new Jupyter Notebook.
We'll use the same bit of code to test Jupyter/TensorFlow-GPU that we used on the commandline (mostly). Take the following snippet of code, and copy it into textbox (aka cell) on the page and then press
import sys import numpy as np import tensorflow as tf from datetime import datetime device_name="/gpu:0" shape=(int(10000),int(10000)) with tf.device(device_name): random_matrix = tf.random_uniform(shape=shape, minval=0, maxval=1) dot_operation = tf.matmul(random_matrix, tf.transpose(random_matrix)) sum_operation = tf.reduce_sum(dot_operation) startTime = datetime.now() with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as session: result = session.run(sum_operation) print(result) print("\n" * 2) print("Shape:", shape, "Device:", device_name) print("Time taken:", datetime.now() - startTime) print("\n" * 2)
The output should be something like this:
Shape: (10000, 10000) Device: /gpu:0 Time taken: 0:00:01.678044
Congratulations, you've got a working installation of TensorFlow with GPU on Windows 10 and running in a Jupyter Notebook.