Requirements¶
If you wish to take advantage of GPU Acceleration, the following are required:
Installation¶
Certain pre-requisites must be installed:
Pre-requisites¶
GPU Acceleration requires CUDA 8.0 and cuDNN 6.0 for CUDA 8.0.
- It is often easier to CUDA install from the NVIDIA site on Linux systems.
- You will need to sign up for the NVIDIA Developer Program to download cudNN.
During the installation process, Montblanc will inspect your CUDA installation to determine if a GPU-supported installation can proceed. If your CUDA installation does not live in
/usr
, it helps to set a number of environment variables for this to proceed smoothly. For example, if CUDA is installed in/usr/local/cuda-8.0
and cuDNN is unzipped into/usr/local/cudnn-6.0-cuda-8.0
, run the following on the command line or place it in your.bashrc
# CUDA 8 $ export CUDA_PATH=/usr/local/cuda-8.0 $ export PATH=$CUDA_PATH/bin:$PATH $ export LD_LIBRARY_PATH=$CUDA_PATH/lib64:$LD_LIBRARY_PATH $ export LD_LIBRARY_PATH=$CUDA_PATH/extras/CUPTI/lib64/:$LD_LIBRARY_PATH # CUDNN 6.0 (CUDA 8.0) $ export CUDNN_HOME=/usr/local/cudnn-6.0-cuda-8.0 $ export C_INCLUDE_PATH=$CUDNN_HOME/include:$C_INCLUDE_PATH $ export CPLUS_INCLUDE_PATH=$CUDNN_HOME/include:$CPLUS_INCLUDE_PATH $ export LD_LIBRARY_PATH=$CUDNN_HOME/lib64:$LD_LIBRARY_PATH # Latest NVIDIA drivers $ export LD_LIBRARY_PATH=/usr/lib/nvidia-375:$LD_LIBRARY_PATH
If the installer cannot find CUDA it will only install the CPU version of tensorflow and only compile multi-threaded CPU operators.
casacore and the measures found in casacore-data. Gijs Molenaar has kindly packaged this on Ubuntu/Debian style systems.
On Ubuntu 14.04, these packages can be added from the radio astronomy PPA :
$ sudo apt-get install software-properties-common $ sudo add-apt-repository ppa:radio-astro/main $ sudo apt-get update $ sudo apt-get install libcasacore2-dev casacore-data
On Ubuntu 16.04 these packages can be added from the kernsuite PPA:
$ sudo apt-get install software-properties-common $ sudo add-apt-repository ppa:kernsuite/kern-1 $ sudo apt-get update $ sudo apt-get install casacore-dev casacore-data
Otherwise, casacore and the measures tables will need to be manually installed.
Check that the python-casacore and casacore dependencies are installed. By default python-casacore builds from pip and therefore from source. To succeed, library dependencies such as
libboost-python
must be installed beforehand. Additionally, python-casacore depends on casacore. Even though kernsuite installs casacore, it may not install the development package dependencies (headers) that python-casacore needs to compile.
Installing the package¶
Set the CUDA_PATH
so that the setup script can find CUDA:
$ export CUDA_PATH=/usr/local/cuda-8.0
If nvcc
is installed in /usr/bin/nvcc
(as in a standard Ubuntu installation)
or somewhere on your PATH
, you can leave CUDA_PATH
unset. In this case
setup will infer the CUDA_PATH as /usr
It is strongly recommended that you perform the install within a
Virtual Environment.
If not, consider adding the --user
flag to the following pip and
python commands to install within your home directory.
$ virtualenv $HOME/mb
$ source virtualenv $HOME/mb/bin/activate
(mb) $ pip install -U pip setuptools wheel
Then, run:
(mb) $ pip install --log=mb.log git+git://github.com/ska-sa/montblanc.git@master
Installing the package in development mode¶
Clone the repository, checkout the master branch and pip install montblanc in development mode.
(mb) $ git clone git://github.com/ska-sa/montblanc.git
(mb) $ pip install --log=mb.log -e $HOME/montblanc
Possible Issues¶
Montblanc doesn’t use your GPU, install the GPU version of tensorflow or compile GPU tensorflow operators. The installation process attempts to find your CUDA install location. It will log information about where it thinks this is and which GPU devices you have installed. Check the install log generated by the
pip
commands given above to see why this fails, searching for “Montblanc Install” entries.It is possible to see if the GPU version of tensorflow is installed by running the following code in a python interpreter:
import tensorflow as tf with tf.Session() as S: pass
If tensorflow knows about your GPU it will log some information about it:
2017-05-16 14:24:38.571320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with properties: name: GeForce GTX 960M major: 5 minor: 0 memoryClockRate (GHz) 1.176 pciBusID 0000:01:00.0 Total memory: 3.95GiB Free memory: 3.92GiB 2017-05-16 14:24:38.571352: I tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0 2017-05-16 14:24:38.571372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y 2017-05-16 14:24:38.571403: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 960M, pci bus id: 0000:01:00.0)
cub 1.6.4. The setup script will attempt to download this from github and install to the correct directory during install. If this fails do the following:
$ wget -c https://codeload.github.com/NVlabs/cub/zip/1.6.4 $ mv 1.6.4 cub.zip $ pip install -e .
python-casacore is specified as a dependency in setup.py. If installation fails here:
- Check that the python-casacore dependencies are installed.
- You will need to manually install it and point it at your casacore libraries.