Skip to content

setup_environments.md

Latest commit

 

History

History
168 lines (113 loc) · 6.23 KB

setup_environments.md

File metadata and controls

168 lines (113 loc) · 6.23 KB
Raw

Overview

This repository provides instructions about setting up a number of Python environments (using Anaconda) with different configurations for Scientific Computing (particularly machine learning and deep learning). Using it, you can set up the following conda environments:

file name environment name description
mlredux.yml mlredux A Python environment based on 3.5.2 (for machine learning)
scratchpad.yml scratchpad A Python 3.5.2 environment that has a lot of libraries for doing a lot of things
cv.yml cv A Python 3.5.2 environment for computer vision
greyatom.yml greyatom A Python 2.7.x environment for the greyatom curriculum
bdap.yml bdap A Python 3.5.2 environment for bdap curriculum (except for deep learning and computer vision, for Spark, Machine Learning (sklearn) etc)
py27.yml py27 A Python environment based on 2.7.x (for machine learning)
py35.yml py35 A Python environment based on 3.5.2 (for machine learning)
py36.yml py36 A Python environment based on 3.6.x (for machine learning)
py36tf.yml py36tf A Python environment based on 3.6.x (for deep learning using tensorflow, keras)
py36tfgpu.yml py36tfgpu A Python environment based on 3.6.x (for gpu-driven deep learning using tensorflow, keras)

For more details on the exact packages and versions being set up, you can look at the correcponding .yml files.

What is Anaconda?

From the Anaconda docs:

Conda is an open source package management system and environment management system for installing multiple versions of software packages and their dependencies and switching easily between them. It works on Linux, OS X and Windows, and was created for Python programs but can package and distribute any software.

Why Anaconda?

Here are a few advantages of using Anaconda:

  • Set-up process is mostly uniform across platforms
  • No risk of breaking packages required by the operating system
    • user-level installation
    • admin privileges not required
  • We don't have to figure out package compatibility any more
  • Makes life bearable on Windows platform

Here is a discussion thread on reddit that talks about it in more detail.

Setting Up with Anaconda

  1. Install Anaconda
  2. Install necessary packages

Notes

  • In the following steps, we install the environment named py27 using the file py27.yml.
    • To install a different environment, change the filename appropriately in the set up instructions that follow (use py36.yml instead of py27.yml, for example).
    • If you plan to use gpus for deep learning, please note that you have to install your graphics card drivers, CUDA and cuDNN beforehand. The environments using gpu versions of packages have gpu at the end of their names.
  • The instructions have been tested only on Ubuntu and OS X, though they should work on Windows as well (please raise a issue if you hit any snags on your Windows system).

Install Anaconda

Overview

Using Anaconda consists of the following:

  1. Install miniconda on your computer
  2. Create a new conda environment
  3. Each time you wish to work, activate your conda environment

Installation

Download the version of miniconda that matches your system. Make sure you download the version for Python 3.x (3.6 is the latest at the time of writing).

Linux Mac Windows
64-bit 64-bit (bash installer) 64-bit (bash installer) 64-bit (exe installer)
32-bit 32-bit (bash installer) 32-bit (exe installer)

Install miniconda on your machine. Detailed instructions:

Install necessary packages

Setup your the py27 environment.

git clone https://github.com/soumendra/python-machinelearning-setup.git
cd python-machinelearning-setup

If you are on Windows, rename
meta_windows_patch.yml to
meta.yml

Create py27. Running this command will create a new conda environment that is provisioned with all libraries listed above.

conda env create -f py27.yml

In case you want to install another environment, use the appropriate yml file. For example, to install an environment including Python 3.6.x and tensorflow (cpu version), you can run,

conda env create -f py36tf.yml

Verify that the py27 environment was created in your environments:

conda info --envs

Cleanup downloaded libraries (remove tarballs, zip files, etc):

conda clean -tp

Uninstalling

To uninstall the environment:

conda env remove -n py27

Using the Anaconda environment

Now that you have created an environment, in order to use it, you will need to activate the environment. This must be done each time you begin a new working session, i.e., open a new terminal window.

Activate the py27 environment:

OS X and Linux

$ source activate py27

Windows

Depending on shell either:

$ source activate py27

or

$ activate py27

That's it. Now all of the py27 libraries are available to you. You can start a Jupyter Notebook with:

jupyter notebook

To exit the environment when you have completed your work session, simply close the terminal window.