RCS Centrally Managed Python Packages

There are many packages that extend Python’s standard functionality. RCS maintains installations of Python for different versions and common packages for each version. While we try to satisfy all requests for specific Python packages, we may not be able to meet your needs for various reasons. When this is the case, we provide directions below to help you install Python packages on your own. You may request installation from RCS if you have difficulty installing or managing packages yourself.

PIP and the Python Package Index (PyPI)

The Python Package Index (PyPI) contains thousands of complete Python packages and is the official repository for Python and python.org. The “pip” utility is the recommended tool for searching, downloading and installing packages from the PyPI. The strategies laid out below will install personal versions of specific libraries. If you are interested in configuring entire Python environments, we suggest looking at virtual environments (virtualenv) as well.

Install to your home directory

Note: Due to limited quotas and other system characteristics, the recommended install location for Python libraries is /projectnb. Please continue reading through Install to a shared location and virtualenv sections, especially when installing multiple libraries.

Installing a package in your home directory is often the easiest install location; it requires less configuration and works out of box. You can install a Python package into your home directory using “pip” with the “--user” option. This option will install the library into ~/.local” using the standard Python directory structure.

scc1$  pip install --user packagename

The Python directory structure separates packages by Python version, where X.Y are the major and minor version. This allows libraries for different Python versions to be installed alongside one another. Python will search these paths automatically and there is no need to modify the $PYTHONPATH variable.

Install location:

~/.local/lib/pythonX.Y/site-packages/packagename

Sometimes, Python packages contains executables, scripts, or wrappers. If this is the case, they will be placed in ~/.local/bin. This path is not searched automatically and should be added to $PATH before calling the executable components of the packages from the commandline.

scc1$  export PATH=~/.local/bin:$PATH

 

Install to a shared location

You can install packages in a specified location using “pip” with the “--prefix” option. This is useful for creating a repository of shared Python packages in a project space such as /projectnb/. The /projectnb space is preferred to the home directory and /project space, where having many files will count against a stricter quota and where the library can be shared by members of the project. The library will be installed into the specified location using the standard Python directory structure.

scc1$  pip install --prefix=/projectnb/projectname/pythonlibs packagename

 

Install location:

/projectnb/projectname/pythonlibs/lib/pythonX.Y/site-packages/packagename

 

The install location for this package is unknown to Python and cannot be used unless it is added to the library search path called $PYTHONPATH. We can export this as an environment variable.

scc1$  export PYTHONPATH=/projectnb/projectname/pythonlibs/lib/pythonX.Y/site-packages/:$PYTHONPATH

 

Additionally, the install path for any library executables, scripts, or wrappers should be added to the $PATH variable.

scc1$  export PATH=/projectnb/projectname/pythonlibs/bin:$PATH

Update a PyPI Package

You can update any personally installed PyPI packages using the “pip” command with the “--upgrade” option. To update a centrally installed package for the general purpose installations of python, please submit a ticket to help@scc.bu.edu.

scc1$  pip install --upgrade packagename

Remove a PyPI Package

You can remove any personally installed PyPI packages using the “pip” command with the “--uninstall” option. The uninstall process with show the files to be removed and ask for confirmation. If working with multiple virtual environments or library locations, it is important to check the paths carefully.

scc1$  pip install --uninstall packagename

Install Third-Party and non-PyPI packages

Not all packages are available from PyPI. In fact, many of the newest packages we see in journals are released directly from developers and cannot be installed from on PyPI at all. We can use pip to install these from source (which sometimes doesn’t work) or install them using the included setup.py script.

Very generally, this looks like this:

# Download package and extract source from developer
scc1$  wget https://third.party.dev/packages/source/module.tar.gz
scc1$  tar xf module.tar.gz
scc1$  cd module/
scc1$  ls
setup.py   otherfiles

# Choose an install location, set PYTHONPATH and create the directories
scc1$  export PACKAGEINSTALLDIR=/projectnb/projectname/mypython
scc1$  export PYTHONPATH=$PACKAGEINSTALLDIR/lib/python2.7/site-packages:$PYTHONPATH
scc1$  mkdir -p $PACKAGEINSTALLDIR/lib/python2.7/site-packages

# Install
scc1$  python setup.py install --prefix=$PACKAGEINSTALLDIR

 

Other Package Management Frameworks

Virtual Environments (virtualenv)

The virtualenv framework allows for creation of isolated Python installations. It is the traditional tool developers use to quickly configure custom Python installations and switch between functioning environments for distinct projects. When used with pip, it is very easy to build environments and distribute “requirements.txt” files for rebuilding these environments on different systems.

More information about virtualenv can be found on our Python Virtual Environments page.

 

Anaconda Python (conda)

Anaconda is a free package and environment manager for Python distributed by Continuum Analytics. It is very similar to virtualenv and has gained traction for ease of packaging and replicating modules or entire Python environments on different systems. The distribution includes a set of core Python packages and additional user packages can be installed from remote “channels”.

More information about Anaconda can be found on our Anaconda Python page.

 

Easy Install (easy_install)

The easy_install framework was released in 2004 as part of the setuptools module. The easy_install.py script installs a Python “egg” instead of building the package from source or using the setup.py script. These “eggs” are pulled from PyPI, just like pip, but put in place directly. This results in a minimal install which often misses associated scripts and complicates package dependencies. While not officially deprecated, it is less commonly used than pip or conventional setup.pyinstallers.