What has changed with Python?

The Linux operating system on the SCC has been upgraded from CentOS 6 to CentOS 7. This page explains what you need to know about changes to the Python language on the SCC since this transition.

System Default Python

The operating system version of Python has been updated to 2.7.5 under CentOS 7. This is the default version you will get if you do not load any Python modules. It was version 2.6.6 under CentOS 6. As the system version is not updated with newer Python libraries (such as numpy), we recommend always using a Python module on the SCC.

Python Modules

The python versions have been updated and also the naming convention of python modules has changed. The python module name will now include the major version number such as python2 and python3. For example, the default CentOS 6 python/2.7.13 module is not available on CentOS 7. The closest equivalent on CentOS 7 is python2/2.7.16. Currently, the available Python modules are python2/2.7.16, python3/3.6.5, and python3/3.7.0. Python 3.7 is the newest major release of Python and it is not yet widely supported by software that is based on Python, e.g. TensorFlow. Python version 3.6.x is recommended for new projects unless the new language features in 3.7 are required. Python-based modules (such as QGIS) have been installed to use Python 3.6.x as 3.7 is not yet supported. To see the available Python language modules use these commands:

module avail python2
module avail python3
Note that you can only load one python module at a time, even if they are different versions. Do not try to load both a python2 module and a python3 module at the same time or you will experience issues.

Anaconda Python

The Anaconda Python modules have been updated to the following versions: anaconda2/5.2.0 and anaconda3/5.2.0. To see the available Anaconda Python modules use these commands:

module avail anaconda2
module avail anaconda3

Intel Python

The Intel Python modules have been updated to newer versions. The Intel Python distribution has been custom compiled and optimized for higher performance. Selecting this version can sometimes significantly improve the performance of Python programs. To see the available Intel Python modules use these commands:

module avail python2-intel
module avail python3-intel

Troubleshooting

Now that we have upgraded to CentOS 7, make sure that your Python codes are running without errors and returning results consistent with the earlier CentOS 6 environment.

  • Update module load commands
    Many modules have been updated to newer versions under CentOS 7. Test out your module load commands used in any batch scripts or tests and update the scripts as necessary. If a required module is not available under CentOS 7 please contact us at help@scc.bu.edu and it will be installed as soon as possible.
  • Python and Intel Python modules
    The majority of Python scripts on the SCC will work the same way under CentOS 7 as they did under CentOS 6. Some Python libraries that involve compiled code (typically C or C++) may need to be re-compiled if they were installed using pip install --user. This is best tested on a case-by-case basis. Due to the way Python versions 2.x handle Unicode text between the two versions of the operating system this is more likely to occur with programs using the python2 modules. If unexpected errors occur with your installed Python libraries, they will need to be recompiled and reinstalled for CentOS 7.
  • Anaconda modules
    The majority of Python scripts on the SCC will work the same way under CentOS 7 as they did under CentOS 6 when using the Anaconda Python modules. However, due to the fact that Anaconda Python largely downloads pre-compiled code some Python libraries that involve compiled code (typically C or C++) may not be compatible. Users should test their code before submitting a job. If there are compatibility issues with a particular conda environment, then it will have to be reinstalled for CentOS 7. Instructions on reinstalling conda environments are provided on our Python Anaconda page.