Intel Python PIP Installation

Note: this setup is now obsolete in favour of python-docker.

How to install custom intel-python modules using pip.

TL;DR

Install packages with intel-pip install --prefix /opt PACKAGE_NAME

Problem

By default pip installs packages in system path /usr/lib. This is wrong for following reasons:

/usr/lib is a system path which is handled exclusively by Ubuntu packages
/usr/lib is out of reach for intel-python, it searches for custom packages from /opt/lib

Solution

Use the --prefix /opt option to install packages at a different location. This will install python packages in /opt/lib/python3.6 and any binaries in /opt/bin.

Example Usage: Installing pylint

To install pylint run, intel-pip install --prefix /opt pylint.

First run it as normal user (non-root) to understand the package dependencies. This will fail in the final step due to file permissions, but it is a test to ensure that the package builds without errors.

After this command succeeds building packages, rerun using sudo.

rsubr@mike:~$ intel-pip install --prefix /opt pylint
Collecting pylint
  Using cached https://files.pythonhosted.org/packages/3f/88/e0c9a506255ae3915ac1f183a13853d5b4a3e435972d0df6ea190d5d1e96/pylint-1.9.1-py2.py3-none-any.whl
Requirement already satisfied: six in /opt/intel/intelpython3/lib/python3.6/site-packages (from pylint)
Collecting isort>=4.2.5 (from pylint)
  Using cached https://files.pythonhosted.org/packages/1f/2c/22eee714d7199ae0464beda6ad5fedec8fee6a2f7ffd1e8f1840928fe318/isort-4.3.4-py3-none-any.whl
Collecting mccabe (from pylint)
  Using cached https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Collecting astroid<2.0,>=1.6 (from pylint)
  Using cached https://files.pythonhosted.org/packages/7f/4f/162e1c859f59031ee7d3ff48ca6a3eaa699e4b8bfcb4f2988b5677d74605/astroid-1.6.4-py2.py3-none-any.whl
Collecting lazy-object-proxy (from astroid<2.0,>=1.6->pylint)
  Using cached https://files.pythonhosted.org/packages/65/1f/2043ec33066e779905ed7e6580384425fdc7dc2ac64d6931060c75b0c5a3/lazy_object_proxy-1.3.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting wrapt (from astroid<2.0,>=1.6->pylint)
Installing collected packages: isort, mccabe, lazy-object-proxy, wrapt, astroid, pylint
Exception:
Traceback (most recent call last):
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/opt/intel/intelpython3/lib/python3.6/site-packages/pip-9.0.3-py3.6.egg/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/opt/intel/intelpython3/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/lib/python3.6/site-packages/isort'
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.