Using Pip on Debian / Ubuntu — pydagogue 0.2 documentation (2024)

In the old days, it was hard and error-prone to use Python tools rather thanyour Debian / Ubuntu package manager to install Python packages.

One of the big problems was the easy_install program, that installedpackages in a way that could make them particularly difficult to uninstall(see Clean out packages installed with easy_install).

Things started to improve as Pip took over from easy_install as thestandard Python package installer.

They got better still when Pip got a binary installer format – wheels.

The combination of virtualenv, Pip and wheels makes it much easier tomaintain a set of Python environments to develop and test code.

This page is a recipe for setting up these environments on your Debian orUbuntu machine.

Do you really want to do this?¶

Debian and Ubuntu package maintainers put a lot of effort into maintainingbinary .deb installers for common Python packages like numpy, scipy andmatplotlib. For example, to get the standard versions of these for yourDebian / Ubuntu distribution, you can do this:

sudo apt-get install python-numpy python-scipy python-matplotlib

Standard apt-get installation may well be all you need. The versions ofnumpy, scipy, matplotlib for your distribution can be a little behind thelatest version available from pypi (the Python package index). If you want amore recent version of common Python packages, you might also considerinstalling Debian / Ubuntu packages from NeuroDebian. Again, you can use thestandard Debian tools like apt-get to do your installs.

The advantage of always using standard Debian / NeuroDebian packages, is thatthe packages are carefully tested to be compatible with each other.The Debian packages record dependencies with other libraries so you willalways get the libraries you need as part of the install.

If you use Pip to install packages, then you don’t get these guarantees. Ifyou use Pip and run into problems with your Python installation, it will beharder for you to get support from the Debian / Ubuntu community, because youare using an installation method that they do not support, and that is morefragile.

So, consider whether you can get away with the package versions in yourdistribution, maybe by using the most recent packages from NeuroDebian. Ifyou can use these, then you probably should not use the Pip installs I’mdescribing below.

Why you might want to use Pip¶

Although Pip installs are more fragile than Debian / Ubuntu package installs,they do have several advantages. With Pip you can:

get the latest version of the package;
install specific packages into virtualenvs;
install packages that have not yet been built for your distribution.

If you do want Pip installs on your Debian / Ubuntu system¶

The recipe I propose is this:

if you have any easy_install installations, remove them;
install Pip and virtualenvwrapper into your user directories (rather thanthe system directories);
use pip install --user to install packages into your day-to-day defaultPython environment;
install Python packages via Pip, and let Pip wheel caching take care ofkeeping a binary wheel ready for the next time you install this package;
have a very low threshold for using virtualenvs, via virtualenvwrapper.

I suggest you never use Pip to change your system-wide packages – so younever use Pip with sudo. This makes sure your Pip-installed packages donot break your system. To avoid sudo you should always install into youruser directories (via pip install --user) or within virtualenvs (seebelow).

Use Pip `--user` installs for your default environment¶

The --user flag to pip install tells Pip to install packages in somespecific directories within your home directory. This is a good way to haveyour own default Python environment that adds to the packages within yoursystem directories, and therefore, does not affect the system Pythoninstallation.

So, if you install a package like this:

pip install --user mypackage

Install, update Pip using `pip install --user`¶

For these steps to work, you will need the Pip --user install binarydirectory on your path. See above for how to do this. Check that your--user binary directory is on the path with:

echo $PATH

The output should contain something like /home/your-user/.local/bin.

You will need Pip version >= 6.0 in order to get Pip wheel caching. This is akiller Pip feature, that means that you only build wheels from source once,the first time you install a package. Pip then caches the wheel so you usethe cached version next time you do an install.

First install the latest version of Pip into your user account by followingthe instructions at install Pip with get-pip.py:

curl -LO https://bootstrap.pypa.io/get-pip.pypython get-pip.py --user

If you are using both python 2 and python 3 versions, do the installation forboth versions, installing last for the Python version that you want to own thePip command, e.g:

# Intall pip for Python 2 installspython get-pip.py --user# Upgrade Pip for Python 3 installs (this one owns "pip" now)python3 get-pip.py --user

Now check the Pip version is >= 6.0:

pip --version

If you installed for both Python 2 and Python 3:

pip2 --versionpip3 --version

Check you are picking up the --user Pip by looking at the output of:

which pipwhich pip2which pip3

This should give you outputs like /home/your-user/.local/bin/pip.

Install, update virtualenvwrapper¶

virtualenvwrapper is a very useful – er – wrapper around – er –virtualenv, that makes it easier and neater to have a library of virtualPython environments. First install the Debian packaged version to your systemdirectories. This sets up bash shell integration:

sudo apt-get install virtualenvwrapper

Now upgrade your user installation to the latest virtualenvwrapper:

pip install --user --upgrade virtualenvwrapper

The --upgrade in the installation is important because virtualenv(installed by virtualenvwrapper) contains its own copy of Pip. We need thelatest version of virtualenv to make sure we will get a recent version of Pipin our virtualenvs.

Check you are getting your new --user installed version, with:

which virtualenv

This should you something like /home/your-user/.local/bin/virtualenv.

The virtualenvwrapper apt package puts useful aliases into the defaultbash shell environment. To get these aliases loaded up in your current shell,this one time you should do:

source ~/.bashrc

Check you have the virtualenvwrapper aliases loaded with:

mkvirtualenv

This should give you the help for the mkvirtualenv virtualenvwrappercommand.

Set up your system to build binary Python packages¶

This will install the tools that Python needs to build any binary package:

# For Python 2sudo apt-get install -y python-dev# For Python 3sudo apt-get install -y python3-dev

Pip will need these tools when installing Python packages that do not alreadyhave binary packages for your platform (see below).

Build or install wheels by installing with Pip¶

Many standard Python packages have binary manylinux wheels. These binaryinstallers will work for almost any Intel-based Linux, including Debian /Ubuntu. If your platform is compatible, Pip will download and install thebinary package when you do a simple:

pip install --user numpy

where numpy is the package to install.

Start up a new virtualenv for Python:

mkvirtualenv venv

Install numpy and cython. If you are on an Intel platform, this will downloadbinary wheels for the latest numpy and cython. If you are not on Intel, Pipwill download the source packages, then build and cache the wheels1:

pip install numpy cython

Now you can install (and, if not on Intel, build and cache) other wheels youmight need:

pip install scipy matplotlib h5py

Finish up by deactivating the virtualenv:

deactivate

This is the same sequence using Python 3:

mkvirtualenv --python=/usr/bin/python3 venv-py3pip install numpy cythonpip install scipy matplotlib h5pydeactivate

Now you are in virtualenv nirvana¶

It’s often good to use virtualenvs to start a development session. Doing someans that you can install exactly the requirements that you need, withoutcausing changes to your other virtualenvs.

You can now make virtualenvs for your testing development quickly. Say youwant to test something out for Python 3:

# Make clean virtual environmentmkvirtualenv --python=/usr/bin/python3 testing-somethingpip install numpy scipy matplotlib h5py# install anything else you want# run your testsdeactivate

Nice.

Even if you are offline, you can always install things you have already builtand cached, by adding the --no-index flag to Pip:

# Make another clean virtual environmentmkvirtualenv --python=/usr/bin/python3 testing-offlinepip install numpy scipy matplotlib h5py --no-index# install anything else you want# run your testsdeactivate

Adding new packages and wheels¶

Adding new wheels is usually as simple as:

# Switch to relevant virtualenv to build, cache, install wheelworkon venvpip install my-package

Sometimes the Python package you are installing has nasty binary dependencies.In this case, usually your easiest path is to install the build dependenciesfor the corresponding Debian / Ubuntu package, and then continue as before:

sudo apt-get build-dep pillowworkon venvpip install pillow

Sometimes, it’s a package too far¶

There are some Python packages that have heavy binary dependencies, or usecomplicated build systems, so that it is not practical to build a wheel withPip. Examples I know of are VTK and ITK. For those cases, your best optionis to install the Python package using apt-get, and then make yourvirtualenv with the --system-site-packages flag, so that it will pick upthe installed packages:

sudo apt-get install python-vtkmkvirtualenv --system-site-packages an-env-including-vtk

Doesn’t work for you? Help improve this page¶

If you try the instructions here, and you can’t get a particular package orset-up to work, then why not make an issue for therepository hosting these pages, and I’ll see if I can work the fix into thispage somewhere.

Footnotes

1

If you need to build common packages such as Numpy (forexample, on platforms like ARM), you should first install the Debian /Ubuntu packages with the build dependencies for these packages. For exampleyou might want to run something like this:

sudo apt-get build-dep python-numpy python-scipy matplotlib h5py

This will take a fairly long time. See Adding new packages and wheels.