It is well known that Python has a problem when it comes to installation of applications since shared libraries can sometimes have significant API changes from one version to another. Added to the complication is the fact that many linux based distros have a preinstalled version of Python that comes with old versions of certain libraries that if overwritten can affect OS stability.

In this guide I will outline the best way to install Python applications & libraries from PyPI without breaking your system and causing a dependency hell.


Is the program I am about to install an application or a library?

In the context of installation we define an application as a Python program which exposes a user interface that can be interacted with. For eg. mitmproxy and dockersh both offer a CLI when called from the terminal which you can use in real time to complete tasks.

Libraries on the other hand are the building blocks of above mentioned applications. They are usually installed on their own only when developing applications or building other libraries. For e.g requests and boto3 can only be imported from within a Python program. Calling them from the terminal serves no purpose.

Sometimes a downloadable package exposes a CLI but also pulls in libraries that you can use in your programs to build new tools. Examples of such hybrid packages are Impacket. When dealing with such a package you should look at the purpose you require the package for. Do you wish to use it’s CLI or do you wish to use the modules it provides to build an application? Depending on this purpose the method of installation changes as described below.


Installing applications

The best way to install Python applications going forward is using pipx. On an Ubuntu system type the following in the terminal

sudo apt install python3-pip pipx python3-venv

python3 -m pipx ensurepath

pipx allows you to run Python applications by installing them in isolated environments along with their dependencies. If your distro’s package repository doesn’t contain pipx you can visit this link and find installation instructions for your platform.

  • Apps that don’t require root privileges

Applications that don’t require root privileges can be installed using the following command

python3 -m pipx install PACKAGE

In the above command replace PACKAGE with the name of the package you wish to install. For eg. python3 -m pipx install mitmproxy

  • Apps that require root privileges

For applications that require root privileges use the following command

sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin python3 -m pipx install PACKAGE

In the above command replace PACKAGE with the name of the package you wish to install. For eg. sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin python3 -m pipx install dockersh


Installing libraries

Since isolated installation of libraries are only really done during development you should only be installing them inside of python virtual environments. To get started we first need to install venv on our system.

sudo apt install python3-venv

Now we create a new folder that will hold our project and then we will configure a virtual environment inside of it.

mkdir myproject & cd myproject

python3 -m venv myproject-venv

Be sure to add the myproject-venv/ folder to .gitignore if you are using git based workflows.


To begin development you need to activate the virtual environment which can be done by

source myproject-venv/bin/activate

This will change your prompt to (myproject-venv) user@mycomputer:~/myproject$ which signifies that the virtual environment is currently active.


You can now install any library from pip and it will be installed in the current virtual environment without affecting your system or any other application present on your system.

python3 -m pip install PACKAGE

Finally when you are done with development work and wish to revert back to your regular system, simply enter deactivate at the terminal.

deactivate

  • Using sudo inside a virtual environment

A lot of times the application you are building or a component library requires root privileges to work correctly. In order to use sudo inside a virtual environment it has to be called differently than usual.

sudo -s "PATH=$PATH" python3 main.py

This will ensure that the sudo uses the current virtual environment’s Python interpreter than the system’s interpreter.


Conclusion

Note that in the above commands we invoke pip, pipx and venv as python modules like python3 -m pipx --version instead of pipx --version. Always invoke these tools in this manner as the latter method can result in unexpected behaviour at times. The practices described above should serve adequately for most users. They will only slightly change if you are working with pyenv or a recent version of Python that you installed on your own. If you have any queries feel free to reach out to me at solamarpreet@protonmail.com