Setting up a Python environment for Mbed TLS
Python and package versions
Each branch of Mbed TLS has different minimum requirements on Python. See README.md
in the branch. Any Python version that’s more recent than the minimum should work, however as of November 2022 this is not well tested by the CI so unintended incompatibilities can happen.
Some of the scripts used by Mbed TLS users and developers require third-party packages. Those have version constraints as well. The constraints are documented via *.requirements.txt
files in the scripts
directory.
We run mypy and pylint to check Python scripts. It is common that newer versions of Pylint reject our code, because some checks have become stricter or new checks have been added.
Set up a Python package environment
What is a virtual environment?
A virtual environment (venv for short) is a directory tree containing an autonomous set of Python packages. The venv is completely independent from the system installation, except for the Python executable itself and the standard library. System packages are not visible in the venv, so the venv can have its own version, and you can even test what happens when a package is missing by creating a venv without it.
Creating a virtual environment for Mbed TLS
Summary:
python3 -m venv ~/venv-mbedtls-development
~/venv-mbedtls-development/bin/python scripts/min_requirements.py
To create a virtual environment, invoke the venv
module (which ships with Python) with one argument, which is the path to the directory to create.
To set up a virtual environment with the reference setup used on the Mbed TLS CI, check out mbedtls and run Mbed TLS’s scripts/min_requirements.py
explicitly with the desired venv’s Python:
~/venv-mbedtls-development/bin/python scripts/min_requirements.py
Using the virtual environment’s Python ensures that the packages are installed inside the virtual environment, and not system-wide or in your home directory.
You can also run scripts/min_requirements.py
after activating the venv.
The virtual environment will use the Python executable that was used to set it up with -m venv
. Thus, if you create a venv with a certain version of Python, this version is the one that will be used inside the venv. For example, to set up the same environment as on most of the CI at the time of writing, install Python 3.5 and run
python3.5 -m venv ~/mbedtls-venv-3.5
~/mbedtls-venv-3.5/bin/python scripts/min_requirements.py
Note that the venv directory contains references to its own path, so if you want to move it, you have to do more than just move the directory.
Working in a virtual environment
In a nutshell: start a new shell (bash, zsh or ksh) and issue this command (where ~/venv-mbedtls-development
is the path to the venv directory, the same path passed to python -m venv
):
. ~/venv-mbedtls-development/bin/activate
Note the .
at the beginning: the commands must run inside the running shell, not as a subprogram. There are also activate
scripts for fish, csh, PowerShell and cmd. The activate
changes the shell prompt to indicate the active virtual environment.
The activate
script sets environment variables so that all calls to python
or python3
will use the Python installation from the virtual environment, with the packages from the virtual environment.
mbedtls$ . ~/venv-mbedtls-development/bin/activate
(venv-mbedtls-development) mbedtls$ make generated_files
Gen error.c
Gen version_features.c
Gen ssl_debug_helpers_generated.c
Gen psa_crypto_driver_wrappers.c
Gen psa/psa_constant_names_generated.c
Gen test/query_config.c
Gen suites/test_suite_psa_crypto_generate_key.generated.data suites/test_suite_psa_crypto_not_supported.generated.data suites/test_suite_psa_crypto_op_fail.generated.data suites/test_suite_psa_crypto_storage_format.current.data suites/test_suite_psa_crypto_storage_format.v0.data ...
Gen suites/test_suite_bignum.generated.data suites/test_suite_bignum_core.generated.data
Gen visualc/VS2010/mbedTLS.sln ...
(venv-mbedtls-development) mbedtls$ tests/scripts/check-python-files.sh
Running pylint ...
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Running mypy ...
Success: no issues found in 27 source files
To run a single Python script in the virtual environment, you don’t need to use activate
: you can invoke Python via the symbolic link from the bin
directory of the virtual environment. This also works with Python programs that are part of the virtual environment. You need activate
when invoking programs that themselves call Python.
Installing different Python versions
Each virtual environment can have a different Python version, and running in a virtual environment makes that version of Python the one on the command search path. But all the Python versions need to be installed at the system level.
You can have multiple Python versions installed at the system level, and they won’t interfere. Here are a few ways you can install additional Python versions:
On Ubuntu or other Debian-based distributions: the deadsnakes PPA.
On Linux: install another Debian or Ubuntu release in a chroot, made painless with schroot and debootstrap.
On macOS with Homebrew: e.g.
brew install python3.6
On macOS with MacPorts: e.g.
sudo port install py36
On macOS with Fink: e.g.
fink install python36
On most systems except Windows, with pyenv.
On most systems, with Conda.