Python hosting with virtual environments and up-to-date Python versions

On servers with old Linux distributions usually only old Python versions are available through the system paket manager. If you need to run an up-to-date Python version for hosting a web application this can be a quite time consuming task. One way to achive this is to compile and install the desired Python version by hand. A common requirement is also to host serveral web applications on one server. This can be achived using virtual environments for each web application.

 

Pyenv

The right tool for this job is pyenv, which is used commonly for Python development for switching between Python versions and virtual environments. pyenv comes with several plugins of which pyenv-virtualenv is the one for handling virtual environments.

With one command you can install a Python version:

user@host:/pyenv install 3.4.3

or a virtual environment (mysite) for a Python version:

user@host:/pyenv virtualenv 3.4.3 mysite

The Plugin pyenv-build takes over the task of compiling Python. This plugin comes with pyenv when it is installed. pyenv-virtualenv is a wrapper for the popular virtualenv. You should also install the plugin pip-rehash for the ease of using pip.

Pyenv installation

Pyenv and its plugins consist of shell scripts which need to be isntalled directly out of their Git repository. The installation in our example is using /usr/local and not an user home directory as install path because we want to use the Python version systemwide.

Compiling Python an a Linux needs to meet certain package requirements. Also Git needs to be installed on that system to cope with the pyenv Git repositories.

The installation is done in a couple of lines:

user@host:/usr/local/$ git clone https://github.com/yyuu/pyenv.git

user@host:/usr/local/pyenv/plugins/$ git clone https://github.com/yyuu/pyenv-virtualenv.git

user@host:/usr/local/pyenv/plugins/$ git clone https://github.com/yyuu/pyenv-pip-rehash.git

If you want to use pyenv on the command line you need to activate the functionality for that user with some entries in .bash_profile (Debian: .bash_rc):

export PYENV_ROOT=/usr/local/pyenv
export export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Now you need to restart your shell:

user@host:/$ exec $SHELL

 

Python 3.4.3 installation

The installation of Python is done in two lines as already mentioned above:

user@host:/pyenv install 3.4.3

user@host:/pyenv virtualenv 3.4.3 mysite

You can find the new Python installation under /usr/local/pyenv/versions.

 

Mod_wsgi installation

Graham Dumpletons mod_wsgi is still the recommended way for running Python web application on the Apache web server. Mod_wsgi supports only one Python version which it was compiled for. When you install mod_using using the system package manager, it matches usually the python system version. If we want to use Python 3.4.3 we need to compile mod_wsgi for that version. For compling it you need the Apache development headers, which you can install via the system package manager Debian: apache2-dev).

To get the job done we download the latest source code of the module and compile it for our Python version 3.4.3:

user@host:/tmp/$ wget https://codeload.github.com/GrahamDumpleton/mod_wsgi/tar.gz/4.4.13

user@host:/tmp/$ tar -xvf 4.4.13

user@host:/tmp/$ cd 4.4.13

user@host:/tmp/4.4.13/$ ./configure --with-python=/usr/local/pyenv/versions/3.4.3/bin/python

user@host:/tmp/4.4.13/$ LD_RUN_PATH=/usr/local/pyenv/versions/3.4.3/lib

user@host:/tmp/4.4.13/$ make

user@host:/tmp/4.4.13/$ make install

Now we need to integrate that module in our Apache configuration. The compile process produces the module at /usr/lib/apache2/modules/mod_wsgi. Running Debian you want a wsgi.load configuration file in /etc/apache2/mods-available and to activate that module:

user@host:/tmp/4.4.13/$ echo "LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so' >> /etc/apache2/mods-available/mod_wsgi.load

user@host:/tmp/4.4.13/$ a2enmod mod_wsgi

 

Apache Konfiguration

We will create a Django example project in usr/local/www/mysite which we will run with Apache:

user@host:/usr/local/www/$ pyenv activate mysite

user@host:/usr/local/www/$ pip install django

user@host:/usr/local/www/$  django-admin startproject mysite

We need to create a site configuration for our example project. You need to make sure to configure the virtual environment right. If you want to the WSGI Daemon mode in a producton environment you need to pass the virtual environment configuration as a Parameter for the WSGIDaemonProcess directive.

WSGIScriptAlias /     /usr/local/www/mysite/mysite/wsgi.py
WSGIPythonPath /usr/local/www/mysite:/usr/local/pyenv/versions/mysite/lib/python3.4/site-packages

<VirtualHost *:80>
ServerName mysite
LogLevel debug

<Directory "/usr/local/www/mysite/mysite">
                   <Files wsgi.py>
                       Order deny,allow
                       Allow from all
                   </Files>
</Directory>

</VirtualHost>

 

The future

During the last months Graham Dumpleton did some serious overhaul of mod_wsgi. With mod_wsgi-express Graham developed a standalone Python Hosting solution which he calles Production capable by default" in his blog. This variant becomes interesting when you install the companion package mod_wsgi-httpd besides the mod_wsgi package. Mod_wsgi-httpd contains the source code of the Apache server, which gets compiled and installed automatically for the use with mod_wsgi. This Apache installation is set up in the site-Packages folder and therfore doesn`t interfere with an Apache installation which comes with the Linux system. There are also scripts to run a HTTP-Daemon installation. I will give this a try soon and see if it is really production ready hosting solution.

Comments, additions or critics are always welcome. Please use our contact form.

Author: Max Pfeiffer - Alp Phone GmbH