Expanding your arsenal with virtualenv and virtualenvwrapper

If you’re a freelancer, then you’re probably working on multiple projects at the same time. If by any chance the projects use different versions of django, then you’re pretty much in version hell, writing bash scripts to change paths and all the other sins. This is where virtualenv will save your rear end.

virtualenv is a python module to help you isolate virtual python enviroments. Each environment will have its own python executable, it’s own site-packages effectively allowing you to install completely diferent dependencies for each project.

Installation

First we’ll simply install virtualenv. We’ll also use Doug Hellman’s awesome virtualenvwrapper (http://www.doughellmann.com/projects/virtualenvwrapper/). It will allow us to organize all virtual environments at once place.

I prefer pip to all the dirty work for me. So simply run,

easy_install pip
pip install virtualenv
pip install virtualenvwrapper

This should install both packages to the global python site-packages. One thing to note during installation is the path to virtualenvwrapper.sh. It will be printed during installation. Typically it’s found at /usr/local/bin/virtualenvwrapper.sh . We’ll need this later.

There’re some extra steps to setup virtualenvwrapper. If you’re on *nix then add the following lines at the top of ~/.bashrc . Here we’re just specifying that we want to save all our environments in a particular directory.

# virtualenvwrapper
export WORKON_HOME=~/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true

Now let’s create that directory, and initialize virtualenvwrapper

mkdir ~/.virtualenvs
source ~/.bashrc

Usage

mkvirtualenv --no-site-packages myenv

This will create an environment and not import the global packages. This is important. This means it only uses libraries/versions in the local env. I prefer to do this on all production environments to make sure upgrading a global package doesn’t affect the web app running on the current venv. The only down side is that you’ve to install all the modules again in each new environment you create. You can choose to skip the argument —no-site-packages, and it will use all the modules in your global python path.

To enable a particular environment, virtualenvwrapper gives us some shortcut scripts. This will change the prompt to show the name of the current environment, eg. (myenv)$>

workon myenv

Tips

You have a brand new virtual environment to play with. While it’s active you can pretty much install any module an it will be installed only for this environment. Your global packages will be safe as ever. Also it’s a good idea to keep all your project dependencies in a requirements.txt file.

cd ~/go/to/project/dir
 pip install -r requirements.txt

A sample requirements file could look like the following.

django==1.2.4
South==0.7.2

You can try $>pip freeze > requirements.txt to port an existing virtualenvironment to a requirements file. Now you can add a step to your deployment script to install all required dependencies each time. Fabric is also an awesome tool that every Django programmer should know and use.

There are other awesome things that will save you time. If you goto your WORKON_HOME dir, you will notice files like postactivate, postdeactivate. These are global shell scripts that you can add commands to. They will be exectued on each activate, deactivate etc. Infact each environment directory inside can have their own hooks like these. Enough spoon feeding, browse the docs to find out more.

THATS IT.

But wait!! it’s not that easy. If you’re running a django project like this, you’ll have a hell of a time installing modules like mysql-python, PIL etc. using pip inside a virtualenv. They need to be compiled, and due to changed paths the correct libraries aren’t found on a fresh system. I’ll explain some more common gotchas on my next post.

Sources

5 responses
Very concise and straightforward guide. Thanks!
This was very helpful. Thanks!
Very, informative. Helped me round out my fabric installation. Thought I'd share one tip though...
If you want to use this with fabric, add the variables to .bash_profile rather than .bashrc as fabric runs the shell with the -l flag
Actually you can add them to .bashrc but you have to add them *ABOVE* this line

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

1 visitor upvoted this post.