If you’re a freelancer, then you’re probably working on multiple projects at the same time. If by any chance the projects use different versions of django, then you’re pretty much in version hell, writing bash scripts to change paths and all the other sins. This is where virtualenv will save your rear end.

virtualenv is a python module to help you isolate virtual python enviroments. Each environment will have its own python executable, it’s own site-packages effectively allowing you to install completely diferent dependencies for each project.

Installation

First we’ll simply install virtualenv. We’ll also use Doug Hellman’s awesome virtualenvwrapper (http://www.doughellmann.com/projects/virtualenvwrapper/). It will allow us to organize all virtual environments at once place.

I prefer pip to all the dirty work for me. So simply run,

easy_install pip
pip install virtualenv
pip install virtualenvwrapper

This should install both packages to the global python site-packages. One thing to note during installation is the path to virtualenvwrapper.sh. It will be printed during installation. Typically it’s found at /usr/local/bin/virtualenvwrapper.sh . We’ll need this later.

There’re some extra steps to setup virtualenvwrapper. If you’re on *nix then add the following lines at the top of ~/.bashrc . Here we’re just specifying that we want to save all our environments in a particular directory.

# virtualenvwrapper
export WORKON_HOME=~/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true

Now let’s create that directory, and initialize virtualenvwrapper

mkdir ~/.virtualenvs
source ~/.bashrc

Usage

mkvirtualenv --no-site-packages myenv

This will create an environment and not import the global packages. This is important. This means it only uses libraries/versions in the local env. I prefer to do this on all production environments to make sure upgrading a global package doesn’t affect the web app running on the current venv. The only down side is that you’ve to install all the modules again in each new environment you create. You can choose to skip the argument —no-site-packages, and it will use all the modules in your global python path.

To enable a particular environment, virtualenvwrapper gives us some shortcut scripts. This will change the prompt to show the name of the current environment, eg. (myenv)$>

workon myenv

Tips

You have a brand new virtual environment to play with. While it’s active you can pretty much install any module an it will be installed only for this environment. Your global packages will be safe as ever. Also it’s a good idea to keep all your project dependencies in a requirements.txt file.

cd ~/go/to/project/dir
 pip install -r requirements.txt

A sample requirements file could look like the following.

django==1.2.4
South==0.7.2

You can try $>pip freeze > requirements.txt to port an existing virtualenvironment to a requirements file. Now you can add a step to your deployment script to install all required dependencies each time. Fabric is also an awesome tool that every Django programmer should know and use.

There are other awesome things that will save you time. If you goto your WORKON_HOME dir, you will notice files like postactivate, postdeactivate. These are global shell scripts that you can add commands to. They will be exectued on each activate, deactivate etc. Infact each environment directory inside can have their own hooks like these. Enough spoon feeding, browse the docs to find out more.

THATS IT.

But wait!! it’s not that easy. If you’re running a django project like this, you’ll have a hell of a time installing modules like mysql-python, PIL etc. using pip inside a virtualenv. They need to be compiled, and due to changed paths the correct libraries aren’t found on a fresh system. I’ll explain some more common gotchas on my next post.

Sources

• http://virtualenv.openplans.org/
• http://www.doughellmann.com/projects/virtualenvwrapper/

Continuing on from the last post, django provides an excellent hook in middlewares to process exception. You can add a new middleware and override the process_handler exception to add any kind of info to the request object. This info then shows up on the errors displayed on the screen(in debug mode) or sent as email. Profit!

class ExceptionUserInfoMiddleware(object):
    """
    Adds user details to request.META, so that they show up in
    the error emails.

    Add to settings.MIDDLEWARE_CLASSES and keep it outermost
    (i.e. on top if possible). This allows it to catch exceptions 
    in other middlewares as well.
    """

    def process_exception(self, request, exception):
        """
        Process the exception.

        :Parameters:
           - `request`: request that caused the exception
           - `exception`: actual exception being raised
        """

        try:
            if request.user.is_authenticated():
                request.META['USERNAME'] = str(request.user.username)
                request.META['USER_EMAIL'] = str(request.user.email)
        except:
            pass

You can also find the latest version of this snippet at http://gist.github.com/646372

Django error emails on 500 errors are pretty useless if you’re the acting
customer support for the day. There’s absolutely not much besides a
session ID, to identify which user actually got that exception. Obviously
for a startup, that’s essential to reacho out to little customers we do
have, if for nothing else than to apologize. Anyways here’s a simple
little snippet that can help you figure out a user from a session id.
Note, that the user might not be logged in, in that case there’s not much
you can do.

def get_user_from_sessionid(sessionid):
       """
       ``sessionid`` - is the identifier for the users session. 
       You can also find it in the Http500 error messages on screen
       or via email.

       >>> get_user_from_sessionid('fd3479022d77794897e5b23f4d461bbf')
       """
       from django.contrib.sessions.models import Session
       from django.contrib.auth.models import User


       session = Session.objects.get(session_key=sessionid)
       uid = session.get_decoded().get('_auth_user_id')
       user = User.objects.get(pk=uid)

       return user.id, user.username, user.email

Links

• http://code.google.com/p/django-command-extensions/ – There is already a command in django-extensions, called ‘print_user_for_session’ that does
  the same.
• http://djangosnippets.org/snippets/1276/ – for a backend agnostic method.
• http://scottbarnham.com/blog/2008/12/04/get-user-from-session-key-in-django/ – is practically the same snippet as above.

There’s also a way to add the username to the error emails themselves, by overriding a process_exception on a middleware. You can read about it here

.

This POST might be slightly out of date from time to time. See http://github.com/sidmitra/django_nonrel_testapp.

The Django nonrel project is an attempt to make django work with non relational backends. They’ve also made a Google AppEngine backend for the same. Thus, now you can easily run native django apps on GAE.

Some caveats first, there are some features from Django that are missing(like joins! which is coming) or some that aren’t perfect yet but it’s a good start to port over any existing simple apps that you already have on Django. Something you should keep in mind is that there’s no way to mix the native django models with those that appengine provides. Some of the steps i took to get it running are specified below. In case you are too lazy :–), I have a Github repository for the same at http://github.com/sidmitra/django_nonrel_testapp.**

Downloading stuff

Getting a simple test app running with nonrel is simple. But you would need to install Mercurial first. Once you have that up and running, they project already provides a sample project to support the dev server. So you can start by cloning all required repositories first.

hg clone http://bitbucket.org/wkornewald/django-nonrel
hg clone http://bitbucket.org/wkornewald/django-testapp/
hg clone http://bitbucket.org/wkornewald/djangoappengine/
hg clone http://bitbucket.org/wkornewald/djangotoolbox/
hg clone http://bitbucket.org/wkornewald/django-dbindexer/src

Setting it up

django-nonrel is the main project containing a modified django, and we need djangoappengine, djangotoolbox as further dependencies. django-testapp is the sample app created with custom configuration to make this quicker.

cd django-testapp
cp -R ../django-nonrel/django django
cp -R ../djangoappengine djangoappengine
cp -R ../djangotoolbox/djangotoolbox djangotoolbox
cp -R ../django-dbindexer/dbindexer dbindexer
python manage.py runserver

And that’s it. Essentially we’ve copied all the dependencies into the project folder. So your directory structure should look something like this.

* django-testapp
     * django
     * djangotoolbox
     * djangoappengine
     * dbindexer 
     * ...

You can also use links on linux instead of copying the directories. On windows install Link shell extension. You can drop junctions into the common-apps folder.

Source:

http://www.allbuttonspressed.com/blog/django/2010/01/Native-Django-on-App-Engine http://www.allbuttonspressed.com/projects/djangoappengine#docu

To find out what queries Django generates behind all that ORM magic, make sure that settings.DEBUG=True, and then run:

>>> from django.db import connection
>>> connection.queries
[{'sql': u'SELECT `auth_user`.`id`, `auth_user`.`username`, 
`auth_user`.`first_name`, `auth_user`.`last_name`, `auth_user`.`email`, 
`auth_user`.`password`, `auth_user`.`is_staff`, `auth_user`.`is_active`, 
`auth_user`.`is_superuser`, `auth_user`.`last_login`, 
`auth_user`.`date_joined` FROM `auth_user` 
WHERE `auth_user`.`id` = 2 ',  'time': '0.002'}, ]

The connection.queries attribute spits out a list of dictionaries of all queries in order of execution. The dictionary contains the actual SQL executed along with the time it took for the same.

To see the last query run, you can use pop:

>>> connection.queries.pop

and to explore further methods, attributes on the connection object, you can always do:

>>> dir(connection.queries)

This is a fairly common gotcha, that i had forgotten about until i wasted half a day troubleshooting it again.

To solve, make sure the database file itself is writable, and not only that, also make sure that the parent directory is writable as well so the database engine can make temporary files. So just chmod 777 to check if it works :-) but remember to choose your permissions carefully later.

Ref: http://code.djangoproject.com/wiki/NewbieMistakes#DjangosaysUnabletoOpenDatabaseFilewhenusingSQLite3