Setting up Django 1.3 + NGinx 1.0.5 + Green Unicorn 0.13 in an Ubuntu 11.10 EC2 instance

58 Replies

Django has become the de-facto web framework for Python. Although, since Django just specializes in dynamic content, you have to combine it (at least in production) with an HTTP server to serve static content such as css, javascript files and images files. In the past, the communication protocol between Python web applications was CGI, FastCGI or mod_python. But after PEP-333 was accepted the faster and more efficient WSGI became the standard.

Green Unicorn is a Python WSGI HTTP Server for UNIX. Its combination with the high performance HTTP server NGinx is gaining lot of momentum in the Python community.

From “man gunicorn”…

Green Unicorn (gunicorn) is an HTTP/WSGI server designed to serve
fast clients or sleepy applications. That is to say; behind a
buffering front-end server such as nginx or lighttpd.

 

We are describing here how to combine a Django application with Green Unicorn and Nginx within a pristine EC2 Ubuntu 11.10 image.




Getting an Ubuntu instance

First of all we are going to http://cloud.ubuntu.com/ami/ to check the last released ami. As described in the image, we are looking for the last 64 bits, Ubuntu 11.10 EB2 image in the US-East region, which so far is ami-bf62a9d6.

  • Why Ubuntu 11.10 ? Because is the last stable release.
  • Why EBS? Because is faster and with more options like on the fly snapshots or easy image resizing than instance-store.
  • Why US-EAST? Because is cheaper and usually with more features than the other regions.
  • We also choose 64 bits so we can eventually expand to a bigger instance.

 



Starting your free EC2 instance



So far there is a nice offer for new customers where you can get a free EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) for one year. Please open your account and then launch your instance as described in the following screen-shots:

 



Launching and configuring your Ubuntu box



At this point you should have your EC2 or your own Ubuntu box up and running. If you are using EC2 you should ssh into it with this command:

ssh -i you_private_key.pem ubuntu@DNS_NAME

Then, please follow this steps to start setting your environment:

#install Python Package Installer
sudo apt-get install python-pip
#upgrade PIP itself
sudo pip install pip --upgrade
#install Green Unicorn
#install Virtualenv to generate our own isolated environment
sudo pip install virtualenv
#Installing NGINX
sudo apt-get install nginx
#Creating our Virtualenv environment
virtualenv --no-site-packages django_app
cd django_app
#activating the environment
source bin/activate
#installing Green Unicorn and Django
pip install django gunicorn
#Creating a Django project
django-admin.py startproject app
cd app
#start Django application with Green Unicorn
gunicorn_django -b 0.0.0.0:8000

Now go to your browser and check if your application is working as expected:



Finally, just Ctrl+C and “deactivate” so you can continue with Nginx…


Setting up Nginx



Now we will set up Nginx in order to have it listening in the port 80, serving static files and working with Gunicorn and Django in order to serve dynamic content.

#prepare directories
sudo mkdir -p /opt/django/logs/nginx/
#Create directories and softlinks for static content and templates
mkdir $HOME/django_app/static
mkdir $HOME/django_app/templates
sudo ln -s $HOME/django_app/static /opt/django
#download and set up Nginx configuration. Basically it will listen in port 80 and forward to port 8000 dynamic content
sudo mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.backup
wget https://bitbucket.org/deccico/django_gunicorn/raw/tip/server/etc/nginx/sites-available/default
sudo cp default /etc/nginx/sites-available/default

 

Testing Static resources


Configuring settings.py



You need to tell Django where your templates are, for that reason, add your absolute “templates” directory to your /home/ubuntu/django_app/app/settings.py file

Now look for the TEMPLATE_DIRS section and add your template directory

TEMPLATE_DIRS = (
# Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
# Always use forward slashes, even on Windows.
# Don't forget to use absolute paths, not relative paths.
'/home/ubuntu/django_app/templates',
)



Configuring urls.py



/home/ubuntu/django_app/app/urls.py will help us telling Django to return our template page when we introduce the /test_static request.

To make things easier we are using a dynamic view:

from django.conf.urls.defaults import patterns
from django.views.generic.simple import direct_to_template

urlpatterns = patterns('',
(r'^test_static/$',             direct_to_template, {'template': 'test_static.html'}),
)



Preparing test template



We need to prepare our template. Please create this file /home/ubuntu/django_app/templates/test_static.html with the following content:


<!--Load static will recover the static prefix variable and saving it in STATIC_PREFIX. We do this only once.-->

{% load static %}

{% get_static_prefix as STATIC_PREFIX %}

<!--Here we are just showing how easy is to recover static resources . -->
Test image
<img src="{{ STATIC_PREFIX }}django.png" />

After creating the template file you will need an actual image :) You can download one from the same repo.


cd /home/ubuntu/django_app/static

wget https://bitbucket.org/deccico/django_gunicorn/raw/tip/static/django.png



Restarting Nginx and voilà



Finally, we just need to restart Nginx and test everything is working.


#First we restart Nginx

sudo service nginx restart

Now we will start our Django application again:

cd /home/ubuntu/django_app/app
#activating the environment
source ../bin/activate
#start Django application with Green Unicorn
gunicorn_django -b 0.0.0.0:8000

Then we just go to our browser and request test_static in port 80. In my case the full address is: http://ec2-107-20-88-133.compute-1.amazonaws.com/test_static

As you can see above, we got Nginx serving the image while Django give us the content.

Type “Ctrl + C” and then “deactivate” again so we can be ready for the next step…

 

Setting Green Unicorn automatic start



We will use Upstart so we don’t have to worry about starting our application whenever we need to restart the server. One more configuration file and a bootstrap script will do the job here:

In /home/ubuntu/django_app/run.sh we create:

#!/bin/bash
set -e
LOGFILE=/var/log/gunicorn/django_app.log
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=3  #recommended formula here is 1 + 2 * NUM_CORES

#we don't want to run this as root..
USER=www-data
GROUP=www-data

cd /home/ubuntu/django_app
source bin/activate
cd app
test -d $LOGDIR || mkdir -p $LOGDIR
exec gunicorn_django -w $NUM_WORKERS 
  --log-level=debug 
  --log-file=$LOGFILE 2>>$LOGFILE 
  --user=$USER --group=$GROUP

After creating the file, please remember to execute chmod a+x /home/ubuntu/django_app/run.sh

On the other hand, Upstart configuration file have to be located in: /etc/init/django_app.conf with the following content:


description "Django Application"

start on runlevel [2345]

stop on runlevel [06]

respawn

respawn limit 10 5

exec /home/ubuntu/django_app/run.sh

Now we can control our application with

sudo start django_app

sudo stop django_app




To sum up



The puropose of this post is to provide a path to make the installation and configuration of this stack easier. While some things could be changed or improved the focus was to stress how easily we can configure everything from scratch rather than exploring or discussing different options.

The code and configuration files of this post can be found here: https://bitbucket.org/deccico/django_gunicorn Have fun!

 

References




Update: these instructions are now automated in this new post

58 thoughts on “Setting up Django 1.3 + NGinx 1.0.5 + Green Unicorn 0.13 in an Ubuntu 11.10 EC2 instance”

  2. Jonas

    Why is Green Unicorn preferrable to running WSGI or uwsgi directly with Nginx (there are modules for it, and uwsgi shows very impressive performance numbers)?

    Reply ↓
  4. suvash

    hi there,

    definitely a great post. but i ran into an issue(which i also kinda fixed but not happily)

    In the section “Launching and configuring your Ubuntu box”
    > gunicorn_django -b 0.0.0.0:8000
    fails with the following exception
    “ImportError: No module named django.core.handlers.wsgi”

    after a while of though, i figured gunicorn_django wasn’t able to see the django installed in the virtualenv, so I deactivated the virtualenv and then installed django globally(without virtualenv)

    i then enterted the virtualenv and ran the gunicorn_django again, this time it didn’t complain.

    would you know a better way than using this ugly hack ?

    Reply ↓

  5. Pingback: Adrián Deccico: Setting up Django 1.3 + NGinx... | Python | Syngu

  6. Austen

    I think the issue could be that we install gunicorn in the global scale of python, but then make a virtualenv without site packages. I tried installing gunicorn inside the virtualenv this error disappeared. @suvash

    Reply ↓
  7. arek

    @suvash
    hi – what worked for me was uninstalling gunicorn (sudo pip uninstall gunicorn ) after deactivating virtualenv (deactivate), and then installing gunicorn in my virtualenv – so source bin/activate, and then pip install gunicorn

    let me know if this worked for you

    Arek

    Reply ↓
  8. arek

    oh, I’m sorry i haven’t noticed Austen already solved suvash’s issue (I haven’t refreshed this page for some time). Great article btw, thanks.

    Reply ↓
  10. admin

    @Jonas
    Doesn’t look like there is a big performance difference between uwsgi and Gunicorn while the second is easier to set up and administer and consumes less cpu/memory.

    Anyway I think both options are valid.

    Reply ↓
  12. admin

    @suvash @arek @austen
    Thanks for the correction, I updated the text as you mention.

    As you mention, the idea is to install everything inside our Virtualenv. At some point I tried using gunicorn.d which replace the Upstart utility but doesn’t work with Virtualenv, that’s why I messed up that line.

    Reply ↓
  14. suvash

    @admin

    Thanks a lot there. great article btw !

    Maybe you’d also want to look into pythonbrew(manage multiple python installs + virtualenv management). I like the fact that pythonbrew(almost similiar to rvm in ruby) makes it simple to manage both pythons and venvs in one folder(home directory) instead of spreading the virtualenvs around in different directories.
    (available in pip as well as https://github.com/utahta/pythonbrew )

    I couldn’t manage to install pythons using pythonbrew on the aws image above, however would be great if you could do so.

    Reply ↓
  15. suvash

    @admin

    Further along, in the “Setting up Nginx” section, I’m not sure if “sudo” is required for creating “static” and “template” directory (line 4,5). This will only let root to create/edit files in that directory.

    Reply ↓
  16. admin

    @suvash
    hi, definitely sudo is not required for directories under $HOME. I updated the text.

    Regarding Pythonbrew, it definitely looks interesting.

    Cheers

    Reply ↓
  17. Daniel

    Thanks so much for this great post. Have been trying to get Django running on an EC2 for the entire last week. This time it took 17 Minutes. Can’t thank you enough.

    Reply ↓
  18. Daniel

    Only problem is that I don’t get start django_app up and running. I tried to reboot machine, but whenever I try to execute django_app I get:

    start: Rejected send message, 1 matched rules; type=”method_call”, sender=”:1.11″ (uid=1000 pid=1610 comm=”start django_app “) interface=”com.ubuntu.Upstart0_6.Job” member=”Start” error name=”(unset)” requested_reply=”0″ destination=”com.ubuntu.Upstart” (uid=0 pid=1 comm=”/sbin/init”)

    Do you have any suggestions where I could have went wrong? Thanks.

    Reply ↓
  20. Daniel

    OK, I solved, I had to use sudo start django_app — now it works like a charm. Thanks so much. I still don’t understand the whole virtualenv thing, only that I need it for Gunicorn. Right?

    Reply ↓
  21. admin

    @Daniel
    Hi Daniel, I just updated the text. Thanks.

    As you surely noticed Upstart is already installed in your Ubuntu system.

    >>I still don’t understand the whole virtualenv thing, only that I need it for Gunicorn. Right?

    Virtualenv is useful for keeping your whole environment isolated, making possible to have two Python/Django applications running in the same system with a different set of (maybe) contradictory dependencies.

    For example you could easily setup another Django application that uses another Gunicorn version different than 0.13.4. For our tutorial purposes, Gunicorn could be installed globally.

    All in all Virtualenv and PIP is the recommended option for any Python development beyond very simple scripts.

    More info:

    http://en.wikipedia.org/wiki/Dependency_hell

    http://pypi.python.org/pypi/virtualenv

    Reply ↓
  22. Daniel

    @admin Thanks a lot. I ended up not using VirtualEnv as I could simply not install psycopg2 to get PostgreSQL running. I may try again some day, but for now I have enough other problems setting up Django/Database etc.

    Reply ↓
  24. adrian Post author

    @Alexis Bellido
    Hi Alexis, I absolutely agree with you. Having the manual steps in this tutorial is just the first step. I am working in a tool (Bellatrix, already in the Cheese shop) to help with the automation of this.

    Probably will take a look at your scripts to combine it with Fabric.

    Reply ↓
  25. James

    Hi Adrian, Thanks for a fantastic walk-through. I’ve been struggling setting this up for ages and your walk through helped me so much. The only bit I was a bit confused about was the part where you say “on the other hand” when talking about Upstart config – I think it’s a translation issue.

    Thanks for sharing your knowledge,
    James

    Reply ↓
  28. memo

    hi, good morning, thanks a lot for the tuto but i have a question… in order to have nginx listening at port 80 i have to run this command “gunicorn_django -b 0.0.0.0:8000″ all the time? thank you for your time

    Reply ↓
  30. Joey

    Just tried the whole setup on my own ubuntu laptop, the problem is I can’t get the pictre show in the webpage, any one had the same problem?

    Reply ↓

  31. Pingback: plumberWorks » Blog Archive » Getting gunicorn,nginx, Django, postgresql up and running in EC2

  33. Igor

    Hi, I don’t can launch app
    In deamon.log i get this:

    Apr 22 16:00:57 ip-10-191-55-70 init: django_app main process (978) terminated with status 1
    Apr 22 16:00:57 ip-10-191-55-70 init: django_app main process ended, respawning
    Apr 22 16:00:57 ip-10-191-55-70 init: django_app main process (984) terminated with status 1
    Apr 22 16:00:57 ip-10-191-55-70 init: django_app respawning too fast, stopped

    Pleas help, why this is happening?

    Reply ↓
  34. Igor

    Thanks, it was my fault, not completely copied the file run.sh

    But if use this method for static files that do not work css styles in Django admin

    Reply ↓
  35. adrian Post author

    Hi Igor, that was a test for checking Django was up. The rest of the tutorial will show you how to configure Nginx to handle static resources.

    Reply ↓
  36. Nick

    @Igor

    Make sure to update the “default” file under “sites-available”. You are looking for this:

    root /path/to/test/lib/python2.7/site-packages/django/contrib;

    Update that to your virtualenv path.

    Reply ↓
  37. Anshul

    Hi adrian,

    I followed the instructions you provided for setting up gunicorn with django 1.3 on amazon ec2 instance that has Ubuntu 11.10.I did all steps that s mentioned in “Launching and configuring your Ubuntu box” section and when I fire this command “gunicorn_django -b 0.0.0.0:8000″ to start django application with gunicorn it gives me no error and showed:

    2012-05-25 07:14:08 [11999] [INFO] Starting gunicorn 0.14.3
    2012-05-25 07:14:08 [11999] [INFO] Listening at: http://0.0.0.0:8000 (11999)
    2012-05-25 07:14:08 [11999] [INFO] Using worker: sync
    2012-05-25 07:14:08 [12002] [INFO] Booting worker with pid: 12002

    that means gunicorn process has started but when I go to my browser and check if my application is working using http://ec2-xxx-xx-xx-xxx.compute-1.amazonaws.com:8000 , I got “could not connect to ec2-23-22-126-211.compute-1.amazonaws.com:8000″ error and even in shell where gunicorn is listening at port 8000 with 1 worker I didn’t see any get request.I don’t know why its happening and gunicorn is not returning me djnago welcome page. A better description of my problem is at “http://stackoverflow.com/questions/10703088/django-nginx-gunicorn-on-amazon-ec2-is-not-working”

    I would be very thankful if you could help me in this..

    Thanks
    Anshul

    Reply ↓
  40. emepe

    Gracias, che! Genial el tutorial, salió andando de una. Ahora tengo que investigarlo para implementarlo para mi app. Gracias de nuevo.

    Reply ↓
    1. adrian Post author

      me alegro que te haya servido. Por cierto, impresionantes las fotos de tu sitio!

      Reply ↓
  41. Xavi

    Hola Adrian! genial el tutorial, estoy configurando una micro instancia siguiéndolo y todo perfecto hasta que tengo que editar el settings.py por que no está en django_app/app/settings.py sino que se encuentra en django_app/app/app/settings.py

    Ya se que es una tontería, pero me gustaría saber qué está pasando. Gracias y saludos desde Barcelona!

    Reply ↓
    1. adrian Post author

      Me alegro qe te haya servido. Respecto a al diferencia en la ubicación del archivo, se debe a un pequeño cambio en la estructura que introduce Django 1.4.

      Reply ↓
  42. Jacob

    Thanks for the info Adrian. What value does nginx provide in conjunction with gunicorn? That is, I can run gunicorn alone without nginx. Why add nginx as a proxy to it?

    Reply ↓
    1. adrian Post author

      Hi Jacob, you need Ngnix in order to process static resources like css or image files, while GUnicorn will work with Django in order to process dynamic resources.

      Reply ↓
  43. Jacob

    Thanks Adrian. As it’s a REST api, there are no static resources. It looked like it was working fine without nginx, but I just added an Amazon ELB to it, and it’s now causing the workers to timeout. I read it may have to do with the ELB prematurely closing connections, so I ended up adding nginx as a proxy in between (with proxy_ignore_client_abort on), and that seems to have resolved that problem. However, now I’m investigating why Django is showing no http headers since I added nginx….

    Reply ↓
  46. Deepan

    Thank you for this tutorial, was a really good reference.
    Only problem I’m facing is that when I try to load test_static, i get:

    ValueError at /test_static/
    Exception Value: need more than 1 value to unpack

    Any ideas?

    Reply ↓
  47. Deepan

    @Deepan

    Ah realised this was an error I made with the template loaders.
    Thank you for the tutorial, definitely the best one I’ve ever seen!

    Reply ↓
  50. bankruptcy honolulu

    I like the helpful information you supply to your articles.
    I’ll bookmark your blog and test again here regularly. I am somewhat sure I will be told plenty of new stuff right right here! Best of luck for the next!

    Reply ↓
  51. Emmaline Dezern

    Wow that was unusual. I just wrote an extremely
    long comment but after I clicked submit my comment didn’t appear. Grrrr… well I’m not writing all that over again.
    Anyway, just wanted to say superb blog!

    Reply ↓
  52. Ice

    First of all, amazing tutorial! I was really lost and this tutorial helped me set up my EC2 and local environment with a full Django stack rather quickly.

    One thing that I had to modify was the symbolic link from /opt/django/static to $HOME/…/static
    I was not able to see any images. I had to create a symbolic link with the explicit path, instead of $HOME in order to get it working (i.e. /home/ubuntu/django_projects_dir/static)

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>