Ubuntu, Django, Postgresql and Nginx, a rock solid web stack.

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Lately I’ve been working on developing a web application project,   I guess everyone involved in such process had to make the uneasy decision to choose the right Web Stack to run the application.  Multiple factors usually drive the final decision,total cost of ownership (TCO), scalability, stability and performances. The TCO factor can be easy overcome with FLOSS (Free and open source software), scalability is a very broad term and concept  that doesn’t have any clear answer. Stability can be achieved with software that is  widely used and widely reviewed and tested for a long time, never pick something new just because it’s cool. Doesn’t matter if the website serve millions of users or just a few hundred, slow or bad performances can make the user experience a nightmare. Performance enhancement can be  achieved with, Memory caching system, Low memory footprint web servers, good and slim database design, query optimization to work under heavy loads. Please  be aware that applications, services and workloads differ from case to case, you might need more write capacity rather than read, serving multimedia files or whatever else, don’t blame me if you have troubles with your production application after following this how-to. For me and some other, this setup works quite well, I hope the same for you :-) .

This tutorial covers the set up a free, full-featured web development framework based on Ubuntu Server 12.04(LTS), Django 1.4 stable, Postgresql database V. 9.1, Nginx web server V.1.1.19 and Gunicorn.

Ubuntu Server
Linux for high-volume server workloads
http://www.ubuntu.com/business/server/overview

Django
The web framework for perfectionists with deadlines
www.djangoproject.com

Postgresql
A powerful, open source object-relational database system.
http://www.postgresql.org

Nginx
A web server with a strong focus on high concurrency, performance and low memory usage.
http://wiki.nginx.org/Main

Gunicorg
A WSGI HTTP Server, the link between Django an Nginx.
http://gunicorn.org/

Postgresql installation and setup

[sourcecode language="bash"]

apt-get install postgresql python-psycopg2

sudo -u postgres psql template1

ALTER USER postgres with encrypted password ‘a_strong_password';

template1=# CREATE DATABASE test;

template1=# CREATE USER test WITH PASSWORD ‘test';

template1=# GRANT ALL PRIVILEGES ON DATABASE test to test;
[/sourcecode]

In /etc/postgresql/9.1/main/pg_hba.conf

Change

[sourcecode language="bash"]
local all postgres peer

To

local all postgres md5

local all all peer

to

local all all md5
[/sourcecode]

Restart Postgresql server with:

/etc/init.d/postgresql restart

To thest the local connection to the database please use:

psql -d postgres -U postgres

and

psql -d test -U test

Django installation and setup.

[sourcecode language="bash"]
apt-get install aria2c

aria2c http://www.djangoproject.com/download/1.4/tarball/

tar xvfz Django-1.4.tar.gz

python setup.py install

mkdir /var/djangoweb/

django-admin.py startproject myproject

django-admin.py startproject myproject

cd myproject/

mkdir /var/djangoweb/myproject/static

ln -s /usr/local/lib/python2.7/dist-packages/django/contrib/admin /var/djangoweb/myproject/static/
[/sourcecode]

in myproject/settings.py

Add

[sourcecode language="bash"]
import os.path
PROJECT_DIR = os.path.dirname(__file__)
[/sourcecode]

Database setup in django

[sourcecode language="bash"]
DATABASES = {
‘default': {
‘ENGINE': ‘django.db.backends.postgresql_psycopg2′, # Add ‘postgresql_psycopg2′, ‘mysql’, ‘sqlite3′ or ‘oracle’.
‘NAME': ‘test’, # Or path to database file if using sqlite3.
‘USER': ‘test’, # Not used with sqlite3.
‘PASSWORD': ‘test’, # Not used with sqlite3.
‘HOST': ”, # Set to empty string for localhost. Not used with sqlite3.
‘PORT': ”, # Set to empty string for default. Not used with sqlite3.
}
}
[/sourcecode]

Then go to the INSTALLED_APPS section and uncomment
django.contrib.admin line.

[sourcecode language="bash"]
INSTALLED_APPS = (
‘django.contrib.auth’,
‘django.contrib.contenttypes’,
‘django.contrib.sessions’,
‘django.contrib.sites’,
‘django.contrib.messages’,
‘django.contrib.staticfiles’,
# Uncomment the next line to enable the admin:
# ‘django.contrib.admin’,
# Uncomment the next line to enable admin documentation:
# ‘django.contrib.admindocs’,
)
[/sourcecode]

and # ‘django.contrib.admin’,

[sourcecode language="bash"]
INSTALLED_APPS = (
‘django.contrib.auth’,
‘django.contrib.contenttypes’,
‘django.contrib.sessions’,
‘django.contrib.sites’,
‘django.contrib.messages’,
‘django.contrib.staticfiles’,
# Uncomment the next line to enable the admin:
‘django.contrib.admin’,
# Uncomment the next line to enable admin documentation:
# ‘django.contrib.admindocs’,
)
[/sourcecode]

in myproject/urls.py uncomment the following lines.

[sourcecode language="bash"]

from django.contrib import admin
admin.autodiscover()

url(r’^admin/’, include(admin.site.urls)),
[/sourcecode]

The  urls.py  file have to look like this.

[sourcecode language="bash"]
from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns(”,
# Examples:
# url(r’^$’, ‘myproject.views.home’, name=’home’),
#url(r’^myproject/’, include(‘myproject.foo.urls’)),

# Uncomment the admin/doc line below to enable admin documentation:
# url(r’^admin/doc/’, include(‘django.contrib.admindocs.urls’)),

# Uncomment the next line to enable the admin:
url(r’^admin/’, include(admin.site.urls)),
)
[/sourcecode]

Now the basic parameters are defined, we must perform the first database synchronization,
go on top of the project root and issue the “python manage.py syncdb” command.

You’ll see the creation of the first Django tables inside the database,  you also need to create
the administrator user.

[sourcecode language="bash"]
Creating tables …
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log

You just installed Django’s auth system, which means you don’t have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use ‘root’): phillip
E-mail address: phillip@foo.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL …
Installing indexes …
Installed 0 object(s) from 0 fixture(s)

[/sourcecode]

At this point the Django interaction with the database looks working properly.

In order to get access to Django administration page you need to run the
embedded python web server.

[sourcecode language="bash"]

python manage.py runserver 0.0.0.0:80
Validating models…

0 errors found
Django version 1.4, using settings ‘myproject.settings’
Development server is running at http://0.0.0.0:80/
Quit the server with CONTROL-C.

[/sourcecode]

Point the browser to:

http://server_ipaddress/admin/

Gunicorn installation and setup

Install gunicorn

[sourcecode language="bash"]

apt-get install git python-setuptools
git clone git://github.com/benoitc/gunicorn.git
cd gunicorn/
python setup.py install

[/sourcecode]

inside the root project folder myproject

vim gunicorn.conf.py

[sourcecode language="bash"]
bind = "127.0.0.1:8888"
workers = 2
#recommended (2 x $num_cores) + 1
[/sourcecode]

To run gunicor go in the root project folder and run:

gunicorn_django -c gunicorn.conf.py

To Daemonize the Gunicorn process add -D

gunicorn_django -c gunicorn.conf.py -D

Yum must get:

[sourcecode language="bash"]
gunicorn_django -c gunicorn.conf.py
2012-05-01 19:53:28 [6176] [INFO] Starting gunicorn 0.14.2
2012-05-01 19:53:28 [6176] [INFO] Listening at: http://127.0.0.1:8888 (6176)
2012-05-01 19:53:28 [6176] [INFO] Using worker: sync
2012-05-01 19:53:28 [6180] [INFO] Booting worker with pid: 6180
2012-05-01 19:53:28 [6181] [INFO] Booting worker with pid: 6181
[/sourcecode]

Nginx installation and configuration

[sourcecode language="bash"]

apt-get install nginx-full openssl

rm /etc/nginx/sites-enabled/default

touch /etc/nginx/sites-available/django

ln -s /etc/nginx/sites-available/django /etc/nginx/sites-enabled/django

[/sourcecode]

In /etc/nginx/sites-enabled/django please add:

[sourcecode language="bash"]

server {

listen server_ipaddress:80;
#server_name *.site1;

access_log /var/log/nginx/djangosite1.access.log;
error_log /var/log/nginx/djangosite1.error.log;

location / {
proxy_pass http://127.0.0.1:8888;

}

#location /static {
# autoindex on;
# root /root/blog/public_html/;
# }

location /static/admin {
autoindex on;
root /var/djangoweb/myproject/static/admin;
}

}
[/sourcecode]

/etc/init.d/nginx restart

Django admin web interface.

Now you can point the browser to  http://server_ipaddress/admin/, be aware the myproject directory is just the schema container for the project, no applications are yet configured or installed. I strongly advise to follow “Writing your first Django app” how-to, do not copy paste the code.

If you want to find more about django development please take a look to:

Django From the Ground Up video series
DjangoCon 2011
The Django book

I hope this document will be useful for who is struggling with Django setup. Stay tuned.


Comments

  1. Great article Phillip,

    To lighten the work and easy setup deploy your project you can use django-fagungis, DJANGO + FAbric + GUnicorn + NGInx + Supervisor deployment.

    Here the code and documentation: Django Fagungis.

    (Disclosure: it’s my project)

    thanks, Denis.

    • Hi Dennis,

      thanks very much for the feedback, I’ll give a shot to Django Fagungis .

      phillip

  2. If you want it to be even more solid and dependable, Apache + mod_wsgi.

    LAPP forever!

  3. Thanks for this page, it was really helpful: I’ve spent two days looking for blogs which could bring me the light on configuring Nginx+gunicorn on my ArchLinux box.

    Thanks to you, it’s done, and it works well.

    Keep writing ;)

  4. Hi Phillip,

    Following your intructions and stuck right at the end…
    “In /etc/nginx/sites-enabled/django please add”…

    Is this a file? does it have an extension? Gedit does not let me save what it thinks is a file called “django”.

    also using gunicorn as webserver and browsing to admin page in django it appears that the css is missing as there is just unformatted html.

    help would be appreciated.

    Thanks

    • Hi leigh,

      the /etc/nginx/sites-enabled/django file is the Nginx virtual host for
      Django application and it doesn’t have extension, are u using Gedit as
      root to edit the django virtual host file?, otherwise you won’t be able
      to save it.

      Phillip

  5. Thanks for that. Finally moving away from Windows so bit of a curve here! Am i right in thinking that nginx will serve my static admin media files rather than gunicorn in this case (and that is why i’m not seeing any images in my admin page when gunicorn is running but nginx is not running? )

    Anyhow nginx is failing with:
    “nginx: [emerg] bind() to 67.215.65.132:80 failed (99: Cannot assign requested address)
    nginx: configuration file /etc/nginx/nginx.conf test failed”

    Will this prevent nginx from running and thus preventing me from connecting via localhost:80 and also not seeing media files in my admin page? BTW i have no ideas why a public ip address is listed as this is all local.

    So close to sidestepping read only file system issue in Heroku….

    Cheers

    • Hi leigh,

      this nginx: [emerg] bind() to 67.215.65.132:80 failed (99: Cannot assign requested address)

      might be generated by another web server running on port 80, ex Apache. Try lsof -i TCP:80
      as root an see which process is binding to port 80.

      Phillip

  6. Hardcoded server ip address into /etc/nginx/sites-enabled/django and it works a treat (static admin media files loading up too).

    Thanks Phillip for your help and providing an accelerated guide to getting Django to play with the same serious stack that Instagram and co use.

    Cheers

  7. Hi,
    glad it worked and helped you to jump into Django arena.

    Phillip

  8. Hi Phillip,

    this post was very helpful for me, thanks a lot!

  9. Your link “www.djangoproject.com” on this page is broken….
    Instead of going to the django project it redirects to this page.

    http://bailey.st/blog/2012/05/02/ubuntu-django-postgresql-and-nginx-a-rock-solid-web-stack/www.djangoproject.com

  10. Hi Phillip. Thanks for this guide.
    When I sudo apt-get install aria2c
    I get: E: Unable to locate package aria2c
    I had to get it through wget (Ubuntu Server 12.04 fresh install today)

  11. First of all I am a newbie.
    Before gunicorn and nginx the admin page was great.
    After installing them I got 502 Bad Gateway and
    in /var/log/nginx/djangosite1.error.log:
    2012/06/25 21:40:15 [error] 5656#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 186.126.152.19, server: , request: “GET /admin/ HTTP/1.1″, upstream: “http://127.0.0.1:8888/admin/”, host: “201.155.276.132”
    In /etc/nginx/sites-enabled/django I have:
    listen 10.6.20.13:80;
    Everything else is the same as in your guide.
    -Any ideas? Thanks!

  12. Thanks,

    it’works perfectly !

    Julien

  13. In my case the “502 Bad Gateway” Error was caused because Gunicorn was not running after i rebooted the server.
    Run
    gunicorn_django -c gunicorn.conf.py -D
    and try again. That solved my problem.

  14. Hi Phillip. Thanks for this guide…
    when i wrote command “python manage.py syncdb”
    some error
    Traceback (most recent call last):
    File “manage.py”, line 8, in
    from django.core.management import execute_from_command_line
    ImportError: No module named django.core.management

    -Any ideas? Thanks!

  15. What is the best way of auto starting gunicorn? I get the 502 error until “gunicorn_django -c gunicorn.conf.py -D” is manually run after a reboot.
    Thanks,
    Ewan

Leave a Reply

Your email address will not be published / Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">