Category > Uncategorized

Ubuntu, Django, Postgresql and Nginx, a rock solid web stack.

» 02 May 2012 » In Uncategorized » 3 Comments

Lately I’ve been working on developing a web application project,   I guess everyone involved in such process had to make the uneasy decision to choose the right Web Stack to run the application.  Multiple factors usually drive the final decision,total cost of ownership (TCO), scalability, stability and performances. The TCO factor can be easy overcome with FLOSS (Free and open source software), scalability is a very broad term and concept  that doesn’t have any clear answer. Stability can be achieved with software that is  widely used and widely reviewed and tested for a long time, never pick something new just because it’s cool. Doesn’t matter if the website serve millions of users or just a few hundred, slow or bad performances can make the user experience a nightmare. Performance enhancement can be  achieved with, Memory caching system, Low memory footprint web servers, good and slim database design, query optimization to work under heavy loads. Please  be aware that applications, services and workloads differ from case to case, you might need more write capacity rather than read, serving multimedia files or whatever else, don’t blame me if you have troubles with your production application after following this how-to. For me and some other, this setup works quite well, I hope the same for you :-) .

This tutorial covers the set up a free, full-featured web development framework based on Ubuntu Server 12.04(LTS), Django 1.4 stable, Postgresql database V. 9.1, Nginx web server V.1.1.19 and Gunicorn.

Ubuntu Server
Linux for high-volume server workloads
http://www.ubuntu.com/business/server/overview

Django
The web framework for perfectionists with deadlines
www.djangoproject.com

Postgresql
A powerful, open source object-relational database system.
http://www.postgresql.org

Nginx
A web server with a strong focus on high concurrency, performance and low memory usage.
http://wiki.nginx.org/Main

Gunicorg
A WSGI HTTP Server, the link between Django an Nginx.
http://gunicorn.org/

Postgresql installation and setup


apt-get install postgresql python-psycopg2

sudo -u postgres psql template1

ALTER USER postgres with encrypted password 'a_strong_password';

template1=# CREATE DATABASE test;

template1=# CREATE USER test WITH PASSWORD 'test';

template1=# GRANT ALL PRIVILEGES ON DATABASE test to test;

In /etc/postgresql/9.1/main/pg_hba.conf

Change

local   all             postgres                                peer

To

local   all             postgres                                md5

local   all             all                                     peer

to

local   all             all                                     md5

Restart Postgresql server with:

/etc/init.d/postgresql restart

To thest the local connection to the database please use:

psql -d postgres -U postgres

and

psql -d test -U test

Django installation and setup.

apt-get install aria2c

aria2c http://www.djangoproject.com/download/1.4/tarball/

tar xvfz Django-1.4.tar.gz

python setup.py install

mkdir /var/djangoweb/

django-admin.py  startproject myproject

django-admin.py  startproject myproject

cd myproject/

mkdir /var/djangoweb/myproject/static

ln -s /usr/local/lib/python2.7/dist-packages/django/contrib/admin /var/djangoweb/myproject/static/

in myproject/settings.py

Add

import os.path
PROJECT_DIR = os.path.dirname(__file__)

Database setup in django

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'test',                      # Or path to database file if using sqlite3.
        'USER': 'test',                      # Not used with sqlite3.
        'PASSWORD': 'test',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
    }
}

Then go to the INSTALLED_APPS section and uncomment
django.contrib.admin line.

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # Uncomment the next line to enable the admin:
    # 'django.contrib.admin',
    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
)

and # ‘django.contrib.admin’,

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # Uncomment the next line to enable the admin:
    'django.contrib.admin',
    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
)

in myproject/urls.py uncomment the following lines.


from django.contrib import admin
admin.autodiscover()

url(r'^admin/', include(admin.site.urls)),

The  urls.py  file have to look like this.

from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'myproject.views.home', name='home'),
    #url(r'^myproject/', include('myproject.foo.urls')),

    # Uncomment the admin/doc line below to enable admin documentation:
    # url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

    # Uncomment the next line to enable the admin:
    url(r'^admin/', include(admin.site.urls)),
)

Now the basic parameters are defined, we must perform the first database synchronization,
go on top of the project root and issue the “python manage.py syncdb” command.

You’ll see the creation of the first Django tables inside the database,  you also need to create
the administrator user.

Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'root'): phillip
E-mail address: phillip@foo.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

At this point the Django interaction with the database looks working properly.

In order to get access to Django administration page you need to run the
embedded python web server.


python manage.py runserver 0.0.0.0:80
Validating models...

0 errors found
Django version 1.4, using settings 'myproject.settings'
Development server is running at http://0.0.0.0:80/
Quit the server with CONTROL-C.

Point the browser to:

http://server_ipaddress/admin/

Gunicorn installation and setup

Install gunicorn


apt-get install git python-setuptools
git clone git://github.com/benoitc/gunicorn.git
cd gunicorn/
python setup.py install

inside the root project folder myproject

vim gunicorn.conf.py

bind = "127.0.0.1:8888"
workers = 2
#recommended (2 x $num_cores) + 1

To run gunicor go in the root project folder and run:

gunicorn_django -c gunicorn.conf.py

To Daemonize the Gunicorn process add -D

gunicorn_django -c gunicorn.conf.py -D

Yum must get:

gunicorn_django -c gunicorn.conf.py
2012-05-01 19:53:28 [6176] [INFO] Starting gunicorn 0.14.2
2012-05-01 19:53:28 [6176] [INFO] Listening at: http://127.0.0.1:8888 (6176)
2012-05-01 19:53:28 [6176] [INFO] Using worker: sync
2012-05-01 19:53:28 [6180] [INFO] Booting worker with pid: 6180
2012-05-01 19:53:28 [6181] [INFO] Booting worker with pid: 6181

Nginx installation and configuration


apt-get install nginx-full  openssl

rm /etc/nginx/sites-enabled/default

touch /etc/nginx/sites-available/django

ln -s /etc/nginx/sites-available/django  /etc/nginx/sites-enabled/django

In /etc/nginx/sites-enabled/django please add:


server {

listen   server_ipaddress:80;
#server_name *.site1;

access_log /var/log/nginx/djangosite1.access.log;
error_log /var/log/nginx/djangosite1.error.log;

location / {
        proxy_pass http://127.0.0.1:8888;

}

#location /static {
#         autoindex on;
#         root /root/blog/public_html/;
#    }

location /static/admin {
         autoindex on;
         root /var/djangoweb/myproject/static/admin;
    }

}

/etc/init.d/nginx restart

Django admin web interface.

Now you can point the browser to  http://server_ipaddress/admin/, be aware the myproject directory is just the schema container for the project, no applications are yet configured or installed. I strongly advise to follow “Writing your first Django app” how-to, do not copy paste the code.

If you want to find more about django development please take a look to:

Django From the Ground Up video series
DjangoCon 2011
The Django book

I hope this document will be useful for who is struggling with Django setup. Stay tuned.

Did you like this? Share it:
Scridb filter

Continue reading...

Tags: , ,

April 2012 events – Nothing will happen (Belgrade) – Richard Stallman speech (Zagreb)

» 16 April 2012 » In Uncategorized » No Comments

Nothing will happen in Belgrade 2012

In the next coming week a few events are taking place over Belgrade and Zagreb, as usual on the middle of april I will attend for nista neće dogoditi (Nothing will happen), an tech unconference for hackers from former Yugoslavian countries. This year the conference is hosted inside the newborn “Hacklab Belgrade” on April 21 and 22, I’m really  eager to visit the hackerspace and to see on what the crew is up to. Nothing will happen is an unique event, no schedule, no plan, but all of sudden things come together, and that is the moment  where to prove your hacker attitude. Hope to see you in Belgrade.

How to get to the Hacklab: http://oosm.org/content/lokacija

 Richard Stallman: For a Free Digital Society

On Monday, 23th April 2012 18:00, at the Zagreb Museum of Contemporary Art, Richard Stallman will deliver a speech  on the subject “For a Free Digital Society”.

“Activities directed at “including” more people in the use of digital technology are predicated on the assumption that such inclusion is invariably a good thing.  It appears so, when judged solely by immediate practical convenience.  However, if we also judge in terms of human rights, whether digital inclusion is good or bad depends on what kind of digital world we are to be included in.  If we wish to work towards digital inclusion as a goal, it behooves us to make sure it is the good kind.”

23th April 2012 18:00
Richard Stallman “For a Free Digital Society”
Location:
Museum of Contemporary Art
Av. Dubrovnik 17, 10000 Zagreb, Croatia
www.msu.hr

Addmission: Free – No registration required.

Event coordinator: Tomislav Medak, tom@mi2.hr

Did you like this? Share it:
Scridb filter

Continue reading...

Tags:

Dirt cheap USB Temperature Sensor with python SMS alerting system.

» 12 April 2012 » In Uncategorized » No Comments


In the last few weeks I had to design an effective and cheap temperature monitoring system for the server room. After searching a bit on the web I found this USB Thermometer Temperature dongle on Ebay, only 10 bucks with free shipping. As soon the sensor arrived, I plugged it into one Ubuntu based server and surprisingly worked immediately, below you can see the dmesg output.

[12737.064004] usb 2-1.3: new low speed USB device using ehci_hcd and address 6
[12737.161227] usb 2-1.3: configuration #1 chosen from 1 choice
[12737.164379] input: RDing TEMPer1V1.1 as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.0/input/input6

Along with monitoring the temperature I wanted to have an alarming system that sends alarm SMS message to mobile phones when the temperature has passed a certain threshold. So I subscribed to textmagic, an SMS gateway provider. Textmagic has a good set of API, and of course they have the python API which I used to write my Pytemp2sms.py script.

USB dongle software and settings.

wget http://www.isp-sl.com/pcsensor-0.0.1.tgz
tar xvfz pcsensor-0.0.1.tgz
cd pcsensor-0.0.1
cp 99-tempsensor.rules /etc/udev/rules.d/
cp pcsensor /usr/local/bin/

Reboot the server.

To test the dongle run the pcsensor command .
You must get this kind of output.

2012/04/11 14:30:50 Temperature 60.58F 15.88C

SMS alarm.

Textmagic python API installation.

https://code.google.com/p/textmagic-sms-api-python/wiki/UserManual


apt-get install python-setuptools
easy_install PyTextMagicSMS

Pytemp2sms script.

Script download: http://bailey.st/code/bitsofpy/Pytemp2sms/


#!/usr/bin/python
import sys
import subprocess
import fileinput
import textmagic.client

#Temperature threshold in C
threshold = 5
#Balance threshold
credits = 100
#TextMagic account details, before using textmagic service
#you need to register and obtain an API password
#(different from your login password)
#
#https://www.textmagic.com/app/wt/account/api/cmd/password
username = "yourusername"
password = "APIpassword"
#Owner of the Textmagic sms account.This number will receive
#the credit balance .
owner = "+44phonenumberoftheowneraccount"

'''License: This software is distributed under
   the terms  and  conditions  of GPL  - GNU
   General  Public  License.'''

def usage():

    print " ------------------------------------------"
    print "|              Pytemp2sms                  |"
    print "|                                          |"
    print "|Temperature Sensor with sms alert system. |"
    print "|                                          |"
    print "|www.bailey.st                             |"
    print "|phillip@bailey.st                         |"
    print " ------------------------------------------ "

    print "\n"
    print "Usage instructions:"
    print "\n"
    print "Pytemp2sms.py checktemp | checkbalance"
    print "\n"

def checktemp():

    output = subprocess.Popen(["/usr/local/bin/pcsensor"], stdout=subprocess.PIPE).communicate()[0]
    output2 = output.split()[4]
    temp = output2[:2]

    if int(temp) > threshold:

        for phonebook in fileinput.input("/usr/local/bin/phonebook.txt"):

            client = textmagic.client.TextMagicClient(username, password)
            client.send("WARNING - Server room temperature above threshold!!! PLEASE CHECK!!!", phonebook)

    else:
        sys.exit()

def checkbalance():

    client = textmagic.client.TextMagicClient(username, password)
    balance = client.account()['balance']
    print balance

    if balance < credits:
        client.send("WARNING - Textmagic credit balance below threshold, please top the account", owner)

    else:
            sys.exit()

if len(sys.argv) < 2:
    usage()
    sys.exit()

if "checktemp" in sys.argv[1]:
        checktemp()

if "checkbalance" in sys.argv[1]:
        checkbalance()

Phonebook file.

The file phonebook.txt must contain all the mobile numbers that need to receive the alarm sms. Please follow this syntax:

+44numberone
+44numbertwo
+44numberthree

Crontabs.


*/30 * * * * /usr/local/bin/Pytemp2sms.py checktemp >/dev/null 2>&1
*/60 * * * * /usr/local/bin/Pytemp2sms.py checkbalance >/dev/null 2>&1
Did you like this? Share it:
Scridb filter

Continue reading...

Tags:

WordPress and Nginx

» 04 April 2012 » In Uncategorized » 1 Comment

Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and it does five of them 50 times faster than Apache.

Chris Lea on nginx and WordPress.

Nginx is a lightweight Web server with a small memory footprint and exceptional flexibility, along with this features the Nginx design is focused to ensure the maximum security. The configuration may look non-intuitive and not so straightforward, but a strong community and an excellent documentation will clear all your doubts. In this how-to I’m going to share my experience on moving a few WordPress blogs from Apache to Nginx, one of the blog is served inside the root directory and the other one is served as WordPress sub directory.

Please note, this how to applies to Ubuntu 10.04 LTS.

php5-fpm installation and configuration.

In order to serve PHP dynamic content we need to install php5-fpm.


deb http://ppa.launchpad.net/brianmercer/php/ubuntu lucid main
deb-src http://ppa.launchpad.net/brianmercer/php/ubuntu lucid main

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8D0DC64F
apt-get update
apt-get  install  php5-fpm

Once php5-fpm is installed we need to create a configuration file www.conf ,
located into /etc/php5/fpm/pool.d/ . Please copy the following values into it.


[www]

user = www-data
group = www-data
listen = 127.0.0.1:9000
pm = dynamic
pm.max_children = 10
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6
chdir = /

Restart the php5-fpm service.

/etc/init.d/php5-fpm restart

Nginx installation and configuration.

apt-get  install nginx

It wise to disable the default site configuration.

unlink /etc/nginx/sites-enabled/default

Now, let’s begin to create our first site configuration, the one
installed inside the root directory and served as www.example1.com .

In /etc/nginx/sites-available/example1.com paste, wich example1.com
is your domain name.

Example NR.1


server {
        listen   80;
        server_name  example1.com;

        access_log  /var/log/nginx/example1.com.access.log;
        error_log  /var/log/nginx/example1.com.error.log;

        location / {
                root   /var/www/example1.com;
                index  index.php index.htm;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        location ~ \.php$ {
                fastcgi_pass   127.0.0.1:9000;
                fastcgi_index  index.php;
                fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
        }

}

To enable example1.com please create a soft link to:

sudo ln -s /etc/nginx/sites-available/example1.com /etc/nginx/sites-enabled/example1.com

Restart Nginx with # /etc/init.d/nginx restart and point the browser to your domain name.

Example NR.2, with wordpress served as www.example2.com/blog

In /etc/nginx/sites-available/example2.com paste, wich example2.com
is your domain.


server {
    server_name     example2.com;
    index           index.php;
    root            /var/www/example2.com;

        access_log  /var/log/nginx/example2.com.access.log;
        error_log  /var/log/nginx/example2.com.error.log;

if (!-e $request_filename) {
rewrite ^.+/?(/ms-.*) $1 last;
rewrite ^/files/(.+) /wp-includes/ms-files.php?file=$1 last;
rewrite ^.+/?(/wp-.*) $1 last;
rewrite ^.+/?(/.*.php)$ $1 last;
rewrite ^(.+)$ /index.php?q=$1 last;
break;
}

    location ~ .php$ {
                fastcgi_pass   127.0.0.1:9000;
                fastcgi_index  index.php;
                fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
}
}

To enable example1.com please create a soft link to:

sudo ln -s /etc/nginx/sites-available/example2.com /etc/nginx/sites-enabled/example2.com

Please restart the Nginx server and test your new blog # /etc/init.d/nginx and point the browser to www.example2.com/blog

Did you like this? Share it:
Scridb filter

Continue reading...

Zenoss over SSL with Nginx reverse proxy.

» 24 March 2012 » In Uncategorized » No Comments

Zenoss is an enterprise infrastructure monitoring tool that can give in real time the big picture of networks, applications, hardware performances and issues running from a small business to a cloud service. While using it I found the lack of a decent SSL support running out off the box, in order to overcome this issue I decided to use Nginx as reverse proxy server to handle and deliver Zenoss with the SSL support. For this example I’m using Zenoss a debian Native Stack installed on Ubuntu server 10.04

Zenoss download:

http://community.zenoss.org/docs/DOC-3240?noregister

Preparing zenoss:

You need to locate the file zope.conf, which in this case is locate into /usr/local/zenoss/zenoss/etc/zope.conf , and find the line that contains “# ip-address 127.0.0.1″ and uncomment it as follows.

# ip-address 127.0.0.1

to

ip-address 127.0.0.1

After you are saved the zope.conf file you need to restart the Zope server as zenoss user.

su – zenoss

zopectl restart

Nginx

You need to install Nginx and the openssl utilities, create an ssl directory to store the certificates and create the certificates.


apt-get install nginx openssl

mkdir /etc/nginx/ssl

cd /etc/nginx/ssl

openssl req -new -x509 -days 365 -nodes -out zenoss.pem -keyout zenoss.key

zenoss.key zenoss.pem

rm /etc/nginx/sites-enabled/default

touch /etc/nginx/sites-enabled/default


server {
    listen 443 default ssl;
    server_name myserver;

      ssl on;
      ssl_certificate      /etc/nginx/ssl/zenoss.pem;
      ssl_certificate_key  /etc/nginx/ssl/zenoss.key;

location / {
        rewrite ^(.*)$ /VirtualHostBase/https/serveripaddress:443$1 break;
        proxy_pass http://127.0.0.1:8080;
    }
}

Restart Nginx before the login.

/etc/init.d/nginx restart

Restarting nginx: nginx.

At this point everything should work smoothly, now you can test your new Zenoss over SSL pointing the browser to https://serveripaddress

Did you like this? Share it:
Scridb filter

Continue reading...

Tags: