Roshan Book

My Tech Notebook

Category Archives: Web Tech

Django Deployment Guide For Ubuntu

Source:http://www.ventanazul.com/webzine/tutorials/django-deployment-guide-ubuntu

Ignite your Django projects with nginx, Apache and mod_wsgi

There’s a time when every Django developer has to think about deployment scenarios, when I first did it last year I thought that a setup involving Lighty, Apache, mod_proxy and mod_python was a good choice but my first approach was not the best. I put Apache as the front server, handling requests for Django generated pages and passing, via mod_proxy, requests for static content to Lighty on the back. A setup where Apache had to work even for files that wasn’t supposed to serve was a very bad idea.

After many helpful comments and some more reading I realized that it was better having the server for static content on the front and Apache, which still talks to Django, on the back.

I replaced Lighty with nginx, which according to many seems to be more stable, and opted for mod_wsgi instead of mod_python to make Apache talk to Django. mod_wsgi has a lower memory overhead and it’s suitable for high performance sites. There’s no need for mod_proxy on Apache anymore as nginx is the one in charge of the proxy work now.

This is an easy to follow and very focused guide for developers who know how to handle their servers so I won’t consider security issues, memcached, Django installation, databases or basic GNU/Linux, Apache and DNS settings. Of course all of those subjects are important and you should take care of them.

The plan

This guide includes all the steps needed to:

  • Setup a domain for your Django project.
  • Create a simple directory layout for Django sites.
  • Configure Apache with mod_wsgi for Django.
  • Configure nginx.
  • Serve Django admin media files.
  • Turn on the heat and show your greatest and latest Django stuff to the world.

After following all the steps you will have a Django site running with nginx on the front and Apache on the back. nginx will manage all static content and will pass Django requests to Apache and mod_wsgi.

I have tested on three Ubuntu servers (two running 8.10, Intrepid Ibex, and one 7.10, Gutsy Gibbon) but everything should be pretty similar in other GNU/Linux distributions.

Shall we start?

Setup a domain for your Django project

Many of us run more than one website in one server, I have several with Drupal and a few with Django living in one box, so let’s consider a two domains scenario: cataybea.com will host the Django project andexample.com anything else. Most of the work will focus in cataybea.com, I just mention example.com to give you some context. Obviously you should replace all the domain names in this guide with yours.

We’ll use a couple of private IP addresses for now and will replace them with public ones and setup in a DNS server when going live. For this guide I’ll just add to /etc/hosts, which is what I usually do in my development environment:

192.168.0.180 cataybea.com 192.168.0.192 example.com

I’m using 192.168.0.180 and two ports, 80 and 8080, for serving the Django site at cataybea.com. The default port 80 will be used by nginx to serve static content and Django will look for files under the /media directory. Port 8080 will be used by Apache to handle Django requests.

Create a simple directory layout for Django sites

Directory layouts depend on personal preferences, I currently use a /home/alexis/djcode directory for all my Django coding and the new project will be called cataybea; hence, we have to run:

mkdir /home/alexis/djcode cd /home/alexis/djcode/ django-admin.py startproject cataybea

Now let’s create some additional directories:

mkdir /home/alexis/djcode/cataybea/apache mkdir /home/alexis/djcode/cataybea/logs mkdir /home/alexis/djcode/cataybea/media

What are they for? apache will contain a Python script to setup mod_wsgi, logs will store Apache and nginx logs and media is the directory nginx will use to serve static files for our Django project.

You can later manage the whole /home/alexis/djcode/cataybea directory with your favorite version control tool.

Take note of the correct paths for using later in the configuration files.

Configure Apache with mod_wsgi for Django

I assume you already have Apache working correctly and just need to tweak a little for our Django setup. As we’ll be running two web servers at once we must make sure that IP addresses and ports won’t conflict. Let’s edit /etc/apache2/ports.conf:

Listen 192.168.0.192:80 Listen 192.168.0.180:8080 #Listen 80

I have specified IP addresses and ports to listen. Notice I commented the default Listen 80 as it means Apache would listen to that port in all IP addresses. If you want to use SSL take care of the 443 port in the same way.

If you are using virtual hosts in Apache confirm it’s listening to the correct IP addresses and ports, I have this in/etc/apache2/sites-enabled/example.com:

NameVirtualHost 192.168.0.192

Restart Apache and make sure your Apache sites work normally. I’m obsessed with testing at every step in the way and I suggest you are too.

sudo /etc/init.d/apache2 restart

Now it’s time to add mod_wsgi to Apache. The latest versions of Ubuntu have it in the repository:

sudo apt-get install libapache2-mod-wsgi

For older versions of Ubuntu, such as Gutsy, you need to find the .deb file and use something like dpkg -i to install.

Now it’s time to create the Apache configuration file for cataybea.com in /etc/apache2/sites-available/cataybea.com:

<VirtualHost 192.168.0.180:8080> ServerAdmin alexis@ventanazul.com ServerName http://www.cataybea.com ServerAlias cataybea.com <Directory /home/alexis/djcode/cataybea/apache/> Order deny,allow Allow from all </Directory> LogLevel warn ErrorLog /home/alexis/djcode/cataybea/logs/apache_error.log CustomLog /home/alexis/djcode/cataybea/logs/apache_access.log combined WSGIDaemonProcess cataybea.com user=www-data group=www-data threads=25 WSGIProcessGroup cataybea.com WSGIScriptAlias / /home/alexis/djcode/cataybea/apache/cataybea.wsgi </VirtualHost>

Notice the IP address and port: 192.168.0.180:8080. This site won’t be accessed directly but via proxy from nginx. We’ll set that up in the next section.

Now we need a Python script to configure our Django project to use mod_wsgi, create/home/alexis/djcode/cataybea/apache/cataybea.wsgi and put the following code inside:

import os, sys apache_configuration= os.path.dirname(__file__) project = os.path.dirname(apache_configuration) workspace = os.path.dirname(project) sys.path.append(workspace) sys.path.append(‘/usr/lib/python2.5/site-packages/django/’) sys.path.append(‘/home/alexis/djcode/cataybea’) os.environ[‘DJANGO_SETTINGS_MODULE’] = ‘cataybea.settings’ import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler()

Don’t restart Apache yet as we need to complete a few more steps.

Configure nginx

nginx will do two things: serve static content from http://cataybea.com/media and pass all other requests to Apache. Let’s install it first:

sudo apt-get install nginx

Now remove the nginx default site:

sudo rm /etc/nginx/sites-enabled/default

Apache and nginx should be using the same user, for Ubuntu this is www-data, and your /etc/nginx/nginx.confshould look like this:

user www-data; worker_processes 2; error_log /var/log/nginx/error.log; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; access_log /var/log/nginx/access.log; sendfile on; tcp_nopush on; keepalive_timeout 65; tcp_nodelay on; gzip on; include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

worker_processes can be set to the number of cores in your server, in my case that’s just 2.

Configure a the cataybea.com site on nginx by creating /etc/nginx/sites-available/cataybea.com:

server { listen 192.168.0.180:80; server_name http://www.cataybea.com cataybea.com; access_log /home/alexis/djcode/cataybea/logs/nginx_access.log; error_log /home/alexis/djcode/cataybea/logs/nginx_error.log; location / { proxy_pass http://192.168.0.180:8080; include /etc/nginx/proxy.conf; } location /media/ { root /home/alexis/djcode/cataybea/; } }

then running:

sudo ln -s /etc/nginx/sites-available/cataybea.com /etc/nginx/sites-enabled/cataybea.com

and finally creating the file that will take care of proxying Django requests to Apache, we’ll call it/etc/nginx/proxy.conf and put this inside:

proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k;

Notice $host, $remote_addr and $proxy_add_x_forwarded_for are variables that will be handled by nginx, you don’t need to change anything else in proxy.conf for a basic setup. When I started playing with nginx I thought these had to be replaced by me, I was wrong.

Finally restart nginx:

sudo /etc/init.d/nginx restart

and Apache:

sudo /etc/init.d/apache2 restart

That should be it! If you visit http://cataybea.com you should see some Django content (could be the welcome page, some of your views or a 404 page depending on your project status). To try the nginx media server create a test.htm file with some dummy content in /home/alexis/djcode/cataybea/media and visithttp://cataybea.com/media/test.htm.

Serve Django admin media files (optional)

If you will use the Django admin application edit /home/alexis/djcode/cataybea/settings.py, make sure your database settings are correct, add ‘django.contrib.admin’ to the INSTALLED_APPS tuple and change theADMIN_MEDIA_PREFIX like this:

ADMIN_MEDIA_PREFIX = ‘/media/admin/’

Let’s enable the admin url by editing /home/alexis/djcode/cataybea/urls.py, uncommenting these two lines:

from django.contrib import admin admin.autodiscover()

and adding this to urlpatterns:

(r’^admin/’, include(admin.site.urls)),

Now let’s create a symbolic link to the media files for the Django’s administration section:

cd /home/alexis/djcode/cataybea/media ln -s /usr/lib/python2.5/site-packages/django/contrib/admin/media/ admin

And finally let’s create the administration user and the required tables in the database:

cd /home/alexis/djcode/cataybea/media python manage.py syncdb

As with every other modifications in your Django code you will have to reload Apache for changes to take effect:

sudo /etc/init.d/apache2 reload

Visit http://cataybea.com/admin/,including the trailing slash, and you should be in business.

Advertisements

Running yacy on ubuntu

Post was originally published at : http://www.muktware.com/articles/3029

YaCy is an open source, decentralized search engine which allows users to run their own instances of search engines without being tracked by someone else or being controlled by totalitarian governments.

It is extremely easy to install YaCy in Ubuntu or other debian based systems. First you need to add YaCy repository to your system, open the terminal and run this command
sudo gedit /etc/apt/source.list

It will open the source list file in Gedit, at the end of the file copy paste this:
deb http://debian.yacy.net ./

save the file and close. Now go back to the terminal and run this command:
sudo apt-get update

Then install YaCy by running this command
sudo apt-get install yacy

It will start installing YaCy, it will prompt you to type ‘y’ for yes couple of times, then you will be greeted by this window which will create your PC as a peer to be used by YaCy. You can choose whatever name you want. Hit Enter

YaCy in Ubuntu

Then you will be greeted by this window asking your root password, enter the password.

Install YaCy in Ubuntu
The you can choose what kind of search engine you want. If you want general purpose internet search, like Google, choose freeworld. You can also enable search for specific sites (like Muktware) ONLy, by choosing webportal. Note: you wont get results from anyother side excepet for that one. S, if you want a general purpose search engine go for freeworld.

Install YaCy in Ubuntu
Since YaCy relies on Java, if will ask you to set initial memory for Java, just keep it what it suggest and hit enter, again a similar windows asking for maximum memory, keep the one suggested enter. Now you will be back to the terminal.

Install YaCy in Ubuntu

Install YaCy in Ubuntu

To run YaCy enter this URL in the browser

http://localhost:8090/

Install YaCy in Ubuntu

You will be running YaCy. Bookmark it or make it your home page!

Macros in Eclipse

How to Get your Twitter and Facebook Stream inside Google+

This post was originally published at http://www.techstuffs.net/2011/07/how-to-get-your-twitter-and-facebook.htmlGoogle is trying to take on Facebook by launching a competitor with some awesome features which you won’t find in Facebook. Although, Facebook has tried to make Google+ down by launching a much needed feature Facebook Video Calling which is powered by Skype.Google Plus can give some tough competition to Twitter and Facebook if you’re addicted to Google+ and don’t want to leave it, at the same want to access twitter and Facebook on Google+ here is how.


CrossRider created two extensions for Google+, “Google+Facebook” and “Google+Tweet” which lets G+ users to view and access – Facebook updates, Tweets from Google Plus even you can posts updates on twitter and Facebook from Google+ it self.

How To Add Facebook Stream To Your Google+ Account:

 

1. Download and install Google+Facebook from any of Firefox/Chrome or Internet Explorer Browser.
2. Now sign into Google+ and Facebook button appears on top of the page  in navigation bar behind Home button.
3. Click it and further proceed to Connect to your Facebook account by clicking “F connect with Facebook” button and “allow“ the app to connect to your FB account to see your Facebook stream inside Google+.


You can now view all your Facebook updates and this extension also allows to share your status/ post updates to wall from Google plus it self .
Note: You’re giving Google+Facebook app to access Facebook data such as Basic Information, Post to Wall, Access posts in your News Feed and can access your data any time on Facebook. So be careful and you can always remove this app access by signing into your Facebook account.

How To Add Twitter Stream To Your Google+ Account:

 

Google+Tweet is a twitter client for Google+ allows to view your timeline, URL shortening, Photo and video sharing, and also translate tweets inside your Google Plus account.
1.Download and install 
Google+ Tweet for your desired browser to choose from – IE, Firefox and Google Chrome Browsers.
2. Sign into Google+ and click “Twitter” icon shown on top of the page behind Facebook button (appear if you installed and activated Google+ Facebook app), click it.
3 . Sign into your Twitter account and authorize this app to connect to your account via “oauth”.
4. After successful integration view your timeline on Google+.

  
Note: You’re giving this app permission to write data to your account on your behalf, so be careful again (this app can’t see  your twitter account password) and you can revoke access to this appany time from 
Apps Tab of your settings.

 

 

Honeypot Captcha

This post was originally published at http://haacked.com/archive/2007/09/11/honeypot-captcha.aspx

 

I was thinking about alternative ways to block comment spam the other day and it occurred to me that there’s potentially a simpler solution than the Invisible Captcha approach I wrote about.

The Invisible Captcha control plays upon the fact that most comment spam bots don’t evaluate javascript. However there’s another particular behavioral trait that bots have that can be exploited due to the bots inability to support another browser facility.

honeypot image from http://www.cs.vu.nl/~herbertb/misc/shelia/You see, comment spam bots love form fields. When they encounter a form field, they go into a berserker frenzy (+2 to strength, +2 hp per level, etc…) trying to fill out each and every field. It’s like watching someone toss meat to piranhas.

At the same time, spam bots tend to ignore CSS. For example, if you use CSS to hide a form field (especially via CSS in a separate file), they have a really hard time knowing that the field is not supposed to be visible.

To exploit this, you can create a honeypot form field that should be left blank and then use CSS to hide it from human users, but not bots. When the form is submitted, you check to make sure the value of that form field is blank. For example, I’ll use the form field named body as the honeypot. Assume that the actual body is in another form field named the-real-body or something like that:

<div id="honeypotsome-div">
If you see this, leave this form field blank 
and invest in CSS support.
<input type="text" name="body" value="" />
</div>

Now in your code, you can just check to make sure that the honeypot field is blank…

if(!String.IsNullOrEmpty(Request.Form["body"]))
  IgnoreComment();

I think the best thing to do in this case is to act like you’ve accepted the comment, but really just ignore it.

I did a Google search and discovered I’m not the first to come up with this idea. It turns out that Ned Batchelder wrote about honeypots as a comment spam fighting vehicle a while ago. Fortunately I found that post after I wrote the following code.

For you ASP.NET junkies, I wrote a Validator control that encapsulates this honeypot behavior. Just add it to your page like this…

<sbk:HoneypotCaptcha ID="body" ErrorMessage="Doh! You are a bot!"
  runat="server"  />

This control renders a text box and when you call Page.Validate, validation fails if the textbox is not empty.

This control has no display by default by setting the style attribute todisplay:none. You can override this behavior by setting theUseInlineStyleToHide property to false, which makes you responsible for hiding the control in some other way (for example, by using CSS defined elsewhere). This also provides a handy way to test the validator.

To get your hands on this validator code and see a demo, download the latestSubkismet source from CodePlex. You’ll have to get the code from source control because this is not yet part of any release.

Reverting back to windows boot loader

1. Remove the Ubuntu partitions (EXT3 and SWAP) and resize the Windows partition (NTFS) to use that space.

To do this, you can follow the blog post I made which uses the Gparted LiveCD. The NTFS partition is Windows, and the EXT3 and SWAP partitions were created by Ubuntu.

http://useopensource.blogspot.com/2006/12/easy-way-to-resize-hard-drive.html

2. Fix the MBR (Master Boot Record) so that Windows will boot, instead of GRUB.

Boot from the Windows XP CD.
Rress the “R” key in the setup to start the restoration console.
Select your windows XP installation from the list, and enter the administrator password.
Enter the command: “FIXMBR” (without the quotes) at the input prompt and confirm the next question with a “Y” (without the quotes).
Use exit to restore the computer.

If you do not have your Windows XP cd, then you can use the SuperGrub LiveCD to fix your MBR.

http://supergrub.forjamari.linex.org/

Another option is to boot to a floppy and use “FDISK.EXE /MBR”, but the first two options are much better. Good luck!

Easy way to embed a mp3 audio on website

Install Lamp stack on Ubuntu linux

To install lamp stack on ubuntu linux, follow the following steps

1. Hit Ctrl+alt+t – This will open up a terminal window

2. Enter sudo apt-get install tasksel

3. sudo  tasksel

4. A window will open

5. Select lamp by clicking space

6. Hit enter, and will will install LAMP stack

Install and Enable mod_python 

Python is installed by default on the machine. However, to be able to use python with your web server, you must install the Apache module mod_python with the following command:
apt-get install libapache2-mod-python

To enable python support for the example site “ducklington.org”, you need to add the following to the site’s VirtualHost entry in that site’s configuration file:

File excerpt: /etc/apache2/sites-available/ducklington.org

AddHandler mod_python .py PythonHandler mod_python.publisher PythonDebug On

If you need support for MySQL you must also install python-mysqldb which you can accomplish with:
apt-get install python-mysqldb

Restart Apache

Finally, restart Apache to make sure everything is loaded correctly:

/etc/init.d/apache2 restart

Configuring Virtual host

When we develop on Zend framework , to check your progress you need to go to http://localhost/phpProj/zend/dev.zendtest.com/public , time and again to get to the index page. We can conveniently set up a virtual host on our local machine, so that if we type dev.zendtest.com we could get to the above mentioned link.

Below are the steps to configure virtual host

Before this, visit /etc/apache2 and see the folders residing there.

Notice two specific folder

a) sites-available

b) sites-enabled

Sites-available is a folder which contains configuration file for each site, which you want to enable

sites-enabled mainly contains link to sites-available. link present inside sites-enabled folder

Suppose we make a new zend framework enabled website in /var/www/zend/dev.zendtest.com , and our objective is to get to its’s public/index.php when we open dev.zendtest.com

Step 1. Create  a new file inside sites-available folder and paste following code :

<VirtualHost *>
ServerAdmin webmaster@localhost
ServerName dev.zendtest.com
DocumentRoot /var/www/zend/dev.zendtest.com/public
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
<Directory /var/www/zend/dev.zendtest.com/public/>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
</VirtualHost>

Step 2. Save the file

Step 3. Open the terminal (ctrl+alt+t) and run “sudo a2ensite dev.zendtest.com
Step 4. Open etc/hosts file ( ensure it’s hosts and not host file)

Step 5. Paste 127.0.0.1 dev.zendtest.com

Step 6. Restart apache by running “sudo /etc/init.d/apache2 start

That’s it, when you now open http://dev.zendtest.com you will get zend welcome page

Notes :

1. In-case you are not able to save site config inside sites-available, try to save the file as root

2. Clear browser chache, if dev.zendtest.com is not giving results

%d bloggers like this: