Django Import Cheat-sheet

In large full fledged Django web project consist the several applications and each applications having the view which are using various django's methods or classes. I  have noticed that some imports for methods or classes are used in almost every view. 

To keep them in sight i'm listing it out here : 

from django.http import HttpResponse, HttpResponseRedirect, Http404
from django.shortcuts import render_to_response
from django.template import RequestContext
from django.contrib.auth.decorators import login_required, permission_required
from django.contrib.admin.views.decorators import staff_member_required
from django.contrib.auth.decorators import login_required
from django.core.exceptions import ObjectDoesNotExist, ViewDoesNotExist

from django.utils import simplejson
from django.template.defaultfilters import slugify
from django.template.loader import render_to_string
from django.core.urlresolvers import reverse
from django.db.models import Q


from django.core.exceptions import PermissionDenied
from django.utils.translation import ugettext as _

from django.forms.models import modelformset_factory

Stacktrace of Django's Request and Response

In this post I'll going to explore what happens behind the scenes when we make a request for a resource (normally a template) and how Django sends the response.
All of the action happens in the django/core/handlers/base.py file.

1. On request server make server instance which is django.core.handlers.BaseHandler object. The server can be mod_python or wsgi type.

2. Import project's setting.py file and Django custom exception classes.

3. loads middleware classes by calling the load_middleware() of BaseHandler. 
    (ImproperlyConfigured exception is thrown from  here)

4. server instance dispatches the request_signal and create the instance of HttpRequest. this request object may be of mod_python [ django.core.handlers.wsgi.ModPythonRequest ]  or wsgi [ django.core.handlers.wsgi.WSGIRequest] depending upon the server type.

5. get_reponse(request) of BaseHandler is called. Its main purpose is to return HttpResponse object. This is most important method to for a request to get successful response or otherwise. 
I have divided the working of get_reposne(request) in following two phases:

phase 1: request
a). Execute the process_request(reqest) of middleware by passing the request object.
b). If process_request() returns the None then resolves the url by calling resolve() on    
     django.core.urlresolvers. 
c). If url resolution is successful then  calls process_view( request, view_fuc, url_arg )
     If url is not successful then returns the django.http.Http404.

process_view( request, view_fuc, url_arg ) either returns None or HttpResponse object.
 If it returns None then Django will continue processing the request and will call appropriate view. 
If it returns the HttpResponse object then django will not call view. 


phase 2: response
When view() is called and it should either response or exception.
If response is  response_object :
a).process_template_response(request, reponse ) is called by view() if response instance has render() associated with it, response parameter is of type TemplateResponse.
It is where you get the required response (html). 
 b).process_template_response(request, reponse ) is called by view() if response instance has render() associated with it, response parameter is of type HttpResponse or StramingHttpResponse.
If response is  None: 
c). process_excpetion(request, exception) is called which throws ValueError like:

raise ValueError("The view %s.%s didn't return an HttpResponse object." % (callback.__module__, view_name))

So for every request(url) either you will get response (e.g. html) or exception. 

No JSON object could be decoded: line 1 column 0 (char 0)

I have written a python code which makes the HTTP requests to fetch the json data.
The json data seems like :
{    
  "items": [

          {
            "question_id": 18384375,
            "answer_id": 18388044,
            "creation_date": 1377195687,
            "last_activity_date": 1377195687,
            "score": 0,
            "is_accepted": false,
            "owner": {
                "user_id": 1745001,
                "display_name": "Ed Morton",
                "reputation": 10453,
                "user_type": "registered",
                "link": "http://stackoverflow.com/users/1745001/ed-morton"
                 }
          },

      ]
}

Code : 

>>url="https://api.stackexchange.com/2.1/answers?order=desc&sort=activity&site=stackoverflow"

>>obj = urllib.urlopen(url)
>>response_str = obj.read()
>>response_json = simplejson.loads(response_str)

Error : 

The last line above shoots following  error.
JSONDecodeError at /
No JSON object could be decoded: line 1 column 0 (char 0)

when i tried to change the simplejson to json object for loads() then following  error occurred
ValueError at /
No JSON object could be decoded

I tried to check the json format using the Jsonlint which gave me error :

The error suggested that problem is of encoding type of the response . Then i inspects the response information like
    >>obj.info()

         Server: nginx
         Date: Fri, 23 Aug 2013 04:51:01 GMT
         Content-Type: application/json; charset=utf-8
         Connection: close
         Cache-Control: private
         Content-Encoding: gzip
         Access-Control-Allow-Origin: *
         Access-Control-Allow-Methods: GET, POST
         Access-Control-Allow-Credentials: false
         Content-Length: 2334

Solution:

The problems seems to be of content Content-Encoding of the response which is gzip format and simplejson or json  is failed to correctly parse that. After many searches on stackoverflow i found the following the solution(s) of problem.
Solution: better to use requests object.

1) using only requests

   import requests

   response_json = requests.get(url).json()

2)  using only requests with simplejson or json.

     import requests

     response = requests.get(url)

     response_json = simplejson.loads(response.text)

     print response_json

Above both the codes  returning the correct Content-Encoding for json.
>>reposnse_json.encoding
     utf-8
>>response_json.headers['content-type']
    "application/json; charset=utf8'

Selecting checkboxes in jquery

In my recent side project (clone of imdb.com), number of checkbox options were provided from which user can select the genre(s) of the movie and on the basis of that movies are filtered and shown.
Basically this is divided into two parts, first in which i'll get the all checked genre(s) and second part is to show movie(s) based on that genre(s).

In this post i'm wrting about how i accomplished first part which is divided into number of tasks.

In my template , following is code for showing genre(s) each having  checkbox option.

<div class="refine_box">
        {% for k, v in genre_dic.items  %}
            <input type="checkbox" class="check_id" name="genre" value="{{ k }}">{{ k }} ({{ v }})<br>
        {% endfor %}
</div>

genre_dic is a dictionary having key as a genre and value is number of movies for that genre.

task1: get the value of checkbox which is checked. 
 $(".refine_box input[type=checkbox]").click(function() { 
        
            var isChecked = $(this).is(':checked');
            if(isChecked){
                    alert(#(this).val())
            }
});


task2: get the values of all checkboxes which are checked.
 $(".refine_box input[type=checkbox]").click(function() { 
        
        var arr_refine = [];    //this will store all clicked checkboxes

        $(".check_id:checked").each(function() {
                arr_refine.push(this.value);

        });
 
  }); 

task3: passing array from ajax request to Django view.
In previous task i have created a array holding all genre, so in this task that array we will send to view.
      $.ajax({
                    url: "{% url ajax_refine_movieslist %}",
                    data: {arr_refine: arr_refine},
                   
            });    


Follwoing is whole code of jquery :
 $(".refine_box input[type=checkbox]").click(function() { 
        
        var arr_refine = [];

        $(".check_id:checked").each(function() {
                arr_refine.push(this.value);
               
        });

        $.ajax({
                    url: "{% url ajax_refine_movieslist %}",
                    data: {arr_refine: arr_refine},
                    dataType:'html',
                    success: function(newData){
                         $('.movie_listing').html(newData);
                    }
                   
            });       
   
    }); 
   
task4 : printing the list holding the selected genre(s)
def ajax_refine_movies(request):
    genre_list = request.GET.getlist('arr_refine[]')
    print genre_list


This is how i'm getting the all checkbox selected options.
Soon i'll post second part of this.

Code that i love and that i hate

A programming language is a  set of specification and rules. It is better to stick to those while you are coding.
Anyone having basic arithmetic ability and knowledge of for loop can code a Fibonacci series.  Since one can understand the problem and apply his/her logic to solve. But writing a good code and writing a code is two different things.
Writing a code is just like typing the problem logic and make it work. Whereas the writing a good code is to present it beautifully so it looks like a piece of art.
A "good code" has to be well structured, clean and understandable.

Good code

Using appropriate names for variables , functions, classes etc.
Comment as much as you can , for which is not obvious .
Divide whole problem into modules.
Proper indentation.
Logical organization of statements.
Appropriate error handling mechanism.
Prefer clarity to "efficiency" read as premature efficiency.
Declare where you use or declare all data at same place.
Choosing appropriate data structure for the problem.
Code as per the the programming language's style conventions or standards.

Code that i hate to work with.

Basically it will be a right opposite of "good code". To name a few
naming variables to anything rather than some meaningful name.
Consistently bad indentation.
No comments to important parts of code.
Code striving for premature efficiency.
Repeating same code which is better to be modularized.
Doing more that one thing in a module.




But in real work environment one has to work with other fellows who habitually wrote opposite of "good code". They tends to follow writing style rather than coding style.
And when you are sharing your code to such a guy and he is making changes to it .. which will diminish the beauty of your code, then you have to write this post and ask them to go through it.

Serving uploaded images in Django

I ran into trouble while serving the uploaded images from database in template. In this post i'll  share what mistakes i made and how i rectified those mistakes.

Problem was :
In template the uploaded image was not showing like
{% for movi in all_movies %}
<div class="movie_frame">
<img src="{{MEDIA_URL}}{{ movi.poster}}" alt="xxx" height="140" width="110" />
</div>
{% endfor %}


For uploaded files one needs to set the MEDIA_ROOT and MEDIA_URL setting in settings.py.
setting.py
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
MEDIA_ROOT = os.path.join(SITE_ROOT, 'media')
MEDIA_URL = '/media/'

Note:
MEDIA_ROOT is a absolute physical location where the images are to be stored. This will create a 'media' folder under SITE_ROOT.
MEDIA_URL  is url reference which is to be used inside template to get the image.

Mistake 1: The value of MEDIA_URL is same as the MEDIA_ROOT value i.e. "media".

Correction : changed the MEDIA_URL value from "media" to "/site_media/". you can give any valid string but make sure it should not be same as of MEDIA_ROOT value.


models.py
from django.db import models
from imdbclone.settings import MEDIA_ROOT
class Movies(models.Model):
    title = models.CharField(max_length=50, blank=False)
    plot =  models.CharField(max_length=500, blank=True)
    actors = models.CharField(max_length=200, blank=True)
    poster = models.ImageField(upload_to=MEDIA_ROOT, blank=True)

here poster will save the image to MEDIA_ROOT which is already been there, that initially i thought.

Mistake 2: upload_to attribute of ImageField is MEDIA_ROOT , is wrong.

Correction : poster filed in database will have the absolute path to the image like /home/navyad/workspace/imdb/imdbclone/imdbclone/media/media/image.jpg, which is wrong.
I changed the upload_to attribute value from MEDIA_ROOT to "poi" (any valid string will work).
Note :
      a)  Corresponding to upload_to value a folder will be created, in this case "poi". This folder will be             created under MEDIA_ROOT. And this is where the image actually stored.
      b)  poster filed in database it will contain image reference like poi/image.jpg,



movies/urls.py :
from imdbclone import settings
from django.views.static import serve
urlpatterns = patterns('',

                           url(r'^site_media/(.*)$', serve, {'document_root': settings.MEDIA_ROOT}),  

)
In template when image is to be displayed using <img src ..> tag , it will make the request to get image. For that we have specified the url.

Mistake 3: placing url for image in wrong file.

Correction : I was using this for movies app and i placed the url for image in movies.urls , which supposed to be in the project main urls.py file alongside where setting.py live.

After correcting all the mistakes , and when image is loaded in template, i found this observation.
The <img src="" /> value  will be like
<img width="110" height="140" alt="xxx" src="/site_media/poi/image.jpg">
note 
src value : MEDIA_URL/upload_to/imagename


With these mistake i have learned how to serve the uploaded content using MEDIA_ROOT and MEDIA_URL.
At end i will share this quote :
If you're not making mistakes, then you're not doing anything. I'm positive that a doer makes mistakes.
John Wooden 

Error: That port is already in use

I was having a trouble to restart a Django server. At first time , i started a server like
>>python manage.py runserver 192.168.1.5:8000
no problem it started running fine.

I had to change the models so i exit the server using ctrl-Z and issue command to create the tables
>>python manage.py syncdb

Now when i tries to again start the server by issuing
>>python manage.py runserver 192.168.1.5:8000 again ,
it returns error message Error: That port is already in use.

This error means that port is already in use and to use same port , first we needs to kill process running on that port. Which involves following two steps :

1. Finding process for port 8000
  >>sudo netstat -lpn |grep :8000
this will return something like:
tcp        0      0 192.168.1.5:8000    0.0.0.0:*        LISTEN      10797/python   

here 10797 is a process id running on port 8000.

2. Kill the process
   >>kill -9 10797

Now
>>python manage.py runserver 192.168.1.5:8000 
would start the server again successfully.

Django path in your system

Django path in your system depends upon on how you have installed Django in your system.

If you have installed Django through pip installer then it should be like

'/usr/local/lib/python2.7/dist-packages/django/ 




If you have installed Django manually i.e. by downloading Django-X.Y.tar.gz file then

Django will be installed in you python installation directory under /site-packages/ directory

Finding Django path

There different ways you can find that out. 

the django path may vary depending upon the OS you are using,following examples are for ubuntu12.04.




>>> import django  

>>> print django.__file__  

'/usr/local/lib/python2.7/dist-packages/django/__init__.py'

or

>> import inspect  

>> import django  

>> print inspect.getabsfile(django)

'/usr/local/lib/python2.7/dist-packages/django/__init__.py'



or

>> import django

>>django

 or

>> import django

>>django.__path__

Reading http JSON response in Python

Make http request that returns JSON response.               
                   >>fileObj = urllib.urlopen("http://www.omdbapi.com/?t=Fight Club")


Get JSON response from fileObj
                    >>jsonData =  fileObj.read()

 if you print this jsonData , it will look like this.
{
    "Title":"Fight Club",
    "Year":"1999",
    "Rated":"R",
    "Released":"15 Oct 1999",
    "Runtime":"2 h 19 min",
    "Genre":"Drama",
    "Director":"David Fincher",
    "Writer":"Chuck Palahniuk, Jim Uhls",
    "Actors":"Brad Pitt, Edward Norton, Helena Bonham Carter, Meat Loaf",
    "Plot":"An insomniac office worker looking for a way to change his life crosses paths with a devil-may-care soap maker and they form an         underground fight club that evolves into something much, much more...",
    "Poster":"http://ia.media-imdb.com/images/M/MV5BMjIwNTYzMzE1M15BMl5BanBnXkFtZTcwOTE5Mzg3OA@@._V1_SX300.jpg",
    "imdbRating":"8.9",
    "imdbVotes":"727,988",
    "imdbID":"tt0137523",
    "Type":"movie",
    "Response":"True"
}


Now you have whole JSON response , what if you are only interested in some particular response field(s) ,say 'Director'  ?
Lets see how we can get particular field(s) of JSON response.

Convert json object into python dictionary
                  >>import simplejson
                  >>response_dict = simplejson.loads(jsonData)


Use json data from response_dict to get field
                >>response_dict['Director']
                     David Fincher


So thats how you can read JSON response ,  by the way that was damn good movie....!!.

Get IMDb movie data

make query http://www.omdbapi.com/?t=moviename
it will return JSON response (default)

example
http://www.omdbapi.com/?t=Fight Club  (in browser)
note : while coding space in movie title should be replaced with %20

JSON response:

{
    "Title":"Fight Club",
    "Year":"1999",
    "Rated":"R",
    "Released":"15 Oct 1999",
    "Runtime":"2 h 19 min",
    "Genre":"Drama",
    "Director":"David Fincher",
    "Writer":"Chuck Palahniuk, Jim Uhls",
    "Actors":"Brad Pitt, Edward Norton, Helena Bonham Carter, Meat Loaf",
    "Plot":"An insomniac office worker looking for a way to change his life crosses paths with a devil-may-care soap maker and they form an        underground fight club that evolves into something much, much more...",
    "Poster":"http://ia.media-imdb.com/images/M/MV5BMjIwNTYzMzE1M15BMl5BanBnXkFtZTcwOTE5Mzg3OA@@._V1_SX300.jpg",
    "imdbRating":"8.9",
    "imdbVotes":"727,988",
    "imdbID":"tt0137523",
    "Type":"movie",
    "Response":"True"
}

To get the XML response add &r=XML at end of request.

http://www.omdbapi.com/?t=Fight%20Club&r=XML

 

XML response:

<root response="True">
<movie title="Fight Club" year="1999" rated="R" released="15 Oct 1999" runtime="2 h 19 min" genre="Drama" director="David Fincher" writer="Chuck Palahniuk, Jim Uhls" actors="Brad Pitt, Edward Norton, Helena Bonham Carter, Meat Loaf" plot="An insomniac office worker looking for a way to change his life crosses paths with a devil-may-care soap maker and they form an underground fight club that evolves into something much, much more..." poster="http://ia.media-imdb.com/images/M/MV5BMjIwNTYzMzE1M15BMl5BanBnXkFtZTcwOTE5Mzg3OA@@._V1_SX300.jpg" imdbRating="8.9" imdbVotes="727,988" imdbID="tt0137523" type="movie"/>
</root>

Random number in JavaScript

Have you ever to use a random number in your application? Of course it depends on the application or program you are in to. Recently I was  working on a mobile application in which i had to rate the movies between 1 to 5 . Since the backend data was not ready i had to use static code in development phase. So the question was how to rate movies,  In other words  i had to assign a random number between 1 to 5 to a movie.

Lets first try Math.random()
Math.random() returns the a floating point number between 0 and 1. It could return anything like 0.9876567876 or 0.4232457568.
It would not work since i need real number between 1 to 5.


Variant of Math.random()
                           Math.random() * (max - min) + min
This would return a floating point random number between min (inclusive) and max (exclusive).
To rate 1 to 5 , i have to use 1 as a min  to 6 as a max. Not 5 as a max because it is exclusive
                    r =   Math.random() * (6 - 1) + 1
r can be 2.2.711616653897717 or 4.5043663586648925

Yes I want something like this , which would return a random number between two given numbers.
But the problem is it is returning a floating number.
Note that r is real.float. where real part would be always be a random number between min and max.
My problem would be solved if i could get only real part out of it.


Will Math.floor(x) help ?
It would returns the nearest largest integer of x.

So if i pass x as 2.2.711616653897717 i can get 2 (real part).
   e.g.   2 = Math.floor(2.2.711616653897717)


No, I need both random() and floor().
Step 1: get floating point random number
               float_random = Math.random() * (6-1) + 1
Step 2: round off floating number to get real integer
              integer_random = Math.floor(float_random)

integer_random would always be between 1 and 5.

Now i can use this integer_random number to rate the movies. Next time you looking for random number in JavaScript , you know the steps.

Mapping a key to many values in javascript

The concept of mapping arises when requires to bind a pair of values together like name:'john'. here name is a key and 'john' is value of key. Generally for mapping unique keys are used to identify particular value.
What if one wants to identify many values with same key ?

Real world example
One of my recent software project , it is required to access the weather feed for particular city. We are using yahoo developer api for that. First you have to get the id of a city , woied, you can can get this id for your city  by searching the http://weather.yahoo.com/.
Now use  this link http://weather.yahooapis.com/forecastrss?w=woeid   to get the data for the particular city. where woeid is the id of the city.

Problem
Lets get back to the mapping question, To describe the weather condition they have specified the description.of condition ( http://developer.yahoo.com/weather/#codes).
For example, All following four weather condition can be characterized as a 'rain'
              mixed rain and sleet
              mixed rain and snow
              drizzle rain
              heavy rain
So whenever either of four conditions are there , application requires to display as a 'rain'.
It was required to map 'rain '(key) to above four conditions (values)

Wrong solution
i created a dictionary having 'rain' as a key and a weather conditions.This wayi created four 'rain' key and map each of weather condition.


var dict = {     mixed rain and sleet,    
                       mixed rain and snow,   
                       drizzle rain,                  
                       heavy rain                    
                 }                                        

When i iterated over dict it show only last mapping.
                for(item in dict){                                   
                      Ti.API.info(item+"....."+dict[item])
                   }                                                          

Wrong Output:    
          rain......mixed rain and sleet.
But it supposed to show all four conditions, what happened? I realized that the dictionary key must be unique.what i was doing wrong? I was mapping multiple keys having same name which is being treated as a one single key.  So i have to map all four weather conditions against only one 'rain' key.



Right Solution
Created a dictionary, this time, with only one key and assigning all four weather conditions as an  array.
                 var dict = {                                                    
                                    'rain' : [   mixed rain and sleet  , 
                                                   mixed rain and snow,  
                                                   drizzle rain,                  
                                                   heavy rain                    
                                           ]                                            
                     }                                                                 
Iterating over dict
            for(item in dict){                                                                  
                        array_values = dict[item]                                         
                         for(ele in array_values){                                         
                               Ti.API.info(item+"......"+array_values[ele])     
                         }                                                                               
            }                                                                                            

Although above code is self explanatory, i'm going to explain anyhow. 'item' is a key which is 'rain'  and for dict['rain'] you get the array of values. Now you can  simply iterate over array, where 'ele' will give weather conditions.
  
Right Output:
         rain..... mixed rain and sleet,
         rain..... mixed rain and sleet,
         rain..... drizzle rain,
         rain..... heavy rain 

Now i can correctly use 'rain' in place of those four weather conditions.

Reverse in python

Reverse using reversed(seq) function
To reverse a sequence one can use the reversed(seq) built-in-function. what is sequence ? str, list and tuple are sequence. This function can take a sequence as a argument and returns an iterator. Lets dig deep into it.
A  reversed(seq) method will call the __reversed__() method of sequence which will reverse the elements of sequence and  and return a iterator for reverse sequence. So the reversed(seq) not just return a iterator but a reverse iterator.

 Feel from example:

>>list=[1,2,3,4,5]

>>reverse_iterator =  reversed(list)

>>for ele in reverse_iterator:

>>....    print ele                         // 5,4,3,2,1

We can also use the next() method of reverse_iterator object to call 'successive' elements.
>>reverse_iterator.next()         //5
>>reverse_iterator.next()         //4
>>reverse_iterator.next()         //3
>>reverse_iterator.next()         //2
>>reverse_iterator.next()         //1
>>reverse_iterator.next()         //StopIteration exception, since no more elements are left


Reverse using extended slice, [start:end:step]
Extended slice is another way to reverse str, list and tuple. First, lets get familiar with what extended slice is all about.  
Sometimes it may be desirable to iterate over only certain elements of a type. for example,
L = [11 22, 33, 44, 55, 66, 77, 88, 99, 100] , it may be required that one wants to iterate over only even or odd numbers thereby skipping certain elements from the list.
>>L = [11 22, 33, 44, 55, 66, 77, 88, 99, 100]
>>L[1:10:2]       -----> 2,4,6,8,10     // all even numbers of L
>>L[0:10:2]       ------> 1,3,5,7,9      // all odd numbers of L

Now let answer,  how ?
Syntax for extended slice is [start:end:step]. start specifies the index (of first element) from which it will start picking the elements, end specifies upto which index (of last element) it will pick elements and step specifies the number of elements to skip.

The default value of start is 0, of end is length of item in used ( length of L) and of step is 1.
So if you try like L[::] it will print the L as it is, assuming it as L[0:10:1]
Note that the step cannot be 0  because you have to take step to iterate (i made that up ). Therefore following will give error.
>>> L[0:10:0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: slice step cannot be zero



Here is important thing, the step part also defines the order in which an item is iterated. if you provide a negative value then it start iterating in reverse order.
>>> L[::-1]
[100, 99, 88, 77, 66, 55, 44, 33, 22, 11]
what L[::-1] means ?
It means that Iterate over L from 0 to length of L in reverse order.

So that all about reverse in python, i hope you find it useful.