13 Nov 2017

Django Internal: create()

A model instance is corresponds to a record in the database once saved. In this post i am going to
explore what happens behind the scene when you call  create(). It create a instance of model and save that instance which creates a record in the database.

let say we have a model defined as:

class Author(models.Model):
   name = models.CharField(max_length=100)
   age = models.IntegerField()


Lets go step by step to deconstruct Author.objects.create()

Creating instance of model

 instance = Author(name="roy", age=47)

1. signals invoked
Once starts it send signal pre_init and on successful completion of this method post_init is called.

2. check for given model fields not values.
check can fail if you more arguments than number of fields or attribute which does not present on model.
for example:
instance = Author(name="roy", age=47, code='python')
TypeError: 'code' is an invalid keyword argument for this function.
Each model has attribute called ._meta through which gives related details for models



As for information the models.Model  has an abstract base class called ModelBase which add attributes on a model itself like ._meta, errors (DoesNotExist, MultipleObjectsReturned) and validating model itself for its concrete definition.


Saving model instance
model method ._save_base() is called by save() to save model instance.  this where pre_save() and post_save() signals are invoked.

There are couple of important actors involved in here, I would like brief them.
 1. InsertQuery: instance of class is initiated for actual SQL' s raw query by passing model fields objects and values.
2. SQLInsertCompiler: This is SQL compiler for InsertQuery. It is a place where django models fields and values are parsed into SQL raw statement.
3. DatabaseWrapper: It is an wrapper for
mysql-python client. It creates connection to SQL server and uses cursor to execute the raw SQL statement.

Queryset private method _insert() is place where things kick off at low level.  creating a record means INSERT SQL query. This happens inside a atomic transaction.
Basically django creates insert query gives that to compiler and compiler invoke database's wrapper method execute() by passing raw SQL statement as

     cursor.execute('INSERT INTO `author` (`name`, `age`) VALUES (%s, %s)', ['roy', 46])

Once above code is executed successfully record is being is created in database. And thereafter post_save signal is invoked. And that s how an model instance creates an record in database. 

Hope you found it interesting, keep reading.

14 Oct 2017

Django Manager's methods

In this post i am going to explore how django.db.models.manager.Manager class supports queryset methods.

>> Book.objects.all()           # returns a queryset.
>> Book.object.filter(author='george-orwell' )        # returns all books by author george-orwell.
>> Book.objects.delete()    # deletes all instances.

 Here Book.objects is an instance of Manager class django.db.models.manager.Manager.
Ok, which means all these methods should be defined for this class.  No, all these methods are not defined for manager class but for Queryset class.

Manager class inherits from  django.db.models.manager.BaseManagerFromQuerySet class.
And magic is there is no such class as BaseManagerFromQuerySet.
That class  is dynamically generated by method from_queryset of class BaseManager.
>> BaseManager.from_queryset(QuerySet)

This is where all magic happens.
What it does is it takes all the methods of QuerySet and dynamically attach all these methods to a newly created class BaseManagerFromQuerySet. And this class is used as a base class for Manager class.

And that' s how filter(), delete() etc. can be invoked on manager.



Hope you guys find it useful.

Reference:
manager.py

10 Oct 2017

python: new-style and old-style class

A new-style class inherits from object (or type) where as old-style class does not. In python 3.0 old-style class is removed.

In this post I am going to explore differences between the two.

1: types













Here we can seen that type of obj is not of class but of built-in type instance.  Attribute __class__ of obj gives information about class to which it belongs.
So type and __class__ attributes differ for old-style class.

Lets see what happens with New-style class (python 3.6): 















type and __class__ attributes does not differ for new-style class.


2: MRO (Method Resolution Order)

In addition to above differences, old-style class does not support mro.












whereas new-style class does have that:






Also one can see that new-style class inherits from a object.


3: super
super() does not works with old-style classes.

Works with new-style















the reason being is that super() expect type object for class In case of old-style type(ABC) is classobj not type But in case of new-style type(NewABC) is type.

4: descriptor protocol
New-style classes have implemented descriptor protocol whereas old-style does not.


5: raising an exception
In new style objects cannot be raised as exception unless derived from Exception.










Thanks for reading.

2 Oct 2017

repr and str in python

The purpose of repr() and str() is to display information about object. But there is slight distinction between the two. In this post we will explore that.

By definition repr() method is official representation of object. That means if i want to create an object repr should be descriptive enough to give that information.
Lets take an example of datetime object.

>> from datetime import datetime
>> now_datetime = datetime.now()
>> repr(now_datetime)

prints

>> 'datetime.datetime(2017, 10, 2, 11, 0, 7, 382330)'

so if a want to create a new datetime object repr should be referred for that matter.

new_datetime = datetime.datetime(2017, 11, 12, 15, 56, 9, 343831)

Thats exactly the purpose of repr.

As per doc str() is a 'informal' representation of object. It has be string and you can put whatever you feel like.

>> str(now_datetime)
>> '2017-10-02 11:00:07.382330'


Now the difference
str() it should only return valid string object whereas repr() should either return string or a python expression. 
In case of python expression which can be used to create object.


>> make_this_repr = "x+1"
>> repr(make_this_repr)
>> "'x+1'"
>> x =10
>> eval(make_this_repr)
>> 11


 
Hope you find it useful, thanks for reading.

13 Apr 2017

Full-Text Search in MySQL

In this post i am going to explore basic of this feature. Following is the table i am going to use for FULL-TEXT search explanation.

SELECT * FROM articles;
+—-+———————–+——————————————+
| id | title | body |
+—-+———————–+——————————————+
| 1 | MySQL Tutorial | DBMS stands for DataBase … |
| 2 | How To Use MySQL Well | After you went through a … |
| 3 | Optimizing MySQL | In this tutorial we will show … |
| 4 | 1001 MySQL Tricks | 1. Never run mysqld as root. 2. … |
| 5 | MySQL vs. YourSQL | In the following database comparison … |
| 6 | MySQL Security | When configured properly, MySQL … |
| 7 | Database Theory | I am going to teach you database theory |
+—-+———————–+——————————————+

Whle creating table i have used FULLTEXT (title,body) to allow full text search on these columns. Behind the scene it creates a index on these columns.

Lets do full-text search for string ‘database’.

SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('database' IN NATURAL LANGUAGE MODE);
+—-+——————-+——————————————+
| id | title | body |
+—-+——————-+——————————————+
| 7 | Database Theory | I am going to teach you database theory |
| 1 | MySQL Tutorial | DBMS stands for DataBase … |
| 5 | MySQL vs. YourSQL | In the following database comparison … |
+—-+——————-+——————————————+

FULL TEXT allows different mode to run search query, here it is IN NATURAL LANGUAGE MODE.

  • It means search string is a natural human language.

  • By default full-text search is case-insensitive.

  • Minimum length of word to be found is 3, which can be manipulated with innodb_ft_min_token_size .
  • The returned rows are sorted with highest relevance first. relevance is kind of score is calculated by MYSQL internally.

Above query search search string in title and description. Now let search in title only.
SELECT * FROM articles WHERE MATCH (title) AGAINST ('database' IN NATURAL LANGUAGE MODE);
since we have defined the full-text search on title and body, To search only in column body we need to define FULL-TEXT index on that column.

Lets now do full-text search for string ‘to’.

SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('to' IN NATURAL LANGUAGE MODE);

No result, but ‘to’ sting exist in body with id=7 ?
This is because full-text search will skip certain words, which are refereed as stop-words.
You can find out list for stop-words like:
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_DEFAULT_STOPWORD;
+——-+
| value |
+——-+
| a |
| about |
| an |
| are |
| as |
| at |
| be |
| by |
| com |
| de |
| en |
| for |
| from |
| how |
| i |
| in |
| is |
| it |
| la |
| of |
| on |
| or |
| that |
| the |
| this |
| to |
| was |
| what |
| when |
| where |
| who |
| will |
| with |
| und |
| the |
| www |
+——-+

What if you want specific case of search, like get all result having string ‘database’ and not having ‘MySQL’. . That is where BOOLEAN mode of full-text search helps.

SELECT * FROM articles WHERE MATCH (title,body) AGAINST ('+database -MySQL' IN BOOLEAN MODE);
+—-+—————–+—————————————–+
| id | title | body |
+—-+—————–+—————————————–+
| 7 | Database Theory | I am going to teach you database theory |
+—-+—————–+—————————————–+

more boolean modes:

  • find all rows which have atleast one of ‘MySQL’ and ‘YourSQL’
    SELECT * FROM articles WHERE MATCH (title,body) AGAINST (‘MySQL YourSQL’ IN BOOLEAN MODE);
    +—-+———————–+——————————————+
    | id | title | body |
    +—-+———————–+——————————————+
    | 5 | MySQL vs. YourSQL | In the following database comparison … |
    | 6 | MySQL Security | When configured properly, MySQL … |
    | 1 | MySQL Tutorial | DBMS stands for DataBase … |
    | 2 | How To Use MySQL Well | After you went through a … |
    | 3 | Optimizing MySQL | In this tutorial we will show … |
    | 4 | 1001 MySQL Tricks | 1. Never run mysqld as root. 2. … |
    +—-+———————–+——————————————+

  • find rows that contains both words
    SELECT * FROM articles WHERE MATCH (title,body) AGAINST (‘+MySQL +YourSQL’ IN BOOLEAN MODE);
    +—-+——————-+——————————————+
    | id | title | body |
    +—-+——————-+——————————————+
    | 5 | MySQL vs. YourSQL | In the following database comparison … |
    +—-+——————-+——————————————+

Similar kind of search can be made using Boolean mode.

So this was very basic of full-text search, hope you find it helpful.