hello world in open source

More recently i have contributed to an open source project. There is a little story behind that.

Contributing accidentally
One late night i was going through the one of the article , quite interesting one. The article explain how there are social platform bots behave as human and what are the cons of such. The article was informative , i learned about the possible use case of machine learning.

Once done with article, my curiosity led me to find if there is any library or something which can determine if post by an social account is a bot or not. Being Back-end developer you think that way, can't help it.

I found this botornot-python library. I installed the library and tried it out. And then i went through the source code of that, i found one little stuff which can be done in better way. Next step was to fix that and open a pull request keeping finger crossed. :)
The owner of the library gladly accepted contribution, thanks to him.

And it was my first open source contribution, hurray !!!.  I felt so exited and satisfied since from long time i have been looking for that moment.

Motivation
Its about contributing to make project better with best of my ability and to learn from open source community.

Keep it going
As they say best place to look for project to contribute are the ones which you are already using.
faker is a interesting library that i have been using from long time for writing unit tests. After about a week or so i contributed to the same project,  joy was more this time.

So this how i started my open source contribution, thanks for reading.

Fun with Python

Today is April Fool and on python mailing list following kept me laughing.

 

Why the version 8? It’s just to be greater than Perl 6 and PHP 7, but 

it’s also a mnemonic for PEP 8. By the way, each minor release will now 

multiply the version by 2. With Python 8 released in 2016 and one 

release every two years, we will beat Firefox 44 in 2022 (Python 64) and 

Windows 2003 in 2032 (Python 2048).

and this too

 

DON’T PANIC! You are still able to import your legacy code into 

Python 8, you just have to rename all your modules to add a “_noqa” suffix 

to the filename. For example, rename utils.py to utils_noqa.py. A side 

effect is that you have to update all imports. For example, replace 

“import django” with “import django_noqa”. After a study of the PSF, 

it’s a best option to split again the Python community and make sure 

that all users are angry.

Seriously we can have fun with python, read full thread

Optimizing Django ORM queries

I’m have been working on a project using Django and Django-REST-Framework. A particular endpoint of the application executing number of database queries happening . Once you know its happening , then need you need to figure out why is it happening.

Findings:

• select_related and prefetch_related
  Most of the time application models have reference to some other model referred by ForeignKey or ManyToManyor OnetoOne.
  When a query is made for base relation and when you use referred relation it will make extra query, this extra query can be avoided.
  using select_related and prefetch_related removed lot of unnecessary queries. A big boost.

• Structure queries well

for person in Person.objects.all():
    if person.create_date >=today_date and person.status not in [X, Y]:
    # do something with person

Above loop can be avoided by constructing the simple query.

for person in Person.objects.filter(created_date__gte=today_date).exclude(status__in=[X, Y]):
    # do something with person

• Beware of the response data (django-rest-framework)
  A serializer may have all the models fields defined, but for specific endpoint you may not need all the fields to be serialized especially the related fields or the model properties which results in database queries. You can write some custom serializers.

Above three gave quite a boost to database performance, to one API end-point i managed to get the number of queries from 469 to 91.

Apart from this stuff one go for database level optimization. In my case it is PostgreSQL database and once can consider to set the following database settings:

shared_buffers          25% of RAM
effective_cache_size    75% of RAM
work_mem                5MB
wal_buffers             16MB
checkpoint_segments     10
maintenance_work_mem    50MB for each GB of RAM

Hope you find this article helpful.

Generic function in python

There times in python’s land when you needs to have different kind of behaviour for a function depending upon the type of arguments.

Decorator singledispatch from functools module from python 3.4 can help us out to create a different definitions of a with common interface.

we can define a generic definition of a function and can call same function with different set of arguments type:

fun("python land") 
fun(23, True)
fun([11, 22, 33])
fun(object())

This is how a generic function will look:

from functools import singledispatch
@singledispatch
def fun(arg, verbose=False):
    if verbose:
        print("verbosing....")
    print(arg)

Now different behaviour can be defined for this generic function

@fun.register(int)
def fun_int(arg, verbose=False):
    if verbose:
        print("numbers: ", end=" ")
    print(arg)

@fun.register(list)
def fun_list(arg, verbose=False):
    if verbose:
        print("Enumerate this:")
    print([ele+1 for ele in arg])

@fun.register(object)
def fun_int(arg):
    print(arg.__class__)

source code can be found here

Hope you liked the article, keep reading.

hello world in Clojure

I have been looking to find some spare time to get started with Clojure programming for quite a whicle, intentionally made time for that today.

Idea is to understand learn basics of the language and write lot of code, try out sample problems, learn in-built libraries, how to use third party libraries, etc.

Apart from that try to learn language design philosophy, kind of ecosystem it has etc.

Here is github-repo link:
https://github.com/navyad/clojure-scripts

leiningen for Clojure

It has been a while since i have been itching to learn Clojure and have been using Python for around 3+ years. Learning new programming language is always a fun for a programmer. Obviously you need to find some spare time, specially on the job. 
But once you get started with Hello world !, your curiosity will find the time for you. 

While finding out what is the standard way to get started with a Clojure, i found that, Clojure community uses lein as a De-facto tool to build Clojure project. Same can be used as REPL (read-eval-print loop) as well.

After Downloading and installing lein,  created the project with name clojure-noob.

>> lein new app clojure-noob

which results in following directory structure.

>> lein run
Hello World !

(This been displayed from the core.clj)

I liked the tag line for lein project:            Automate Clojure projects without setting your hair on fire. 

  Happy learning :).