Vivek on Software

Amazing functools features in Python

Amazing functools features in Python

Vivek Kumar Singh's photo
Vivek Kumar Singh
·

5 min read

Table of contents

I was recently reading Django’s Source Code, and I came across the @wraps decorator, which led me to the functools docs, where I discovered some fantastic functools features. That discovery led to the creation of this article. This tutorial will teach you how to use some fantastic functools methods to make your life simpler.

What is functools?

functools is a Python built-in module that contains Higher Order functions that can interact with other functions. A complete functools documentation may be found here. Let’s see some decorators in action.

lru_cache

When invoking a function with the same arguments, this decorator in the functools module saves n number of function calls in cache, which saves a lot of time. Assume for the sake of demonstration that we have a very large function that takes a long time to execute. The function a_heavy_operation() takes 3 seconds to execute in this example.

import time

start = time.time()
def a_heavy_operation():
    time.sleep(3)
    return 11 + 22


print(a_heavy_operation())
print(a_heavy_operation())

print(time.time() - start)

# Output
# 33
# 33
# 6.024240255355835

It takes about 6 seconds to run the above code. To the above function, we’ll add lru cache.

import time
from functools import lru_cache

start = time.time()


@lru_cache()
def a_heavy_operation():
    time.sleep(3)
    return 11 + 22


print(a_heavy_operation())
print(a_heavy_operation())

print(time.time() - start)

# Output
# 33
# 33
# 3.0158064365386963

Take a look at how using lru_cache made our code run faster. Python saved the function’s cache and retrieved the cached value, reducing our execution time.

wraps

Wraps is used in functools to keep the function details. When we decorate a function, the function’s information is gone. We utilise the @wraps decorator on the decorator wrapper function to prevent this. Take a look at this code to see what I mean.

def my_decorator(func):
    def log(*args, **kwargs):
        print("Running ")
        return func(*args, *kwargs)

    return log

@my_decorator
def add(a, b):
    """my beautiful doc"""
    return a + b

Run the above code in -i mode using, python -i file.py Let's see what we have:

>>> add(1,2)
Running 
3
>>> add(3,4)
Running 
7
>>> add.__name__
log
>>> add.__doc__
>>>

We can see that our decorator is operating properly in the previous example, since it is consistently “Running” on each run. However, our function’s information has been lost, and it is unable to return the name or the docstring. We have @wraps to help us with this problem. Make the changes below to the code.

from functools import wraps

def my_decorator(func):
    @wraps(func)
    def log(*args, **kwargs):
        print("Running ")
        return func(*args, *kwargs)

    return log


@my_decorator
def add(a, b):
    """my beautiful doc"""
    return a + b

Now again run the code using python -i file.py

>>> add(1,2) 
Running 
3       
>>> add.__name__
'add'
>>> add.__doc__
'my beautiful doc'
>>>

Voila! The function information is now saved in our function.

singledispatch

To create a generic function, singledispatch is utilised. Generic functions are those that perform the same operation on a variety of data types. Assume I want to create a function that returns the first value from an iterable of several data types.

def return_first_element(data):
    if isinstance(data, list):
        print(data[0])
    elif isinstance(data, str):
        print(data.split()[0])
    elif isinstance(data, dict):
        print(list(data.values())[0] )
    else:
        print(print(data))

Now run python -i file.py to run the code in interactive mode.

>>> return_first_element({"Age":20, "Height": 180})
20
>>> return_first_element("Hello Mr Python")
Hello
>>> return_first_element([12,432,563])      
12
>>>

Our function is effective, but it isn’t clean. Using if/elif/else statements to create generic functions is not recommended in Python. So, what’s the solution? singledispatch, of course. Let’s make a few modifications to our code.

from functools import singledispatch

@singledispatch
def return_first_el(data):
    return data


@return_first_el.register(list)
def _(data):
    return data[0]


@return_first_el.register(dict)
def _(data):
    return list(data.values())[0]


@return_first_el.register(str)
def _(data):
    return data.split()[0]

To check the results, run the code again in interactive mode with python -i file.py.

>>> return_first_el({"Age":20, "Height": 180}) 
20
>>> return_first_el("Hello Mr Python")             
'Hello'
>>> return_first_el([124, 765, 897])   
124
>>> return_first_el({12,31,1})       
{1, 12, 31}

Look how our return_first_el function acted as a fallback function when no data type matched for ‘set’.

Look at how much cleaner our code is now; the singledispatch made it easier to add more data types, and each datatype now gets its own place where we can perform further operations on the data.

total_ordering

The total_ordering decorator saves a ton of time in Object Oriented Progrmming. Consider this example, the below class declares a class Man with name and age property and (=) __eq__ and (<) __lt__ dunder methods.

class Man:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def __eq__(self, o):
        return self.age == o.age

    def __lt__(self, o):
        return self.age < o.age

Let’s see what happens if we run the code.

>>> obj = Man("Vivek", 20)
>>> obj2 = Man("Alex", 24) 
>>> obj = obj
>>> obj == obj2
False
>>> obj < obj2
True
>>> obj >= obj2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>=' not supported between instances of 'Man' and 'Man'

Our code worked for (==) and (<), but it didn’t work when we used an operator that wasn’t defined in the class. Given that we create at least one operator dunder method and __eq__ method, @total_ordering generates the,>,=,>=, and more comparison operators for our class. Let’s add our decorator just above the class.

from functools import total_ordering

@total_ordering
class Man:
.....
.....

Now again run thee code in interactive mode to see the results

>>> o = Man("Vivek", 20) 
>>> b = Man("Alex", 24) 
>>> o == b
False
>>> o >= b  
False
>>> o <= b
True

Take a look at how total ordering generated our class’s comparison operators.

Conclusion

I hope you found this post useful; I strongly advise you to study the documentation in order to fully comprehend the internal mechanics of these higher level functions. If you enjoyed this, please consider following me on Twitter , where I share stuff related to Python, Web development, and open source software. I’ll see you there.

Pythonpython beginnerPython 3python projectsFunctional Programming