Function annotations are a Python 3 feature that lets you add arbitrary metadata to function arguments and return value. They were part of the original Python 3.0 spec.
In this tutorial I’ll show you how to take advantage of general-purpose function annotations and combine them with decorators. You will also learn about the pros and cons of function annotations, when it is appropriate to use them, and when it is best to use other mechanisms like docstrings and plain decorators.
Function Annotations
Function annotations are specified in PEP-3107. The main motivation was to provide a standard way to associate metadata to function arguments and return value. A lot of community members found novel use cases, but used different methods such as custom decorators, custom docstring formats and adding custom attributes to the function object.
It’s important to understand that Python doesn’t bless the annotations with any semantics. It purely provides a nice syntactic support for associating metadata as well as an easy way to access it. Also, annotations are totally optional.
Let’s take a look at an example. Here is a function foo() that takes three arguments called a, b and c and prints their sum. Note that foo() returns nothing. The first argument a is not annotated. The second argument b is annotated with the string ‘annotating b’, and the third argument c is annotated with type int. The return value is annotated with the type float. Note the “->” syntax for annotating the return value.
def foo(a, b: 'annotating b', c: int) -> float:
print(a + b + c)
The annotations have no impact whatsoever on the execution of the function. Let’s call foo() twice: once with int arguments and once with string arguments. In both cases, foo() does the right thing, and the annotations are simply ignored.
foo('Hello', ', ', 'World!')
Hello, World!
foo(1, 2, 3)
6
Default Arguments
Default arguments are specified after the annotation:
def foo(x: 'an argument that defaults to 5' = 5):
print(x)
foo(7)
7
foo()
5
Accessing Function Annotations
The function object has an attribute called ‘annotations’. It is a mapping that maps each argument name to its annotation. The return value annotation is mapped to the key ‘return’, which can’t conflict with any argument name because ‘return’ is a reserved word that can’t serve as an argument name. Note that it is possible to pass a keyword argument named return to a function:
def bar(*args, **kwargs: 'the keyword arguments dict'):
print(kwargs['return'])
d = {'return': 4}
bar(**d)
4
Let’s go back to our first example and check its annotations:
def foo(a, b: 'annotating b', c: int) -> float:
print(a + b + c)
print(foo.__annotations__)
{'c': <class 'int'>, 'b': 'annotating b', 'return': <class 'float'>}
This is pretty straightforward. If you annotate a function with an arguments array and/or keyword arguments array, then obviously you can’t annotate individual arguments.
def foo(*args: 'list of unnamed arguments', **kwargs: 'dict of named arguments'):
print(args, kwargs)
print(foo.__annotations__)
{'args': 'list of unnamed arguments', 'kwargs': 'dict of named arguments'}
If you read the section about accessing function annotations in PEP-3107, it says that you access them through the ‘func_annotations’ attribute of the function object. This is out of date as of Python 3.2. Don’t be confused. It’s simply the ‘annotations’ attribute.
What Can You Do With Annotations?
This is the big question. Annotations have no standard meaning or semantics. There are several categories of generic uses. You can use them as better documentation and move argument and return value documentation out of the docstring. For example, this function:
def div(a, b):
"""Divide a by b
args:
a - the dividend
b - the divisor (must be different than 0)
return:
the result of dividing a by b
"""
return a / b
Can be converted to:
def div(a: 'the dividend',
b: 'the divisor (must be different than 0)') -> 'the result of dividing a by b':
"""Divide a by b"""
return a / b
While the same information is captured, there are several benefits to the annotations version:
- If you rename an argument, the documentation docstring version may be out of date.
- It is easier to see if an argument is not documented.
- There is no need to come up with a special format of argument documentation inside the docstring to be parsed by tools. The annotations attribute provides a direct, standard mechanism of access.
Another usage that we will talk about later is optional typing. Python is dynamically typed, which means you can pass any object as an argument of a function. But often functions will require arguments to be of a specific type. With annotations you can specify the type right next to the argument in a very natural way.
Remember that just specifying the type will not enforce it, and additional work (a lot of work) will be needed. Still, even just specifying the type can make the intent more readable than specifying the type in the docstring, and it can help users understand how to call the function.
Yet another benefit of annotations over docstring is that you can attach different types of metadata as tuples or dicts. Again, you can do that with docstring too, but it will be text-based and will require special parsing.
Finally, you can attach a lot of metadata that will be used by special external tools or at runtime via decorators. I’ll explore this option in the next section.
Multiple Annotations
Suppose you want to annotate an argument with both its type and a help string. This is very easy with annotations. You can simply annotate the argument with a dict that has two keys: ‘type’ and ‘help’.
def div(a: dict(type=float, help='the dividend'),
b: dict(type=float, help='the divisor (must be different than 0)')
) -> dict(type=float, help='the result of dividing a by b'):
"""Divide a by b"""
return a / b
print(div.__annotations__)
{'a': {'help': 'the dividend', 'type': float},
'b': {'help': 'the divisor (must be different than 0)', 'type': float},
'return': {'help': 'the result of dividing a by b', 'type': float}}
Combining Python Annotations and Decorators
Annotations and decorators go hand in hand. For a good introduction to Python decorators, check out my two tutorials: Deep Dive Into Python Decorators and Write Your Own Python Decorators.
First, annotations can be fully implemented as decorators. You can just define an @annotate decorator and have it take an argument name and a Python expression as arguments and then store them in the target function’s annotations attribute. This can be done for Python 2 as well.
However, the real power of decorators is that they can act on the annotations. This requires coordination, of course, about the semantics of annotations.
Let’s look at an example. Suppose we want to verify that arguments are in a certain range. The annotation will be a tuple with the minimum and maximum value for each argument. Then we need a decorator that will check the annotation of each keyword argument, verify that the value is within the range, and raise an exception otherwise. Let’s start with the decorator:
def check_range(f):
def decorated(*args, **kwargs):
for name, range in f.__annotations__.items():
min_value, max_value = range
if not (min_value <= kwargs[name] <= max_value):
msg = 'argument {} is out of range [{} - {}]'
raise ValueError(msg.format(name, min_value, max_value))
return f(*args, **kwargs)
return decorated
Now, let’s define our function and decorate it with the @check_range decorators.
@check_range
def foo(a: (0, 8), b: (5, 9), c: (10, 20)):
return a * b - c
Let’s call foo() with different arguments and see what happens. When all arguments are within their range, there is no problem.
foo(a=4, b=6, c=15) 9
But if we set c to 100 (outside of the (10, 20) range) then an exception is raised:
foo(a=4, b=6, c=100) ValueError: argument c is out of range [10 - 20]
When Should You Use Decorators Instead of Annotations?
There are several situations where decorators are better than annotations for attaching metadata.
One obvious case is if your code needs to be compatible with Python 2.
Another case is if you have a lot of a metadata. As you saw earlier, while it’s possible to attach any amount of metadata by using dicts as annotations, it is pretty cumbersome and actually hurts readability.
Finally, if the metadata is supposed to be operated on by a specific decorator, it may be better to associate the metadata as arguments for the decorator itself.
Dynamic Annotations
Annotations are just a dict attribute of a function.
type(foo.__annotations__) dict
This means you can modify them on the fly while the program is running. What are some use cases? Suppose you want to find out if a default value of an argument is ever used. Whenever the function is called with the default value, you can increment the value of an annotation. Or maybe you want to sum up all the return values. The dynamic aspect can be done inside the function itself or by a decorator.
def add(a, b) -> 0:
result = a + b
add.__annotations__['return'] += result
return result
print(add.__annotations__['return'])
0
add(3, 4)
7
print(add.__annotations__['return'])
7
add(5, 5)
10
print(add.__annotations__['return'])
17
Conclusion
Function annotations are versatile and exciting. They have the potential to usher in a new era of introspective tools that help developers master more and more complex systems. They also offer the more advanced developer a standard and readable way to associate metadata directly with arguments and return value in order to create custom tools and interact with decorators. But it takes some work to benefit from them and utilize their potential.
Comments