Wednesday, January 07, 2015

Custom Python Decorators Considered Harmful

In line with the classic post about GOTO being harmful, I must extend this to include the 'Decorators' language feature of Python.

"Just because you can do a thing doesn't mean you should do a thing."

Decorators, IMHO, as a general rule, are Evil in the classical sense: they tempt you with easy riches, but there are complications and ambiguities down the road that make life difficult.

I recently read a blog post about someone advocating for creating a @retry decorator that would automatically retry a function a certain number of times until it returned success.

As decorated, the method would fetch a URL a max number of times or until it got a certain HTTP success code. I've seen such methods used before at various places, but have found them to be VERY troublesome.

Test driven (TDD) requires we create a method that can be tested through all its code paths, or at least most of them. Decorators make writing tests more complicated as well.

So, here's my reasoning:

  • The code without a decorator, retrying n times is very obvious in function.
  • Presumably one would want to use a decorator so the same decorator could be used on multiple functions, yet the undecorated version can be parameterized easily, also.
  • Modifying this code will happen an industry average of 6 more times;
  • Each time it's modified, the person reading it will have to understand not just loops, but decorators and how they can go wrong;
  • Some of the people who come after you, who modify this code, will do it incorrectly and this will result in more time spent;
  • Writing unit tests for decorated code is usually far more difficult, since how do you test a decorator without testing the code it decorates?
  • People will be tempted to extend this concept and make more decorators that do far more things (database operations, disk writes, etc.) and this sets a bad precedent that decorators are in frequent use.
  • Some decorators are fine: @staticmethod, @classmethod, @property.  These are common and useful.  Further, some libraries like Twisted have some predefined that are useful.

As a general rule, wherever I see custom decorators used, I'm sorely tempted to yank them for the future good of humanity. Of course, I'm not free to do that a lot of the time, but it's a temptation.

TL;DR version: If you have custom decorated code, yank if possible; Don't write custom decorators, it makes for crazymaking code.

I think frequently people write this stuff just to prove they're smarter than I am. Perhaps that's true. But, I'm smart enough to realize that the simplest possible code, even if it takes more lines, is the best possible code.

-- Kevin J. Rice

9 years in Python and counting, and loving it (except rampant use of decorators and dependency injection).