--
I’m a software engineer at Lab Digital. In this article I want to take a look at how to apply concepts from functional programming into everyday python/django programming.
Before I even became a software engineer, some of my friends were preaching about ‘Functional Programming’. It honestly seemed like a cult. Every single member of this cult was very well informed; they were smart, they knew a lot about programming, and they would gladly tell you why FP is the future of programming with infectious enthusiasm. They would give you a number of reasons for why FP is the only solution you ever really need.
Complex problems like concurrency become trivial, you can use async everywhere without worry!
Because FP operates on this notion of pure functions, you will never have runtime errors!
It has lazy evaluation and function composition, which makes it extremely efficient in data transformation!
Thanks to its ‘high-level’ syntax it is also way more readable!
But no matter which combination of benefits they would mention, they would always end on the same note: “Functional Programming has made me a better programmer”. That common sentiment is what I will focus on in this article. I hope to show you the non-technical superpowers you obtain from thinking from an FP point of view and how I incorporate them into my way of working without ever having to actually write in an FP style and violate django best practices.
While you can do FP in python, it is very clearly not designed for that. The standard library lacks compile-time optimizations and concepts like pattern-matching or proper type checking. This is a very clear design choice; Guido van Rossum (creator of Python) himself has once stated that he did not envision python as a functional language, so it isn’t one. Django very much builds on top of this design, embracing Object Oriented and Imperative paradigms. Take the CreateView for instance, a staple of class-based views (which are the django gold standard for view logic). It has 9 ancestors demonstrating some pretty nifty inheritance and dependency injection, a big no-no in FP. You could potentially write your django application with function based views, which are more akin to FP, but the reality is that nobody does this and 90% of all examples you will find online will be about class-based views. Concurrency, broadly speaking, is a non-issue in most django applications. The data transformations in web-development more often than not are relatively trivial, their complexity coming from dealing with state. So why then should you even care about FP as a django developer?
I argue that some really basic FP ideas make you really good at:
factoring your code into small (easily testable) units
using (and abusing) the django ORM
interpreting user requirements
expressing the data flow of your application
This will be the first article in the history of FP that does not talk about the citizenship status of functions, suffice it to say that functions are really important. What I think is more relevant for using FP in django is the idea of pure functions. These are functions that have no external dependencies. They operate on input and produce an output. They are completely deterministic and have no side-effects. fn = lambda x, y: x+ y is an example of a pure function. What I want to impose on you is that there is purity in almost every meaningful computation and identifying and isolating it makes your code simpler. In django you will often find yourself querying the database, then doing something with that data, and finally feeding it into a view to display it. The 'doing something with that data' generally is where you will find your pure computation. Take the following example:
There are many criticisms to be had over this code, but if you disregard some of its obvious flaws, I think it lends itself beautifully to showcase purity in an otherwise stateful transaction. Looking at this code we can see that the ingredient filtering can be completely deterministic. Given a list of ingredients and a list of ingredient names we should always produce the same set of ingredients.
You could argue this is abstraction for abstractions sake and that it is pointless. It doesn’t improve readability by all that much after all, yet I think this is way more powerful than the initial scenario, because filter_ingredients can be tested in complete isolation and has no side effects. You can always assume that your return value will be a subset of the ingredients that you passed in. It is said that a human brain has, on average, 7 threads that it can follow (give or take 2 depending on your physical well-being). Anytime you can free up a thread by factoring away some code into a pure function is a win for future maintenance and I think not doing it is a wasted opportunity to introduce simplicity. This specific example is really short and could actually be solved by some builtin python functionality, but imagine the filtering process being even slightly more convoluted; being able to forget about 5 or 6 lines of code is worth introducing an additional function, even if it is used only once. Though I generally opt to tackle a problem this way, I am aware of a pretty important tradeoff here: readability. It's fine when you lift one function out, but it becomes really problematic when you have to scroll up and down in a file (or worse, switch between multiple files) just to understand what is going on. I think there cannot be a general rule in this case, you have to evaluate the implications on readability every single time when making the choice to abstract away some pure computation.
Functional Programming is married to the concept of declarative programming. This is in contrast to the imperative style that python promotes. Generally the difference is explained by saying that declarative programming describes what you want and imperative programming describes how you want to get what you want.
This example is a little unfair, but it is a fantastic demonstration of declarative programming with django and its ORM. I have many things to say about the django ORM, but for now all I will say is that using values and values_list almost exclusively is one of the best decisions you can make. It forces you to understand what data you actually need rather than just grabbing everything, it communicates your intent to any developer that will touch the code after you, and the optimization is just crazy good. This example leverages SQL, but list comprehensions are a good example of a more declarative style of programming. Thanks to list comprehensions you do not need to implement the way that the list is built up, that's abstracted by the syntactic sugar. This frees up more of those precious threads in your brain.
Thinking in functions is the core of FP, much as thinking in objects and object relations is in OOP and I think for me this has become most evident in how I interpret user requirements. Lets say we have 3 tickets on our Jira board:
As a monkey I want to drink tea
As a giraffe I want to drink coffee
As a fish I want to go on long walks on the beach
If I look at this from an OO perspective, instinctively out comes the crystal ball. The common theme is animals, there is my base class. I see that 2 of the animals want to drink, so I have to have a philosophical debate about whether I am subdividing my animals in amphibians and non-amphibians, or drinkers and non-drinkers, or some other specification. Even if I know that at this point in time there is absolutely no reason to make any abstractions and creating 3 separate classes would probably be fine, the thought of abstraction does cross my mind. And even if I do the smart thing and create 3 classes, the semi-duplicate drinking behavior will bother me.
Looking at the same requirements from an FP point of view, I know that all I want is functions. I have a function that takes a monkey and tea and produces a zen monkey. I have a function that takes a giraffe and a coffee and produces a hyper giraffe. There is no thinking about hierarchies and classes, just inputs and outputs. This implicitly also makes it easier to solve a more complex problem, if you know the input and output of a complex use case you can start breaking it down at both ends, maybe you need to transform your input to achieve your output, or maybe your output needs to be composed of multiple properties that need to be computed. All of this results in a better idea of what you are going to design.
Inherently order matters in an imperative language. The implications of changing the order of your code are pretty good indicator of how well-factored, declarative, and composable your code is. Inheritance and dependency injection are good ways to introduce ordering issues into your code. You can see problems like this arise when you are constructing a Django View with mixins, the order of the mixin is extremely important in the behavior of the view. For a simpler example take the imperative pizza toppings example from earlier, if we moved the topping_names underneath the for loop, this function would not work. There is absolutely no reason to do it in this particular case, but the assignments in this code are volatile in that they have an influence on the code around it. This is fine for a function of 5 lines, but becomes more problematic as your functions get more complex. When you keep in mind that order should matter less, the logical consequence is that your code is more tailored towards composition, because you want to your code to be 'mobile'.
In this example the function call order impacts the output. To make it more declarative we compose the functions in a way that declares the intent of what we want to achieve. Not having to remember the order of your program frees up precious brain registers.
I cannot stress enough how integral functions are to the experience of FP. A great thing about functions is that they have a very limited scope in which they have access to values, unlike classes and objects that have this volatile self property that can be modified by any method in the class. Think of the CreateView and how often the context is overloaded through all of its 9 ancestors, constantly mutating the state of the view. Functional Programming prides itself on having very few side-effects and this is achieved by isolating and minimizing state. Anything that comes into contact with a stateful transaction is tainted with unreliability, so you should contain this to a box and stick warning signs all around it. When I write django code this box comes in 2 forms: state retrieval and state mutation. I keep 2 rules in my head when it comes to state management in django. First, if I need data from a stateful source, I will create an interface function that returns a python type (9 out of 10 times this is a dataclass purely for storing record-like data) that I can then pipe into my pure functions. Custom Querysets are fantastic places to store this type of logic. The other rule is that if I am mutating state, I will do so in a function that has no return value. What this ensures is that I am never dependent on a stateful transaction to have access to my data. I think Manager methods are a pretty good place to store this type of logic. Ultimately it doesn't really matter how you structure your code, just as long as you are aware that state is a contaminant. It always introduces complexity, so if you can limit the places where its complexity is relevant, then that will greatly improve the maintainability of your code.
When it comes to state management, invariably you also have to think about data mutability, to which FP says no. Mutating values can give a very false sense of security about your functions. I recently came across something like this:
A very real danger here is that the variable naming makes it seem as though you have created a new object, when in reality you have only modified an existing object. If you reuse the obj later with the assumption that it has no field property, you will have a bad time. The easy solution is to just not do this. Be mindful of mutability and if at all possible, avoid it. In this case we could rewrite the function:
With this solution you are no longer mutating the existing object, which is good from an immutability and no side-effects point of view. On the flipside you could argue you have introduced complexity to your function, which has made your application more complex. I would argue however that having mutable data inherently is more complex than any pure function you could write and if you can move complexity away from state into a pure function, you should.
I think that at the center of high-quality code lies simplicity, without it maintainability becomes a nightmare. It’s no wonder the third rule of the zen of python is that Simple is better than Complex. Edsger Dijkstra famously said that simplicity is a requirement for reliability. Many smart people seem to agree that simple software is good software. The superpower that functional programmers get from figuring out this unintuitive, mathematically sound way of programming is to make simplicity in code slightly easier. It achieves this by imposing some ideologies about how you approach design and problem solving. It forces you to let go of real-world comparisons to your code and in turn allows you to think clearly about the task at hand without worrying about the future. Its focus on purity in your computation makes it easier to understand how to factor your code into small units. It implores you to write down what you want rather than how to get there which allows you to express your thought process more succinctly. Just being aware of all of these concepts makes you better at designing solutions in your software, which is why it feels so fulfilling to do and why eventually you start preaching FP as if it is some holy deity. If any of this has piqued your interest and you feel like you want to experience all this and finally learn what a monad is, I recommend Learn You a Haskell for Great Good, it is a fantastic (and free) resource about learning haskell. It has amazing examples that introduce FP and showcase its utility and it is the best resource I have found on the internet so far.
Furthermore, if you are looking for a place where you can learn all of this and more, you might want to check out our vacancies page :)