Functional Python with Map and Filter
Functional Programming
Functional programming is a programming paradigm which decomposes programs into a set of functions. The functions should take an input and produce an output that is deterministic. In other words, a function should always return the same output given a certain input. Furthermore, in functional programming, functions should not maintain internal state thus eliminating potential side effects.
Python is not a pure functional programming language, like Haskell or others, but there are functional features in Python for applying and composing functions. The built-in functions map and filter are examples of functional programming within Python.
While list comprehensions are usually the preferred syntax for concisely creating lists, map and filter are also applicable in some scenarios for an elegant solution.
Map
map accepts a function and one or more iterables as inputs. The result of calling map is a map object iterator. Each item from the iterable or sequence is passed through the supplied function per iteration. The snippet below demonstrates the basic usage.
func = lambda x: x**2 # anonymous function to operate on item values
iterable = range(1, 11) # sequence of integers from 1 to 10
squares = map(func, iterable) # generates a map object iterator
# Calling next processes and removes the next item from the iterator
print(next(squares)) # Output: 1
print(next(squares)) # output 4
# List the remaining 8 items in the iterator
print(list(squares)) # Output: [4, 9, 16, 25, 36, 49, 64, 81, 100]
print(list(squares)) # Output: []
If multiple iterables are passed to map, such as two lists, the applied function should accept a matching number of arguments.
func = lambda x, y: x+y # anonymous function to operate on item values
iterable = range(1, 11) # sequence of integers from 1 to 10
neg_iterable = range(-1, -11, -1) # sequence of integers from -1 to -10
squares = map(func, iterable, neg_iterable) # generates a map object iterator
print(list(squares)) # Output: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Filter
Similar to map, filter accepts a function as an argument but only up to one iterable. filter also returns an iterator (filter object). Only the items from the iterable which pass the conditional logic are included in the final output.
func = lambda x: x % 2 == 0 # anonymous function to conditionally filter items
iterable = range(1, 11) # sequence of integers from 1 to 11
squares = filter(func, iterable) # generates a filter object
print(list(squares)) # Output: [2, 4, 6, 8, 10]
Map and Filter
The two built-in functions can also be combined together creating a functional pairing.
map_func = lambda x: x**2 # anonymous function to operate on item values
filter_func = lambda x: x % 2 == 0 # anonymous function to conditionally filter
iterable = range(1, 11) # sequence of integers from 1 to 11
# Apply the filter result as an iterable input to map
squares = map(map_func, filter(filter_func, iterable))
print(list(squares)) # Output: [4, 16, 36, 64, 100]
Complex Scenario
For some complex scenarios, map and filter offer more graceful and readable solutions than normal for loops or list comprehensions.
The objective in the example problem below is to create average student grades per class (not to be confused with a Python class object). The sample data contains a list of three classes each containing three exam grades from three students (3x3x3 matrix).
For Loops
The first implementation to review is regular for loops. The result is code bloat and logic that is mentally taxing to follow.
# Helper function to calculate an average given a list of numbers
from statistics import mean
# Sample data of 3 classes with 3 exam grades from 3 students
grades = [
[[65, 87, 94], [60, None, 58], [93, 83, 76]],
[[79, 85, 82], [72, 70, 77], [88, 86, 90]],
[[92, 88, 96], [84, 88, 86], [95, 96, 97]]
]
# Compute student grade averages per class using for loops
averages = [] # Stateful list of averages
for class_grades in grades:
class_avgs = [] # Stateful list of student averages per class
for student_grades in class_grades:
exam_avgs = [] # Stateful list of exam averages per student
for exam_grade in student_grades:
if exam_grade is not None:
exam_avgs.append(exam_grade)
class_avgs.append(mean(exam_avgs))
averages.append(class_avgs)
print(averages)
# Output: [[82, 59, 84], [82, 73, 88], [92, 86, 96]]
List Comprehensions
The second implementation, for comparison, uses list comprehensions. Frankly, it's challenging to format coherently and more difficult to reason about.
# Helper function to calculate an average given a list of numbers
from statistics import mean
# Sample data of 3 classes with 3 exam grades from 3 students
grades = [
[[65, 87, 94], [60, None, 58], [93, 83, 76]],
[[79, 85, 82], [72, 70, 77], [88, 86, 90]],
[[92, 88, 96], [84, 88, 86], [95, 96, 97]]
]
# Compute student grade averages per class using a list comprehension
averages = [[mean([exam_grade for exam_grade in student_grades
if exam_grade is not None])
for student_grades in class_grades]
for class_grades in grades]
print(averages)
# Output: [[82, 59, 84], [82, 73, 88], [92, 86, 96]]
Map and Filter
Lastly, map and filter create a polished solution in conjunction together. The logic is explicitly described with named functions rather than anonymous, lambda functions like the initial examples at the top.
# Helper function to calculate an average given a list of numbers
from statistics import mean
# Sample data of 3 classes with 3 exam grades from 3 students
grades = [
[[65, 87, 94], [60, None, 58], [93, 83, 76]],
[[79, 85, 82], [72, 70, 77], [88, 86, 90]],
[[92, 88, 96], [84, 88, 86], [95, 96, 97]]
]
# Conditional filter logic exclude missed exam grades
def filter_exam_grades(exam_grade):
return exam_grade is not None
# Calculate average per list of student exam grades
def calculate_student_avgs(student_grades):
exam_grades = filter(filter_exam_grades, student_grades)
return mean(exam_grades)
# Calculate student averages per list of class grades
def calculate_class_avgs(class_grades):
return list(map(calculate_student_avgs, class_grades))
# Calculate student grade averages per class
averages = map(calculate_class_avgs, grades)
print(list(averages))
# Output: [[82, 59, 84], [82, 73, 88], [92, 86, 96]]
Coding Challenge
For an interactive challenge, can you fix the code below? The exam grades use a different data format per class. Running the program as-is throws the error exception TypeError: can't convert type 'str' to numerator/denominator. We need to resolve the error and match the same output from above [[82, 59, 84], [82, 73, 88], [92, 86, 96]].
Hint: map or list comprehension will be helpful to normalize all exam grades as integers.
Summary
The first two implementations contain deeply nested loops that are difficult to read and follow. On the other hand, the map and filter solution is well-defined. Moreover, the functional programming implementation contains bite size units of functionality which are independently testable with a deterministic output.
Regular for loops and especially list comprehensions cover a wide range of scenarios in Python for iterating over data sequences. Occasionally, you may want to pull from your functional programming toolbox with map and filter for an elegant alternative.