Day 17 of 50 Days of Python: Functional Programming using Lambdas, Map, Filter and Reduce

Part of week 3: Intermediate Python

Welcome to Day 17! Today, we’re going to be covering the functional programming paradigm in Python. Functional programming is a tool for writing concise and reusable code. Which is especially powerful in Data Engineering pipelines.

We’ll be focusing on:

• Lambda Functions - Something we have mentioned briefly in week 1, Day 6. [Recap: These are small anonymous functions]

• Map, Filter, and Reduce - Function tools for transforming, filtering and aggregating data.

---

Lambda Functions

A lambda function is a small function defined with the lambda keyword. You’ll typically use it for short, single-expression operations.

lambda arguments: expression

square = lambda x: x ** 2
print(square(5)) # Outputs: 25

Using Lambdas with Built-In Functions

pairs = [(1, 2), (3, 1), (5, 4)]
sorted_pairs = sorted(pairs, key=lambda x: x[1])

---

Map Function

The map() function applies a function to each item in a iterable (Lists etc).

map(function, iterable)

numbers = [1, 2, 3, 4]
doubled = map(lambda x: x*2, numbers)
print(list(doubled))

---

Filter Function

The filter() function does exactly what you think, it filters elements of an iterable based on a condition.

filter(function, iterable)

numbers = [1, 2, 3, 4]
evens = filter(lambda x: x% 2 == 0, numbers)
print(list(evens))

---

---

Reduce Function

The reduce() function from the functools module, applies a rolling computation to pairs of items in a sequence, reducing it to a single value.

from functools import reduce

reduce(function, iterable)

from functools import reduce
numbers = [1, 2, 3, 4]
product = reduce(lambda x, y: x * y, numbers
print(product)

---

When to Use Functional Programming

Functional Programming is ideal for:

• Data Transformation Pipelines: Efficiently process data step-by-step.

• Immutability: Avoid modifying original data directly.

• Cleaner, Readable Code: Simple logic with concise function applications.

---

Use Cases for Data Engineers

• Use lambda for inline data transformations

• Map, Filter and Reduce is great for ELT pipelines.

• Functional programming is ideal for concise pipeline steps and intuitive flow.

---

---

Next Up: Day 18 - Decorators & Generators

Tomorrow, we’ll cover decorators and generators. Decorators show you how to modify the behaviour of functions dynamically. Generators, which we have touched on before, will teach you to work with lazy evaluation.

So, see you tomorrow and happy coding!