Apply and Mapping Overview
Learn how to transform Pandas data using functions, apply operations, mapping techniques, and element-wise calculations.
1. Introduction to Apply and Mapping
Pandas provides several ways to apply custom logic to Series and DataFrames. These techniques help transform values, create calculated columns, and perform operations that are not directly available through built-in methods.
Learning Objectives
- Understand custom functions in Pandas.
- Use the Series
map()method. - Apply functions to Series and DataFrame objects.
- Understand row-wise and column-wise operations.
- Learn when to use vectorized operations instead.
- Recognize common performance and data-handling mistakes.
A function receives data, performs a calculation, and produces transformed or analyzed results.
2. Creating Sample Data
Start with a Series of product prices. This dataset will be used to demonstrate different function application techniques.
import pandas as pd
prices = pd.Series(
[100, 200, 300, 400, 500],
name="Price"
)
print(prices)
Output
0 100
1 200
2 300
3 400
4 500
Name: Price, dtype: int64
3. Defining a Custom Function
A custom function contains reusable logic. You can define a
function using Python's def keyword and then use
it with Pandas data.
def add_tax(price):
return price * 1.10
print(add_tax(100))
Output
110.00000000000001
The function adds ten percent to a price. Python may display a
small floating-point precision difference. For formatted output,
use round().
print(round(add_tax(100), 2))
Output
110.0
4. Using map() with a Custom Function
The Series map() method applies a function to each
value in a Series. It returns a new Series containing the
transformed values.
def add_tax(price):
return price * 1.10
tax_included_prices = prices.map(add_tax)
print(tax_included_prices.round(2))
Output
0 110.0
1 220.0
2 330.0
3 440.0
4 550.0
Name: Price, dtype: float64
map() is useful
when each individual value needs to be transformed or replaced.
5. Using map() with a Dictionary
A dictionary can map existing values to new values. This is useful for replacing category codes with descriptive labels.
grades = pd.Series(["A", "B", "C", "A", "B"])
grade_labels = {
"A": "Excellent",
"B": "Good",
"C": "Average"
}
descriptions = grades.map(grade_labels)
print(descriptions)
Output
0 Excellent
1 Good
2 Average
3 Excellent
4 Good
dtype: object
If a value is not present in the dictionary, the mapped result is normally missing. Check your mapping keys when working with real datasets.
6. Using map() with a Lambda Function
A lambda function is a small anonymous function. It is convenient when the transformation is short and does not need a separate function definition.
prices_doubled = prices.map(lambda value: value * 2)
print(prices_doubled)
Output
0 200
1 400
2 600
3 800
4 1000
Name: Price, dtype: int64
7. Using apply() with a Series
The Series apply() method applies a function to
each value. It is another option for applying custom logic to
Series elements.
def classify_price(price):
if price >= 400:
return "High"
elif price >= 200:
return "Medium"
else:
return "Low"
price_categories = prices.apply(classify_price)
print(price_categories)
Output
0 Low
1 Medium
2 Medium
3 High
4 High
Name: Price, dtype: object
In this example, each price is classified according to a conditional rule.
8. Applying a Function to a DataFrame
DataFrame apply() can apply a function to each
column or row. By default, axis=0 applies the
function to each column.
df = pd.DataFrame({
"Math": [80, 90, 75],
"Science": [85, 95, 70],
"English": [78, 88, 82]
})
print(df)
Output
Math Science English
0 80 85 78
1 90 95 88
2 75 70 82
Calculate the total of each column:
column_totals = df.apply(sum, axis=0)
print(column_totals)
Output
Math 245
Science 250
English 248
dtype: int64
9. Row-Wise Operations with apply()
Set axis=1 to apply a function to each row.
This is useful when a calculation depends on multiple columns
in the same record.
df["Total"] = df.apply(
lambda row: row["Math"]
+ row["Science"]
+ row["English"],
axis=1
)
print(df)
Output
Math Science English Total
0 80 85 78 243
1 90 95 88 273
2 75 70 82 227
apply().
10. Understanding map(), apply(), and Vectorization
| Technique | Typical Use | Example |
|---|---|---|
Series.map() |
Transform individual values or map labels. | prices.map(add_tax) |
Series.apply() |
Apply custom logic to Series elements. | prices.apply(classify_price) |
DataFrame.apply() |
Apply a function along rows or columns. | df.apply(sum, axis=0) |
| Vectorized operations | Perform calculations directly on Series or columns. | prices * 1.10 |
11. Prefer Vectorized Operations When Possible
Many calculations can be performed directly on an entire Series without writing a custom function. These vectorized expressions are usually easier to read and often more efficient.
vectorized_prices = prices * 1.10
print(vectorized_prices.round(2))
Output
0 110.0
1 220.0
2 330.0
3 440.0
4 550.0
Name: Price, dtype: float64
This produces the same result as applying the tax function to every price individually.
12. Handling Missing Values
Custom functions and mapping operations may encounter missing values. Decide how missing data should be handled before performing a transformation.
values = pd.Series([10, 20, None, 40])
result = values.map(
lambda value: value * 2
if pd.notna(value)
else value
)
print(result)
Output
0 20.0
1 40.0
2 NaN
3 80.0
dtype: float64
13. Common Mistakes and Best Practices
Common Mistakes
- Using a dictionary that does not contain all expected keys.
- Forgetting that row-wise DataFrame operations use
axis=1. - Ignoring missing values inside custom functions.
- Using
apply()for simple arithmetic unnecessarily. - Writing complicated lambda expressions that are difficult to maintain.
Best Practices
- Use named functions when logic is complex or reused.
- Use
map()for Series value mapping. - Use DataFrame
apply()when row or column logic is required. - Prefer vectorized calculations when available.
- Test custom functions with missing and unexpected values.
14. Practice Exercises
Exercise 1: Apply a Discount
Create a function that reduces each price by 15 percent. Apply it to a Series of prices.
prices = pd.Series([100, 250, 400, 600])
# Write your solution below
Exercise 2: Map Department Codes
Map department codes such as IT, HR, and FIN to their descriptive department names.
departments = pd.Series(["IT", "HR", "FIN", "IT"])
# Write your solution below
Exercise 3: Calculate Row Totals
Use a DataFrame function to calculate the total marks for each student.
marks = pd.DataFrame({
"Math": [80, 90, 75],
"Science": [85, 95, 70]
})
# Write your solution below
Summary
Pandas supports custom data transformations through
Series.map(), Series.apply(), and
DataFrame.apply(). These methods help apply reusable
logic to values, rows, and columns.
Use map() to transform Series values or map
categories, and use apply() when custom logic must
operate on Series elements or DataFrame rows and columns.
Prefer vectorized operations for straightforward arithmetic because they are generally clearer and often more efficient. Handle missing values explicitly and choose the method that matches the structure of the calculation.