Boolean Filtering in Pandas
Learn how Boolean conditions produce True and False values, how Boolean masks select DataFrame rows, and how to combine logical conditions for precise data analysis.
1. What Is Boolean Filtering?
Boolean filtering is a technique that selects rows using
conditions that evaluate to True or
False. Pandas keeps the rows whose corresponding
condition is true.
For example, if Samadhan wants to find students with marks of at least 80, Pandas can check each student's Marks value and return only the matching rows.
import pandas as pd
df = pd.DataFrame({
"Name": ["Samadhan", "Pradnya", "Samiksha", "Amit", "Neha"],
"Age": [21, 22, 20, 23, 21],
"City": ["Pune", "Mumbai", "Pune", "Nashik", "Mumbai"],
"Marks": [85, 92, 78, 88, 67]
})
print(df)
Output
Name Age City Marks 0 Samadhan 21 Pune 85 1 Pradnya 22 Mumbai 92 2 Samiksha 20 Pune 78 3 Amit 23 Nashik 88 4 Neha 21 Mumbai 67
2. Create a Boolean Mask
A Boolean mask is a Series of Boolean values generated by applying a condition to a column.
mask = df["Marks"] >= 80 print(mask)
Output
0 True 1 True 2 False 3 True 4 False Name: Marks, dtype: bool
The mask is aligned with the DataFrame's row index. Each Boolean value indicates whether that row satisfies the condition.
result = df[mask] print(result)
Output
Name Age City Marks 0 Samadhan 21 Pune 85 1 Pradnya 22 Mumbai 92 3 Amit 23 Nashik 88
3. Boolean Operators
Comparison operators are used to build Boolean conditions.
| Operator | Meaning | Example |
|---|---|---|
== |
Equal to | df["City"] == "Pune" |
!= |
Not equal to | df["City"] != "Pune" |
> |
Greater than | df["Marks"] > 80 |
< |
Less than | df["Age"] < 22 |
>= |
Greater than or equal to | df["Marks"] >= 80 |
<= |
Less than or equal to | df["Age"] <= 21 |
Example: Equal to
print(df["City"] == "Pune")
Output
0 True 1 False 2 True 3 False 4 False Name: City, dtype: bool
Example: Greater than
print(df["Marks"] > 85)
Output
0 False 1 True 2 False 3 True 4 False Name: Marks, dtype: bool
4. Visual Guide: Boolean Mask to Filtered Rows
Figure 1: Boolean indexing uses a mask to select matching rows.
5. Use the NOT Operator (~)
The tilde operator ~ performs element-wise
logical negation on a Boolean mask. It changes True to False
and False to True.
mask = df["City"] == "Pune" print(~mask)
Output
0 False 1 True 2 False 3 True 4 True Name: City, dtype: bool
Filter students who are not from Pune
not_pune = df[~(df["City"] == "Pune")] print(not_pune)
Output
Name Age City Marks 1 Pradnya 22 Mumbai 92 3 Amit 23 Nashik 88 4 Neha 21 Mumbai 67
6. Combine Boolean Conditions with AND (&)
The & operator keeps rows where both conditions
are true.
mask = (
(df["Marks"] >= 80) &
(df["Age"] <= 22)
)
print(df[mask])
Output
Name Age City Marks 0 Samadhan 21 Pune 85 1 Pradnya 22 Mumbai 92
Both students meet the marks requirement and the age requirement.
7. Combine Boolean Conditions with OR (|)
The | operator keeps rows where at least one
condition is true.
mask = (
(df["City"] == "Pune") |
(df["Marks"] >= 90)
)
print(df[mask])
Output
Name Age City Marks 0 Samadhan 21 Pune 85 1 Pradnya 22 Mumbai 92 2 Samiksha 20 Pune 78
Samadhan and Samiksha match the city condition. Pradnya matches the marks condition.
8. Check Whether Any or All Values Match
Boolean masks can be summarized with any()
and all().
any(): At least one value is True
mask = df["Marks"] >= 90 print(mask.any())
Output
True
all(): Every value is True
mask = df["Marks"] >= 60 print(mask.all())
Output
True
Use any() to check whether at least one record
matches and all() to check whether every record
matches.
9. Filter and Select Specific Columns
Boolean filtering can be combined with loc[]
to return only the columns needed in a report.
result = df.loc[
df["Marks"] >= 80,
["Name", "Marks"]
]
print(result)
Output
Name Marks 0 Samadhan 85 1 Pradnya 92 3 Amit 88
This avoids displaying unnecessary columns while keeping the original row labels.
10. Visual Guide: AND, OR, and NOT
Figure 2: AND combines requirements, OR allows alternatives, and NOT reverses a Boolean mask.
11. Common Mistakes and Best Practices
Mistake 1: Using Python's and/or
For element-wise Pandas conditions, use
& and | rather than
and and or.
Mistake 2: Forgetting parentheses
# Correct mask = (df["Marks"] >= 80) & (df["Age"] <= 22)
Mistake 3: Misunderstanding the mask
A mask contains Boolean values aligned to the DataFrame index. It is not the filtered DataFrame itself.
Mistake 4: Ignoring missing values
Missing values can affect comparisons and text operations. Decide explicitly how missing values should be handled.
high_marks or
pune_students. This makes filtering code easier
to read and debug.
12. Practice Exercises
Exercise 1: Boolean Masks
- Create a mask for marks greater than or equal to 80.
- Print the Boolean mask.
- Use the mask to display matching students.
Exercise 2: Logical Operators
- Find students aged 21 or younger who scored at least 80.
- Find students from Pune or Mumbai.
- Find students who are not from Pune.
Exercise 3: Mask Summary
- Check whether any student scored at least 90.
- Check whether all students scored at least 60.
- Display only the Name and Marks columns for scores of 80 or more.
Suggested solution
# Boolean mask mask = df["Marks"] >= 80 print(mask) print(df[mask]) # AND condition print(df[(df["Age"] <= 21) & (df["Marks"] >= 80)]) # OR condition print(df[df["City"].isin(["Pune", "Mumbai"])]) # NOT condition print(df[~(df["City"] == "Pune")]) # Mask summaries print((df["Marks"] >= 90).any()) print((df["Marks"] >= 60).all()) # Filter rows and select columns print(df.loc[df["Marks"] >= 80, ["Name", "Marks"]])
13. Revision Questions
- What is Boolean filtering?
- What is a Boolean mask?
- How do comparison operators generate Boolean values?
- What is the purpose of the tilde operator
~? - What is the difference between
&and|? - Why are parentheses important in multiple conditions?
- What do
any()andall()return? - How can you filter rows and select specific columns together?
Summary
Boolean filtering uses conditions that evaluate to True or False. A Boolean mask can be applied to a DataFrame to keep only rows that meet the required conditions.
Use & for AND, | for OR, and
~ for NOT. Methods such as any()
and all() help summarize Boolean results.
Combine masks with loc[] when you need to select
specific columns as well as rows.