Multiple Conditions in Pandas
Learn how to combine multiple filtering conditions using AND, OR, and NOT operators to retrieve specific rows from a Pandas DataFrame.
1. Introduction to Multiple Conditions
In Pandas, multiple conditions allow us to filter DataFrame rows based on two or more requirements. This technique is useful when a single condition is not sufficient to identify the required data.
For example, a student management system may need to find students who scored more than 70 marks and have attendance of at least 80%. Both requirements must be checked together.
Sample Student DataFrame
| Name | Marks | Attendance | City |
|---|---|---|---|
| Samadhan | 85 | 90 | Pune |
| Pradnya | 68 | 85 | Mumbai |
| Samiksha | 92 | 75 | Pune |
| Rahul | 76 | 88 | Nashik |
| Neha | 55 | 60 | Mumbai |
Multiple conditions are combined to determine which DataFrame rows satisfy the filtering requirements.
2. Create a Sample DataFrame
First, import Pandas and create a DataFrame containing student names, marks, attendance percentages, and cities.
import pandas as pd
df = pd.DataFrame({
"Name": [
"Samadhan",
"Pradnya",
"Samiksha",
"Rahul",
"Neha"
],
"Marks": [85, 68, 92, 76, 55],
"Attendance": [90, 85, 75, 88, 60],
"City": [
"Pune",
"Mumbai",
"Pune",
"Nashik",
"Mumbai"
]
})
print(df)
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune 1 Pradnya 68 85 Mumbai 2 Samiksha 92 75 Pune 3 Rahul 76 88 Nashik 4 Neha 55 60 Mumbai
3. Using the AND Operator (&)
The AND operator selects rows only when every specified condition
evaluates to True. In Pandas, use the ampersand operator
& to combine Boolean Series element by element.
Example Requirement
Select students whose marks are greater than 70 and whose attendance is at least 80.
result = df[
(df["Marks"] > 70) &
(df["Attendance"] >= 80)
]
print(result)
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune 3 Rahul 76 88 Nashik
Explanation
- Samadhan satisfies both conditions because his marks are 85 and attendance is 90.
- Rahul satisfies both conditions because his marks are 76 and attendance is 88.
- Pradnya does not qualify because her marks are not greater than 70.
- Samiksha does not qualify because her attendance is below 80.
- Neha does not satisfy either condition.
4. Using the OR Operator (|)
The OR operator selects a row when at least one condition is True.
If both conditions are True, the row is also included.
In Pandas, use the pipe operator |.
Example Requirement
Select students whose marks are greater than 90 or whose attendance is at least 85.
result = df[
(df["Marks"] > 90) |
(df["Attendance"] >= 85)
]
print(result)
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune 1 Pradnya 68 85 Mumbai 2 Samiksha 92 75 Pune 3 Rahul 76 88 Nashik
Explanation
- Samadhan qualifies because attendance is 90.
- Pradnya qualifies because attendance is 85.
- Samiksha qualifies because marks are 92.
- Rahul qualifies because attendance is 88.
- Neha does not qualify because neither condition is True.
AND requires all conditions to be True, while OR requires at least one condition to be True.
5. Using the NOT Operator (~)
The NOT operator reverses a Boolean condition. In Pandas,
the tilde operator ~ inverts each Boolean value
in a Boolean Series.
Example Requirement
Select all students whose city is not Pune.
result = df[
~(df["City"] == "Pune")
]
print(result[["Name", "City"]])
Expected Output:
Name City 1 Pradnya Mumbai 3 Rahul Nashik 4 Neha Mumbai
Explanation
The condition df["City"] == "Pune" identifies
rows where the city is Pune. Applying ~ reverses
the Boolean mask, so rows where the city is not Pune are selected.
~ to invert a Pandas Boolean Series.
Python's not operator cannot directly evaluate an
entire Series as a single Boolean value.
6. Combining Three or More Conditions
Pandas allows you to combine three or more conditions in a single filter expression. Connect each condition with the appropriate Boolean operator.
Example Requirement
Select students whose marks are greater than 70, attendance is at least 80, and city is Pune.
result = df[
(df["Marks"] > 70) &
(df["Attendance"] >= 80) &
(df["City"] == "Pune")
]
print(result)
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune
Samadhan is the only student who satisfies all three conditions. Although Samiksha lives in Pune and has marks above 70, her attendance is below 80.
7. Combining AND and OR in One Expression
Some filtering requirements involve both AND and OR operators. Use parentheses to group conditions and make the intended logic explicit.
Example Requirement
Select students who either have marks greater than 90 and attendance of at least 80, or live in Pune.
The logic is: (Marks > 90 AND Attendance >= 80) OR City = Pune
result = df[
(
(df["Marks"] > 90) &
(df["Attendance"] >= 80)
) |
(df["City"] == "Pune")
]
print(result[["Name", "Marks", "Attendance", "City"]])
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune 2 Samiksha 92 75 Pune
How the Expression Works
- The first group checks whether marks are greater than 90 and attendance is at least 80.
- The second group checks whether the city is Pune.
- The OR operator combines the results of these two groups.
- A row is selected if either group evaluates to True.
& or |.
This avoids operator-precedence errors and makes the expression
easier to read.
8. Filtering with isin()
The isin() method checks whether each value in a
Series belongs to a specified collection of values. It is
particularly useful when a column can contain several acceptable
values.
Example Requirement
Select students who live in either Pune or Nashik.
result = df[
df["City"].isin(["Pune", "Nashik"])
]
print(result[["Name", "City"]])
Expected Output:
Name City 0 Samadhan Pune 2 Samiksha Pune 3 Rahul Nashik
Combining isin() with Another Condition
You can combine isin() with other conditions to
narrow down the results further.
result = df[
(df["City"].isin(["Pune", "Mumbai"])) &
(df["Marks"] > 70)
]
print(result[["Name", "Marks", "City"]])
Expected Output:
Name Marks City 0 Samadhan 85 Pune 2 Samiksha 92 Pune
9. Filtering Numeric Ranges with between()
The between() method checks whether values fall
within a specified range. By default, both endpoints are included.
result = df[
df["Marks"].between(70, 90)
]
print(result[["Name", "Marks"]])
Expected Output:
Name Marks 0 Samadhan 85 3 Rahul 76
This example selects marks from 70 through 90, including both endpoints. In the sample data, Samadhan and Rahul fall within that range.
between() for a range-based condition and
isin() when checking membership in a collection
of specific values.
10. Using query() for Multiple Conditions
Pandas provides the query() method as another way
to filter DataFrame rows. It can make some compound conditions
more readable.
result = df.query(
"Marks > 70 and Attendance >= 80"
)
print(result)
Expected Output:
Name Marks Attendance City 0 Samadhan 85 90 Pune 3 Rahul 76 88 Nashik
The result matches the earlier AND example. Inside a
query() expression, the words and
and or can be used to express logical conditions.
query() with trusted expressions. Avoid building
query strings directly from untrusted user input.
11. Common Mistakes and Best Practices
| Incorrect Approach | Correct Approach | Explanation |
|---|---|---|
df["Marks"] > 70 and df["Attendance"] >= 80
|
(df["Marks"] > 70) & (df["Attendance"] >= 80)
|
Use element-wise operators to combine Pandas Series. |
df["Marks"] > 70 & df["Attendance"] >= 80
|
(df["Marks"] > 70) & (df["Attendance"] >= 80)
|
Parentheses ensure each comparison is evaluated as a separate condition. |
df["City"] == "Pune" or df["City"] == "Nashik"
|
df["City"].isin(["Pune", "Nashik"])
|
Use isin() to check multiple allowed values.
|
df[not (df["City"] == "Pune")]
|
df[~(df["City"] == "Pune")]
|
Use the tilde operator to invert a Boolean Series. |
| Writing a long filter without grouping | Group comparisons with parentheses | Explicit grouping improves readability and prevents operator-precedence errors. |
12. Practical Exercises
Exercise 1: AND Filtering
Select students whose marks are at least 80 and whose attendance is at least 85.
Expected Result: Samadhan.
Exercise 2: OR Filtering
Select students whose marks are greater than 90 or whose city is Mumbai.
Expected Names: Pradnya, Samiksha, and Neha.
Exercise 3: NOT Filtering
Select all students who do not live in Pune.
Expected Names: Pradnya, Rahul, and Neha.
Exercise 4: Combined Conditions
Select students from Pune or Mumbai whose marks are greater than 70.
Expected Names: Samadhan and Samiksha.
Exercise 5: Range Filtering
Select students whose marks are between 60 and 90, including both endpoints.
Expected Names: Samadhan, Pradnya, and Rahul.
13. Quick Revision Questions
- Which operator combines Boolean Series using AND?
- Which operator combines Boolean Series using OR?
- What does the tilde operator do to a Boolean mask?
- Why should parentheses be used around individual comparisons?
- When is the
isin()method useful? - What is the purpose of the
between()method? - How can the
query()method filter multiple conditions? - What is the difference between AND and OR filtering?
Summary
Multiple-condition filtering allows you to select DataFrame rows
using several requirements at the same time. Use
& for AND, | for OR, and
~ for NOT when working with Pandas Boolean Series.
Use parentheses around comparisons to create clear and correct
filter expressions. The isin() method checks whether
values belong to a specified collection, while
between() checks whether numeric values fall within
a range. The query() method offers an alternative
syntax for many filtering operations.
These techniques are useful for filtering student records, analyzing sales, selecting customer segments, and extracting relevant information from large datasets.