NumPy Unique Values and Set Operations
NumPy provides functions for identifying unique values and performing mathematical set operations on arrays. These operations help remove duplicates, compare datasets, find common elements, and identify values that occur in only one dataset. In this lesson, you will learn how to use unique values, intersections, unions, differences, and membership checks for practical data analysis.
1. Introduction to Unique Values
A unique value is a value that appears only once in the resulting
collection of distinct values. The np.unique() function
returns the distinct values from an array in sorted order by default.
Output:
Repeated values are included only once in the unique result. The original array is not modified.
Why Use Unique Values?
- Remove duplicate values from a dataset.
- Identify distinct categories or numerical values.
- Count the number of different values.
- Prepare data for grouping and further analysis.
2. Counting Unique Values
You can determine how many distinct values occur in an array by
checking the size of the result returned by np.unique().
Output:
Count the Frequency of Each Unique Value
Set return_counts=True to obtain the frequency of each
unique value.
Output:
| Value | Frequency |
|---|---|
| 10 | 3 |
| 20 | 2 |
| 30 | 1 |
| 40 | 1 |
3. Finding Unique Values and Their Indices
NumPy can also return the indices of the first occurrences of unique values and the inverse mapping that reconstructs the original array from its unique values.
Output:
return_index=True reports the original index of the
first occurrence of each returned value.
4. Unique Values in Two-Dimensional Arrays
When an array contains multiple dimensions, np.unique()
can return unique elements across the entire array or unique rows
when axis=0 is specified.
Unique Elements Across the Entire Array
Output:
Unique Rows
Output:
With axis=0, NumPy compares complete rows and removes
duplicate rows. The remaining rows are returned in sorted order.
5. Introduction to Set Operations
Set operations compare two collections of values. They are useful when you need to find common values, combine distinct values, or identify values that exist in one array but not another.
Set operations compare the distinct values present in two arrays.
NumPy provides dedicated functions for these operations. Their results are generally sorted and contain distinct values.
6. Union of Two Arrays
The np.union1d() function returns the unique values that
occur in either of two arrays.
Output:
Values that occur in both arrays appear only once in the union.
Example Use Cases
- Combining distinct product IDs from two lists.
- Creating a combined list of categories.
- Finding every distinct value across two datasets.
7. Intersection of Two Arrays
The np.intersect1d() function returns values that are
present in both arrays. Duplicate values are removed from the result.
Output:
Return the Indices of Common Values
Output:
The returned index arrays identify the first matching occurrences of the common values in each input array.
8. Difference Between Two Arrays
The np.setdiff1d() function returns the unique values
that occur in the first array but not in the second array.
Output:
np.setdiff1d(a, b) is not necessarily equal to
np.setdiff1d(b, a).
9. Symmetric Difference
Symmetric difference returns values that occur in either array but
not in both. NumPy provides np.setxor1d() for this
operation.
Output:
| Operation | Function | Result for A = [1, 2, 3], B = [2, 3, 4] |
|---|---|---|
| Union | np.union1d() |
[1, 2, 3, 4] |
| Intersection | np.intersect1d() |
[2, 3] |
| Difference A − B | np.setdiff1d() |
[1] |
| Symmetric difference | np.setxor1d() |
[1, 4] |
10. Checking Membership with np.isin()
The np.isin() function checks whether each element of
one array appears in a collection of test values. It returns a
Boolean array with the same shape as the input array.
Output:
Find Values Not in a Given Collection
Output:
np.isin() to create a
membership mask, then use that mask to select matching or
non-matching elements.
11. Practical Project: Comparing Student Enrollments
Imagine two courses with different student enrollment lists. We can use NumPy set operations to find all enrolled students, students attending both courses, and students enrolled in only one course.
Output:
Project Learning Outcomes
- Combine distinct values using union.
- Identify shared values using intersection.
- Find values exclusive to each dataset using set difference.
- Find values appearing in exactly one collection using symmetric difference.
12. Common Mistakes and Practice Exercises
Common Mistakes
-
Expecting unique values in original order:
np.unique()sorts its output by default. - Expecting duplicates in set operations: set operation results contain distinct values.
- Confusing set difference direction: A minus B is different from B minus A.
-
Using membership checks incorrectly:
np.isin(a, b)checks each element ofaagainst the values inb.
Exercise 1: Unique Values
Create an array containing repeated values. Use
np.unique() to find the distinct values.
Exercise 2: Frequency Counting
Use return_counts=True to count how many times
each unique value appears.
Exercise 3: Union and Intersection
Given two arrays, find all distinct values and the values shared by both arrays.
Exercise 4: Difference
Use np.setdiff1d() to find values that occur in
the first array but not in the second.
Exercise 5: Symmetric Difference
Use np.setxor1d() to find values present in
exactly one of two arrays.
Exercise 6: Membership
Use np.isin() to select values that match a
specified list of target values.
Summary
In this lesson, you learned to identify distinct array values with
np.unique(), count their frequencies, retrieve first
occurrence indices, and identify unique rows in multidimensional
arrays.
You also practiced set operations using np.union1d(),
np.intersect1d(), np.setdiff1d(), and
np.setxor1d(). Finally, you learned to check membership
with np.isin() and apply these operations to compare
datasets.