NumPy Aggregate and Statistical Functions
NumPy provides powerful aggregate and statistical functions for summarizing numerical data. These functions help calculate totals, averages, medians, variances, standard deviations, and other useful measurements. In this lesson, you will learn how to summarize one-dimensional and multidimensional arrays, use the axis parameter, and apply statistical functions to practical data analysis tasks.
1. Introduction to Aggregate Functions
An aggregate function combines multiple values into a summary result. For example, a list of student marks can be summarized by calculating the total marks, average marks, minimum mark, and maximum mark.
Common Aggregate Functions
np.sum()calculates the sum of values.np.mean()calculates the arithmetic mean.np.min()returns the minimum value.np.max()returns the maximum value.np.median()calculates the median.np.std()calculates standard deviation.np.var()calculates variance.
Output:
These functions provide a quick overview of the values in an array without requiring you to write manual loops.
2. Calculating Sum and Product
The sum of an array is the total of its elements. The product is the result of multiplying its elements together.
Output:
Related Functions
| Function | Purpose | Example Result |
|---|---|---|
np.sum() |
Adds the array elements. | 2 + 3 + 4 + 5 = 14 |
np.prod() |
Multiplies the array elements. | 2 × 3 × 4 × 5 = 120 |
np.cumsum() |
Returns cumulative sums. | [2, 5, 9, 14] |
np.cumprod() |
Returns cumulative products. | [2, 6, 24, 120] |
Output:
3. Mean, Median, and Average
Mean and median are measures of central tendency. They help describe the center of a dataset, but they behave differently when extreme values are present.
Mean
The arithmetic mean is calculated by adding all values and dividing the total by the number of values.
Output:
Median
The median is the middle value after sorting the data. If the dataset contains an even number of values, the median is the average of the two middle values.
Output:
The extreme value of 1000 increases the mean considerably, while the median remains at 30. This makes the median useful when data contains unusually large or small observations.
4. Minimum, Maximum, and Range
Minimum and maximum functions identify the smallest and largest values. The range is the difference between the maximum and minimum.
Output:
Visualizing the Range
The range measures the difference between the largest and smallest observations.
5. Variance and Standard Deviation
Variance measures the average squared deviation from the mean. Standard deviation is the square root of variance and expresses dispersion in the same units as the original data.
Output:
Population and Sample Calculations
By default, np.var() and np.std() use
ddof=0, which calculates population variance and standard
deviation. For a sample estimate, use ddof=1.
Output:
ddof according to the
statistical question. Population and sample calculations use different
denominators and therefore produce different results.
6. Percentiles and Quantiles
Percentiles indicate the position below which a given percentage of observations falls. Quantiles divide ordered data into specified proportions. They are commonly used to understand distributions and compare values within a dataset.
Output:
| Function | Input Scale | Example |
|---|---|---|
np.percentile() |
0 to 100 | np.percentile(scores, 90) |
np.quantile() |
0 to 1 | np.quantile(scores, 0.90) |
The default interpolation method is linear. Percentile results can therefore be values that do not appear directly in the original array.
7. Understanding the Axis Parameter
For multidimensional arrays, the axis parameter specifies
the direction in which an operation is performed.
axis=0reduces rows and calculates results for each column.axis=1reduces columns and calculates results for each row.axis=Noneuses all elements in the array. This is the default for many aggregate functions.
Output:
Axis operations summarize a matrix by column or by row.
axis=0 returns one
result per column, while axis=1 returns one result per row.
8. Counting Values and Handling NaN
The np.count_nonzero() function counts values that are
considered nonzero. NumPy also provides NaN-aware aggregate functions
that can ignore missing numerical values represented by np.nan.
Output:
Handling Missing Numerical Values
Output:
9. Practical Project: Student Marks Analysis
In this example, we will analyze marks for three students across three subjects. We will calculate each student's total and average, along with the average score for each subject.
Output:
What This Project Demonstrates
np.sum(marks, axis=1)calculates each student's total.np.mean(marks, axis=1)calculates each student's average.np.mean(marks, axis=0)calculates each subject's average.np.max(marks)identifies the highest mark in the matrix.np.min(marks)identifies the lowest mark in the matrix.
10. Common Mistakes and Best Practices
Confusing Mean and Median
The mean uses every value in the arithmetic calculation, while the median depends on the middle position of the sorted data. Choose the measure that best represents your dataset.
Using the Wrong Axis
Before calculating row-wise or column-wise summaries, confirm the arrangement of the data and select the correct axis.
Ignoring Missing Values
A regular mean or sum can propagate NaN values. Consider NaN-aware functions when ignoring missing numerical values is appropriate.
Confusing Population and Sample Standard Deviation
The default ddof=0 differs from the sample calculation
with ddof=1. Use the setting that matches your analysis.
Rounding Too Early
Keep full precision during intermediate calculations when possible. Round the final result for display or reporting.
11. Practice Exercises
Exercise 1: Basic Aggregates
Create an array containing [12, 18, 24, 30, 36].
Calculate its sum, product, mean, minimum, and maximum.
Exercise 2: Mean and Median
Create an array containing [10, 15, 20, 25, 100].
Compare its mean and median. Explain why they differ.
Exercise 3: Variance and Standard Deviation
Calculate population variance and standard deviation for
[5, 10, 15, 20, 25]. Repeat the calculations using
ddof=1.
Exercise 4: Axis Operations
Create a 3 by 3 array of numbers. Calculate row totals, column totals, row averages, and column averages.
Exercise 5: Percentiles
Use np.percentile() to calculate the 25th, 50th, and
75th percentiles of a numerical dataset.
Exercise 6: Student Analysis
Create a matrix containing marks for three students in four subjects. Find the total and average for each student and the average for each subject.
Summary
Aggregate and statistical functions turn raw numerical arrays into useful summaries. They help describe totals, central tendency, spread, and the distribution of values.
- Use
np.sum()andnp.prod()for totals and products. - Use
np.mean(),np.average(), andnp.median()to describe central tendency. - Use
np.min(),np.max(), and the difference between them to calculate the range. - Use
np.var()andnp.std()to measure data variability. - Use
np.percentile()andnp.quantile()to study the distribution of values. - Use the
axisparameter to calculate row-wise or column-wise results. - Use NaN-aware functions when missing numerical values should be excluded from a calculation.
These functions are essential tools for data analysis and prepare you for more advanced numerical work with NumPy.