Resampling Data in Pandas
Resampling is a powerful Pandas technique for changing the frequency of time series data. It allows you to aggregate high-frequency observations into daily, weekly, or monthly summaries and convert data into different time intervals.
1. Learning Objectives
After completing this lesson, you will be able to:
- Understand the meaning and purpose of resampling.
- Use
resample()with a DatetimeIndex. - Aggregate data into daily, weekly, and monthly intervals.
- Apply sum, mean, minimum, maximum, and count operations.
- Resample multiple DataFrame columns.
- Understand upsampling and missing values.
- Use resampling to analyze real-world time series data.
2. What Is Resampling?
Resampling changes the frequency of time series observations. For example, hourly temperature measurements can be converted into daily averages, or daily sales can be grouped into monthly totals.
Common Applications
- Convert daily sales into monthly sales totals.
- Calculate daily average temperatures from hourly readings.
- Summarize minute-level sensor readings into hourly values.
- Aggregate financial data into weekly or monthly reports.
- Prepare time series data for visualization and analysis.
Resampling can reduce many detailed observations into a smaller summary dataset.
3. Understanding the resample() Method
The resample() method groups observations into
time-based intervals. You then apply an aggregation method such
as sum() or mean() to calculate a
result for each interval.
General syntax:
df.resample("frequency").aggregation_method()
dfis the DataFrame or Series.frequencydefines the time interval.aggregation_method()calculates the summary value.
resample(), the index must be a DatetimeIndex,
TimedeltaIndex, or PeriodIndex. Alternatively, specify a
datetime column using the on parameter.
4. Creating Sample Time Series Data
Before resampling, create a DataFrame containing dates as the index and sales values as the data.
import pandas as pd
dates = pd.date_range(
start="2026-01-01",
periods=10,
freq="D"
)
df = pd.DataFrame({
"Sales": [100, 150, 120, 180, 200,
170, 220, 190, 250, 230]
}, index=dates)
print(df)
Expected output:
Sales
2026-01-01 100
2026-01-02 150
2026-01-03 120
2026-01-04 180
2026-01-05 200
2026-01-06 170
2026-01-07 220
2026-01-08 190
2026-01-09 250
2026-01-10 230
This DataFrame contains daily sales observations and can be resampled into larger time intervals.
5. Resampling Daily Data into Monthly Totals
Use the month-end frequency ME and
sum() to calculate the total sales for each month.
monthly_sales = df["Sales"].resample("ME").sum()
print(monthly_sales)
Expected output:
2026-01-31 1810
Freq: ME, Name: Sales, dtype: int64
All ten observations fall in January 2026. Their sum is 1,810, so the monthly result contains one observation labeled with the month-end date.
ME for
month-end frequency. Some older examples use the legacy
M alias.
6. Resampling into Weekly Data
Weekly resampling groups observations into weekly intervals.
By default, the W frequency uses weeks ending
on Sunday.
weekly_sales = df["Sales"].resample("W").sum()
print(weekly_sales)
Expected output:
2026-01-04 550
2026-01-11 1260
Freq: W-SUN, Name: Sales, dtype: int64
The first interval contains January 1 through January 4. The second contains January 5 through January 11, including only the available observations through January 10.
7. Resampling into Daily Averages
Resampling can also calculate statistics other than totals.
Use mean() to calculate the average value in
each interval.
daily_average = df["Sales"].resample("D").mean()
print(daily_average.head())
Expected output:
2026-01-01 100.0
2026-01-02 150.0
2026-01-03 120.0
2026-01-04 180.0
2026-01-05 200.0
Freq: D, Name: Sales, dtype: float64
Because each daily interval contains one observation in this dataset, the daily average equals the original daily sales value.
8. Common Aggregation Methods
Pandas provides several aggregation methods that can be applied after resampling.
| Method | Purpose |
|---|---|
sum() |
Calculates the total for each interval. |
mean() |
Calculates the average value. |
min() |
Returns the smallest value. |
max() |
Returns the largest value. |
count() |
Counts non-missing observations. |
first() |
Returns the first valid observation in each interval. |
last() |
Returns the last valid observation in each interval. |
median() |
Calculates the median value. |
std() |
Calculates the sample standard deviation. |
Example: Apply Multiple Aggregations
result = df["Sales"].resample("W").agg(
["sum", "mean", "min", "max", "count"]
)
print(result)
Expected output:
sum mean min max count
2026-01-04 550 137.500000 100 180 4
2026-01-11 1260 210.000000 170 250 6
9. Resampling Multiple Columns
A DataFrame may contain several numeric columns. You can resample multiple columns together and calculate summaries for each one.
import pandas as pd
dates = pd.date_range(
"2026-01-01",
periods=6,
freq="D"
)
df = pd.DataFrame({
"Sales": [100, 150, 200, 120, 180, 250],
"Orders": [10, 15, 20, 12, 18, 25]
}, index=dates)
monthly = df.resample("ME").sum()
print(monthly)
Expected output:
Sales Orders
2026-01-31 1000 100
Pandas sums each numeric column independently for the monthly interval.
10. Resampling with a Datetime Column
A DatetimeIndex is not always required. If your dates are stored
in a DataFrame column, use the on parameter to
identify the datetime column.
import pandas as pd
df = pd.DataFrame({
"Date": pd.to_datetime([
"2026-01-01",
"2026-01-15",
"2026-02-05",
"2026-02-20"
]),
"Sales": [100, 200, 150, 250]
})
monthly = df.resample("ME", on="Date")["Sales"].sum()
print(monthly)
Expected output:
Date
2026-01-31 300
2026-02-28 400
Freq: ME, Name: Sales, dtype: int64
The on="Date" argument tells Pandas to use the
Date column for grouping observations into monthly intervals.
11. Understanding Upsampling
Upsampling increases the frequency of a time series. For example, daily data can be converted into hourly-indexed data. New time intervals may contain missing values because the original dataset has no observations for those timestamps.
Example: Convert Daily Data to 12-Hour Intervals
import pandas as pd
dates = pd.date_range(
"2026-01-01",
periods=3,
freq="D"
)
series = pd.Series(
[100, 200, 300],
index=dates
)
result = series.resample("12h").asfreq()
print(result)
Expected output:
2026-01-01 00:00:00 100.0
2026-01-01 12:00:00 NaN
2026-01-02 00:00:00 200.0
2026-01-02 12:00:00 NaN
2026-01-03 00:00:00 300.0
Freq: 12h, dtype: float64
The new noon timestamps have no original values, so Pandas
represents them as NaN.
12. Filling Missing Values After Upsampling
After upsampling, you can fill missing values using forward filling or backward filling. The correct strategy depends on the meaning of the data.
Example: Forward Fill
result = series.resample("12h").ffill()
print(result)
Expected output:
2026-01-01 00:00:00 100
2026-01-01 12:00:00 100
2026-01-02 00:00:00 200
2026-01-02 12:00:00 200
2026-01-03 00:00:00 300
Freq: 12h, dtype: int64
Forward filling propagates the most recent available value into newly created intervals until another observation appears.
13. Resampling with First and Last Values
Some time series applications require the first or last observation in each interval rather than a sum or average.
import pandas as pd
dates = pd.date_range(
"2026-01-01",
periods=6,
freq="D"
)
prices = pd.Series(
[100, 110, 105, 120, 125, 130],
index=dates
)
weekly_first = prices.resample("W").first()
weekly_last = prices.resample("W").last()
print("First:")
print(weekly_first)
print("Last:")
print(weekly_last)
Expected output:
First:
2026-01-04 100
2026-01-11 125
Freq: W-SUN, dtype: int64
Last:
2026-01-04 120
2026-01-11 130
Freq: W-SUN, dtype: int64
The first weekly interval ends on January 4. The second ends on January 11 and contains the remaining available observations.
14. Common Resampling Frequencies
| Frequency | Description | Example Application |
|---|---|---|
h | Hourly intervals | Hourly sensor readings |
12h | 12-hour intervals | Half-day summaries |
D | Daily intervals | Daily sales reports |
W | Weekly intervals ending Sunday by default | Weekly sales reports |
MS | Month-start intervals | Monthly periods beginning at month start |
ME | Month-end intervals | Month-end summaries |
QE | Quarter-end intervals | Quarterly financial summaries |
YE | Year-end intervals | Annual totals |
The correct frequency depends on the reporting period. Check the supported aliases for your installed Pandas version, especially when using calendar-end frequencies.
15. Real-World Example: Monthly Revenue Report
Consider a business that records transactions on different days. Resampling can calculate total monthly revenue from the daily records.
import pandas as pd
df = pd.DataFrame({
"Date": pd.to_datetime([
"2026-01-05",
"2026-01-18",
"2026-02-03",
"2026-02-22",
"2026-03-10"
]),
"Revenue": [5000, 7000, 6000, 9000, 8000]
})
monthly_revenue = (
df.resample("ME", on="Date")["Revenue"]
.sum()
)
print(monthly_revenue)
Expected output:
Date
2026-01-31 12000
2026-02-28 15000
2026-03-31 8000
Freq: ME, Name: Revenue, dtype: int64
The result shows total revenue for January, February, and March 2026. This type of summary is useful for monthly reporting and trend analysis.
16. Common Mistakes to Avoid
Using a Non-Datetime Index
Convert the date column with pd.to_datetime()
and set it as the index, or pass the datetime column through
the on parameter.
Choosing the Wrong Frequency
Daily, weekly, and month-end intervals produce different groups. Choose the frequency that matches the reporting rule.
Ignoring Missing Values
Upsampling may create missing observations. Inspect them before deciding whether to fill, interpolate, or leave them missing.
Assuming Every Interval Has Data
Some periods may have no observations. Depending on the operation, the resulting interval can contain missing values or an aggregation identity such as zero for a sum.
17. Practice Exercises
Exercise 1: Monthly Sales
Create daily sales data for January and February 2026 and calculate monthly sales totals.
Exercise 2: Weekly Average
Generate 20 daily observations and calculate their weekly average.
Exercise 3: Multiple Aggregations
Calculate the sum, mean, minimum, and maximum of weekly sales.
Exercise 4: Upsampling
Convert daily observations into 12-hour intervals and identify the newly created missing values.
Exercise 5: Revenue Report
Create a DataFrame with a Date column and Revenue column, then calculate monthly revenue using the on parameter.
18. Knowledge Check
Question 1
Which method is used to resample time series data?
Answer: resample()
Question 2
Which operation calculates monthly sales totals?
Answer: resample("ME").sum()
Question 3
What is upsampling?
Answer: Increasing the frequency of a time series, potentially creating new intervals without original observations.
Question 4
How can you resample using a datetime column without setting it as the index?
Answer: Use the on parameter, for example df.resample("ME", on="Date").
Summary
Resampling is used to change the frequency of time series data.
The resample() method groups observations into
daily, weekly, monthly, or other time intervals, after which
aggregation methods calculate totals, averages, and other
statistics.
Resampling works with datetime-based indexes or a datetime
column specified through on. Downsampling creates
summaries over larger intervals, while upsampling creates
finer intervals and may introduce missing values. Choosing
appropriate frequencies and missing-value strategies is
essential for accurate analysis.