Time Series Data in Pandas
Time series analysis is the process of working with data collected, recorded, or measured over time. Pandas provides powerful tools for handling dates, timestamps, time-based indexing, resampling, and rolling calculations.
1. Learning Objectives
After completing this lesson, you will be able to:
- Understand time series data and its applications.
- Convert strings into Pandas datetime values.
- Create date ranges using
pd.date_range(). - Use datetime values as a DataFrame index.
- Select records using dates and time periods.
- Resample data into daily, weekly, or monthly intervals.
- Use shifting and rolling calculations for time-based analysis.
2. What Is Time Series Data?
Time series data consists of observations associated with specific points or periods in time. The observations may be collected every second, minute, hour, day, month, or year.
Examples of Time Series Data
- Daily stock prices.
- Hourly temperature readings.
- Monthly sales revenue.
- Website traffic recorded every minute.
- Daily electricity consumption.
A time series chart displays how a value changes over time.
3. Import Pandas
Start by importing the Pandas library.
import pandas as pd
Expected output:
# No output is displayed.
# Pandas is now available through the pd alias.
4. Understanding Datetime Values
Dates are sometimes stored as strings. Pandas can convert these strings into datetime values, making date-based operations easier.
Example: Convert Strings to Datetime
import pandas as pd
dates = pd.Series([
"2026-01-01",
"2026-01-02",
"2026-01-03"
])
dates = pd.to_datetime(dates)
print(dates)
print(dates.dtype)
Expected output:
0 2026-01-01
1 2026-01-02
2 2026-01-03
dtype: datetime64[ns]
datetime64[ns]
pd.to_datetime() when you
need to convert date strings into datetime values.
5. Creating a Date Range
The pd.date_range() function generates a sequence of
dates or timestamps. It is useful when creating time series indexes.
Example: Generate Daily Dates
import pandas as pd
dates = pd.date_range(
start="2026-01-01",
periods=5,
freq="D"
)
print(dates)
Expected output:
DatetimeIndex([
'2026-01-01',
'2026-01-02',
'2026-01-03',
'2026-01-04',
'2026-01-05'
], dtype='datetime64[ns]', freq='D')
Common Frequency Aliases
| Frequency | Meaning | Example |
|---|---|---|
D |
Daily | One interval per calendar day |
h |
Hourly | One interval per hour |
min |
Every minute | One interval per minute |
W |
Weekly | Weekly date intervals |
MS |
Month start | First day of each month |
ME |
Month end | Last day of each month |
ME for
month-end frequency; older examples may use M.
6. Using Datetime as a DataFrame Index
A datetime index allows Pandas to identify records by time and supports convenient date-based selection and resampling.
Example: Create a Time Series DataFrame
import pandas as pd
dates = pd.date_range(
start="2026-01-01",
periods=4,
freq="D"
)
df = pd.DataFrame({
"Sales": [1200, 1500, 1350, 1800]
}, index=dates)
print(df)
Expected output:
Sales
2026-01-01 1200
2026-01-02 1500
2026-01-03 1350
2026-01-04 1800
Check the Index Type
print(type(df.index))
Expected output:
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
7. Selecting Data by Date
Once the index contains datetime values, you can select individual dates or date ranges using familiar DataFrame indexing techniques.
Select One Date
print(df.loc["2026-01-02"])
Expected output:
Sales 1500
Name: 2026-01-02 00:00:00, dtype: int64
Select a Date Range
print(df.loc["2026-01-02":"2026-01-03"])
Expected output:
Sales
2026-01-02 1500
2026-01-03 1350
.loc includes both endpoints when those labels exist
in the index.
8. Resampling Time Series Data
Resampling changes the frequency of time series observations. It can aggregate detailed data into larger time intervals or convert data into a finer frequency.
For example, daily sales can be grouped into monthly totals. The index must be datetime-like for standard datetime resampling.
Example: Calculate Monthly Sales
import pandas as pd
dates = pd.date_range(
start="2026-01-01",
periods=6,
freq="D"
)
sales = [100, 200, 150, 300, 250, 400]
df = pd.DataFrame({
"Sales": sales
}, index=dates)
monthly_sales = df["Sales"].resample("ME").sum()
print(monthly_sales)
Expected output:
2026-01-31 1400
Freq: ME, Name: Sales, dtype: int64
Common Resampling Operations
| Operation | Purpose |
|---|---|
.sum() |
Calculate the total for each interval. |
.mean() |
Calculate the average for each interval. |
.min() |
Find the minimum value for each interval. |
.max() |
Find the maximum value for each interval. |
.count() |
Count observations in each interval. |
9. Shifting Time Series Data
The shift() method moves values by a specified number
of rows. It is commonly used to compare current values with earlier
or later observations.
Example: Compare Current and Previous Sales
import pandas as pd
sales = pd.Series(
[100, 120, 90, 150],
index=pd.date_range(
"2026-01-01",
periods=4,
freq="D"
)
)
previous_sales = sales.shift(1)
print(previous_sales)
Expected output:
2026-01-01 NaN
2026-01-02 100.0
2026-01-03 120.0
2026-01-04 90.0
Freq: D, dtype: float64
The first observation becomes NaN because no previous
value exists. The other values move down by one position.
10. Rolling Window Calculations
A rolling window calculates a statistic over a moving group of observations. Rolling averages can help smooth short-term fluctuations and reveal patterns.
Example: Three-Observation Moving Average
import pandas as pd
sales = pd.Series([100, 120, 90, 150, 180])
rolling_average = sales.rolling(window=3).mean()
print(rolling_average)
Expected output:
0 NaN
1 NaN
2 103.333333
3 120.000000
4 140.000000
dtype: float64
With a window size of three, the first two positions are
NaN because they do not yet contain three observations.
By default, a complete window is required.
rolling(window=3).mean()
for a three-row moving average. A row-based window is different
from a time-based window such as rolling("3D").
11. Extracting Date and Time Components
The datetime accessor .dt provides properties for
extracting information such as year, month, day, and weekday.
Example
import pandas as pd
dates = pd.Series(
pd.to_datetime([
"2026-01-05",
"2026-02-10",
"2026-03-15"
])
)
print(dates.dt.year)
print(dates.dt.month)
print(dates.dt.day)
Expected output:
0 2026
1 2026
2 2026
dtype: int32
0 1
1 2
2 3
dtype: int32
0 5
1 10
2 15
dtype: int32
The exact integer dtype displayed can vary by Pandas version. The extracted values remain the same.
12. Handling Missing Time Series Values
Time series data may contain missing observations because of sensor failures, incomplete records, or irregular data collection. Pandas provides methods to identify and handle missing values.
Example: Fill Missing Values
import pandas as pd
series = pd.Series(
[10.0, None, 30.0, None, 50.0]
)
print(series.ffill())
Expected output:
0 10.0
1 10.0
2 30.0
3 30.0
4 50.0
dtype: float64
The ffill() method propagates the most recent
non-missing value forward. Choose a filling strategy based on
the meaning of your data; carrying values forward is not always
appropriate.
13. Important Time Series Methods
| Method | Purpose |
|---|---|
pd.to_datetime() |
Convert values into datetime objects. |
pd.date_range() |
Generate a sequence of dates. |
.resample() |
Group observations into time intervals. |
.shift() |
Move values across rows. |
.rolling() |
Calculate statistics over moving windows. |
.dt.year |
Extract the year from datetime values. |
.dt.month |
Extract the month from datetime values. |
.ffill() |
Fill missing values using previous values. |
14. Practice Exercises
Exercise 1: Create Dates
Create a date range containing seven daily dates starting from February 1, 2026.
Exercise 2: Build a Time Series
Create a DataFrame with daily dates as the index and a column containing seven sales values.
Exercise 3: Select a Date Range
Select observations from February 3 through February 5, 2026.
Exercise 4: Resample Data
Calculate monthly total sales from a DataFrame containing daily sales.
Exercise 5: Calculate a Moving Average
Calculate a three-observation rolling average for a Series of daily temperatures.
15. Knowledge Check
Question 1
Which function converts a column of date strings into datetime values?
Answer: pd.to_datetime()
Question 2
Which function creates a sequence of dates?
Answer: pd.date_range()
Question 3
Which method groups time series observations into monthly intervals?
Answer: .resample(), using an appropriate monthly frequency.
Question 4
Why can a rolling average begin with missing values?
Answer: The window may not contain enough observations to calculate a complete result.
Summary
Time series data represents observations over time. Pandas makes it easier to convert strings to datetime values, generate date ranges, create datetime indexes, select date-based records, and aggregate observations with resampling.
Methods such as shift(), rolling(),
.dt, and ffill() support comparisons,
moving calculations, date-component extraction, and missing-value
handling. These tools are useful for analyzing sales, finance,
weather, sensor readings, and other time-dependent datasets.