Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

Basic Plotting with Pandas

Learn how to create charts directly from Pandas DataFrames and Series using the plot() method, customise charts and save visualisations for reporting and analysis.

1. Introduction to Basic Plotting

Pandas provides a built-in plotting interface that makes it easy to create charts from DataFrames and Series. Instead of writing complex plotting code, we can use the plot() method.

Pandas uses Matplotlib as its default plotting backend. Therefore, we can create a chart using Pandas and further customise it using Matplotlib functions.

Important: The plot() method is useful for quick data visualisation. For advanced chart customisation, we can use Matplotlib directly.

Basic Syntax

DataFrame.plot()

Syntax with Parameters

DataFrame.plot(
    kind='line',
    x=None,
    y=None,
    title=None,
    figsize=None,
    color=None,
    grid=False,
    legend=True
)

2. Creating Sample Data for Plotting

Before creating charts, we need a dataset. Let us create a DataFrame containing monthly sales and expenses.

import pandas as pd
import matplotlib.pyplot as plt

data = {
    'Month': ['January', 'February', 'March',
              'April', 'May', 'June'],
    'Sales': [12000, 15000, 18000,
              14000, 22000, 25000],
    'Expenses': [8000, 9000, 11000,
                 8500, 13000, 15000]
}

df = pd.DataFrame(data)

print(df)

Expected Output

      Month  Sales  Expenses
0   January  12000      8000
1  February  15000      9000
2     March  18000     11000
3     April  14000      8500
4       May  22000     13000
5      June  25000     15000

We will use this DataFrame in the following examples to understand different plotting methods.

3. Creating a Line Plot

A line plot is used to display changes in numerical values over a continuous sequence, such as time or months. It connects data points using lines.

Example: Monthly Sales Line Plot

df.plot(
    x='Month',
    y='Sales',
    kind='line'
)

plt.title('Monthly Sales')
plt.xlabel('Month')
plt.ylabel('Sales')
plt.show()

Explanation

  • x='Month' selects the Month column for the X-axis.
  • y='Sales' selects the Sales column for the Y-axis.
  • kind='line' creates a line chart.
  • plt.title() sets the chart title.
  • plt.xlabel() sets the X-axis label.
  • plt.ylabel() sets the Y-axis label.
  • plt.show() displays the chart.
Use Case: A business can use a line plot to understand whether monthly sales are increasing or decreasing.

4. Creating a Bar Plot

A bar plot represents data using rectangular bars. It is useful when comparing numerical values across different categories.

Example: Month-wise Sales Comparison

df.plot(
    x='Month',
    y='Sales',
    kind='bar',
    color='skyblue'
)

plt.title('Month-wise Sales Comparison')
plt.xlabel('Month')
plt.ylabel('Sales')
plt.show()

Explanation

  • kind='bar' creates a vertical bar chart.
  • color='skyblue' sets the bar colour.
  • Each bar represents the sales value for one month.
Bar charts are commonly used to compare sales by product, students by department, employees by team and revenue by month.

5. Creating a Horizontal Bar Plot

A horizontal bar plot displays bars from left to right. It is useful when category names are long or when we want to compare categories vertically.

Example

df.plot(
    x='Month',
    y='Sales',
    kind='barh',
    color='orange'
)

plt.title('Monthly Sales - Horizontal Bar Chart')
plt.xlabel('Sales')
plt.ylabel('Month')
plt.show()

Difference Between bar and barh

Method Direction Usage
bar Vertical bars Comparing categories from bottom to top.
barh Horizontal bars Comparing categories from left to right.

6. Plotting Multiple Columns

Pandas allows us to plot multiple numerical columns in the same chart. This is useful for comparing two or more datasets.

Example: Sales vs Expenses

df.plot(
    x='Month',
    y=['Sales', 'Expenses'],
    kind='line',
    marker='o'
)

plt.title('Monthly Sales vs Expenses')
plt.xlabel('Month')
plt.ylabel('Amount')
plt.legend()
plt.grid(True)
plt.show()

Explanation

  • y=['Sales', 'Expenses'] selects two columns.
  • marker='o' displays a circular marker at each data point.
  • plt.legend() displays the labels for both lines.
  • plt.grid(True) displays grid lines.
Practical Example: A company can compare monthly sales and expenses to understand its business performance.

7. Setting Chart Size with figsize

Sometimes the default chart size is too small. Pandas provides the figsize parameter to control the width and height of a chart.

Syntax

DataFrame.plot(figsize=(width, height))

Example

df.plot(
    x='Month',
    y='Sales',
    kind='line',
    figsize=(10, 5)
)

plt.title('Monthly Sales')
plt.show()

In this example, the chart width is 10 inches and the height is 5 inches.

8. Adding Titles and Axis Labels

A chart should have a meaningful title and proper axis labels. These elements help users understand what information the chart represents.

Example

df.plot(
    x='Month',
    y='Sales',
    kind='bar',
    title='Monthly Sales Report'
)

plt.xlabel('Month')
plt.ylabel('Total Sales (Rs.)')
plt.show()

Important Functions

Function Purpose
plt.title() Adds the chart title.
plt.xlabel() Adds the X-axis label.
plt.ylabel() Adds the Y-axis label.
plt.legend() Displays the chart legend.
plt.grid() Displays or hides grid lines.

9. Adding Grid Lines and Legend

Grid lines help users compare values against the chart axes. A legend identifies the different datasets displayed in a chart.

Example

df.plot(
    x='Month',
    y=['Sales', 'Expenses'],
    kind='line',
    marker='o'
)

plt.title('Sales and Expenses')
plt.xlabel('Month')
plt.ylabel('Amount')
plt.grid(True)
plt.legend(title='Category')
plt.show()

The legend title helps identify the different lines, while grid lines make it easier to compare numerical values.

10. Creating Multiple Subplots

Subplots allow us to display multiple charts in a single figure. Each chart is placed in a separate plotting area.

Example: Separate Sales and Expenses Charts

df.plot(
    x='Month',
    y=['Sales', 'Expenses'],
    subplots=True,
    layout=(2, 1),
    figsize=(10, 8),
    kind='line',
    marker='o'
)

plt.suptitle('Monthly Business Report')
plt.tight_layout()
plt.show()

Explanation

  • subplots=True creates separate charts for each selected column.
  • layout=(2, 1) arranges the charts in two rows and one column.
  • figsize=(10, 8) controls the overall figure size.
  • plt.suptitle() adds a common title for the figure.
  • plt.tight_layout() adjusts spacing between chart elements.

11. Saving a Chart as an Image

After creating a chart, we may need to save it for reports, presentations or documentation. Matplotlib provides the savefig() function for this purpose.

Example

df.plot(
    x='Month',
    y='Sales',
    kind='bar',
    figsize=(10, 5)
)

plt.title('Monthly Sales Report')

plt.savefig(
    'monthly_sales.png',
    dpi=300,
    bbox_inches='tight'
)

plt.show()

Important Parameters

  • monthly_sales.png is the output image file name.
  • dpi=300 sets the image resolution.
  • bbox_inches='tight' reduces unnecessary surrounding whitespace.
Note: The image is saved in the current working directory unless you provide a different file path.

12. Plotting a Pandas Series

Pandas plotting is not limited to DataFrames. We can also create charts directly from a Series.

Example

import pandas as pd
import matplotlib.pyplot as plt

sales = pd.Series(
    [12000, 15000, 18000, 14000, 22000],
    index=['Jan', 'Feb', 'Mar', 'Apr', 'May']
)

sales.plot(
    kind='line',
    marker='o',
    color='green',
    title='Sales Trend'
)

plt.xlabel('Month')
plt.ylabel('Sales')
plt.grid(True)
plt.show()

In this example, the Series index is used for the X-axis and the Series values are used for the Y-axis.

13. Complete Practical Example – Sales Analysis

Let us combine the concepts learned in this lesson. We will create a DataFrame and generate a sales comparison chart.

Complete Python Program

import pandas as pd
import matplotlib.pyplot as plt

# Create sample data
data = {
    'Month': ['January', 'February', 'March',
              'April', 'May', 'June'],
    'Sales': [12000, 15000, 18000,
              14000, 22000, 25000],
    'Expenses': [8000, 9000, 11000,
                 8500, 13000, 15000]
}

# Create DataFrame
df = pd.DataFrame(data)

# Display data
print(df)

# Create sales and expenses chart
ax = df.plot(
    x='Month',
    y=['Sales', 'Expenses'],
    kind='bar',
    figsize=(11, 6),
    width=0.8
)

# Add chart details
ax.set_title('Monthly Sales and Expenses Report')
ax.set_xlabel('Month')
ax.set_ylabel('Amount (Rs.)')
ax.legend(title='Business Details')
ax.grid(axis='y', linestyle='--', alpha=0.5)

# Save chart
plt.savefig(
    'sales_expenses_report.png',
    dpi=300,
    bbox_inches='tight'
)

# Display chart
plt.show()

What We Learned from This Example

  • Created a DataFrame using a Python dictionary.
  • Selected multiple numerical columns for plotting.
  • Created a grouped bar chart.
  • Customised chart title and axis labels.
  • Added a legend and grid lines.
  • Saved the chart as a PNG image.

14. Common Pandas Plotting Parameters

Parameter Description Example
kind Type of chart. kind='bar'
x X-axis column. x='Month'
y Y-axis column or columns. y='Sales'
color Chart colour. color='green'
figsize Figure dimensions. figsize=(10, 5)
title Chart title. title='Sales'
grid Displays grid lines. grid=True
legend Displays the legend. legend=True
subplots Creates separate plots. subplots=True
rot Rotates axis tick labels. rot=45

15. Best Practices for Basic Plotting

  • Choose the correct chart type according to the data.
  • Always provide meaningful titles and axis labels.
  • Use readable category names.
  • Choose an appropriate figure size.
  • Use legends when multiple datasets are displayed.
  • Avoid unnecessary visual elements.
  • Use grid lines when they improve readability.
  • Save important charts for reports and documentation.
  • Check missing values and incorrect data before plotting.

Summary

Pandas provides a simple plotting interface through the DataFrame.plot() and Series.plot() methods. These methods make it easy to convert numerical data into meaningful charts.

In this lesson, we learned how to create line plots, vertical bar plots and horizontal bar plots. We also covered plotting multiple columns, setting figure size, adding titles and labels, displaying legends and grid lines, creating subplots and saving charts as image files.

Pandas uses Matplotlib as its default plotting backend. By combining Pandas and Matplotlib, we can create useful visualisations for business reports, student analysis, sales reports and other real-world applications.

In the next lesson, we will explore histograms, line charts, scatter plots and box plots in detail.