Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

Matplotlib and Seaborn

Learn how to create professional data visualisations using Matplotlib and Seaborn with Pandas DataFrames. Explore chart customisation, statistical plots, heatmaps and practical data analysis examples.

1. Introduction to Matplotlib and Seaborn

Matplotlib and Seaborn are two popular Python libraries used for data visualisation. They help convert numerical and categorical data into meaningful charts and graphs.

Matplotlib provides detailed control over chart elements, while Seaborn provides a high-level interface for creating attractive statistical visualisations.

Both libraries work with Pandas DataFrames and Series, making them useful for data analysis and reporting.

Important: Matplotlib is commonly used as the foundation for custom charts, while Seaborn simplifies statistical visualisation and works on top of Matplotlib.

Key Features

  • Create different types of charts and graphs.
  • Customise chart titles, labels, colours and legends.
  • Analyse numerical and categorical datasets.
  • Generate statistical visualisations.
  • Display relationships between multiple variables.
  • Save charts for reports and presentations.

2. Difference Between Matplotlib and Seaborn

Feature Matplotlib Seaborn
Purpose General-purpose plotting. Statistical data visualisation.
Customisation Detailed control over chart elements. Provides convenient visual styles and options.
Syntax More manual configuration may be required. Often requires less code for statistical charts.
Data Support Works with arrays and DataFrames. Works well with Pandas DataFrames.
Statistical Charts Can create statistical charts manually. Provides built-in statistical plotting functions.
Dependency Independent plotting library. Uses Matplotlib for rendering.

3. Installing Matplotlib and Seaborn

Before using these libraries, install them using the Python pip package manager.

Install Matplotlib

pip install matplotlib

Install Seaborn

pip install seaborn

Install Pandas, Matplotlib and Seaborn Together

pip install pandas matplotlib seaborn

Verify Installation

import pandas as pd
import matplotlib
import seaborn as sns

print("Pandas:", pd.__version__)
print("Matplotlib:", matplotlib.__version__)
print("Seaborn:", sns.__version__)
Tip: If you are using Jupyter Notebook, run installation commands in a notebook cell using the appropriate pip command for your Python environment.

4. Importing Required Libraries

After installing the libraries, import them into your Python program.

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

Explanation

  • pandas as pd is used for data manipulation.
  • matplotlib.pyplot as plt is used to create and customise charts.
  • seaborn as sns is used to create statistical visualisations.

5. Creating a Sample Dataset

Let us create a student performance dataset. We will use this DataFrame in the following examples.

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

data = {
    'Student': [
        'Amit', 'Rahul', 'Priya', 'Sneha',
        'Omkar', 'Neha', 'Kiran', 'Pooja',
        'Rohit', 'Anjali'
    ],
    'StudyHours': [2, 3, 4, 5, 6, 7, 3, 8, 5, 9],
    'Marks': [45, 50, 55, 65, 70, 75, 52, 85, 68, 92],
    'Attendance': [65, 70, 75, 80, 85, 90, 72, 95, 82, 98],
    'Department': [
        'Computer', 'IT', 'Computer', 'IT',
        'Computer', 'Mechanical', 'IT',
        'Computer', 'Mechanical', 'Computer'
    ]
}

df = pd.DataFrame(data)

print(df)

Expected Output

  Student  StudyHours  Marks  Attendance  Department
0    Amit           2     45          65    Computer
1   Rahul           3     50          70          IT
2   Priya           4     55          75    Computer
3   Sneha           5     65          80          IT
4   Omkar           6     70          85    Computer
5    Neha           7     75          90  Mechanical
6   Kiran           3     52          72          IT
7   Pooja           8     85          95    Computer
8   Rohit           5     68          82  Mechanical
9  Anjali           9     92          98    Computer

6. Understanding Figure and Axes in Matplotlib

Matplotlib uses two important components when creating charts: Figure and Axes.

Figure

A Figure is the complete window or canvas that contains one or more plots.

Axes

An Axes object represents an individual plotting area inside the Figure. It contains the X-axis, Y-axis, title, labels and plotted data.

Example

fig, ax = plt.subplots(figsize=(8, 5))

ax.plot(
    df['Student'],
    df['Marks'],
    marker='o'
)

ax.set_title('Student Marks')
ax.set_xlabel('Student')
ax.set_ylabel('Marks')

plt.xticks(rotation=45)
plt.tight_layout()
plt.show()

Explanation

  • plt.subplots() creates a Figure and Axes.
  • fig represents the complete Figure.
  • ax represents the plotting area.
  • ax.plot() creates the line chart.
  • ax.set_title() sets the chart title.
  • ax.set_xlabel() and ax.set_ylabel() set axis labels.
Using Figure and Axes objects is a recommended approach when creating multiple charts or customising individual plots.

7. Creating a Line Chart Using Matplotlib

A line chart is used to show trends and changes in numerical values across an ordered sequence.

Example

plt.figure(figsize=(9, 5))

plt.plot(
    df['Student'],
    df['Marks'],
    marker='o',
    linestyle='-',
    linewidth=2,
    color='blue',
    label='Marks'
)

plt.title('Student Marks Analysis')
plt.xlabel('Student')
plt.ylabel('Marks')
plt.xticks(rotation=45)
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

Important Parameters

  • marker specifies the shape of data point markers.
  • linestyle specifies the line style.
  • linewidth controls line thickness.
  • color specifies the line colour.
  • label provides the legend name.

8. Creating a Bar Chart Using Matplotlib

A bar chart compares numerical values across different categories. It is useful for comparing student marks, department performance and sales figures.

Example

plt.figure(figsize=(9, 5))

plt.bar(
    df['Student'],
    df['Marks'],
    color='steelblue',
    edgecolor='black'
)

plt.title('Student Marks Comparison')
plt.xlabel('Student')
plt.ylabel('Marks')
plt.xticks(rotation=45)
plt.grid(axis='y', linestyle='--')
plt.tight_layout()
plt.show()

Horizontal Bar Chart

plt.figure(figsize=(9, 5))

plt.barh(
    df['Student'],
    df['Marks'],
    color='orange'
)

plt.title('Student Marks - Horizontal Bar Chart')
plt.xlabel('Marks')
plt.ylabel('Student')
plt.tight_layout()
plt.show()

9. Introduction to Seaborn

Seaborn is a Python visualisation library built on top of Matplotlib. It provides a high-level interface for creating statistical graphics.

Seaborn works especially well with Pandas DataFrames. It automatically handles many plotting details and provides attractive default chart styles.

Setting a Seaborn Theme

sns.set_theme(style='whitegrid')

Common Seaborn Styles

Style Description
darkgrid Dark background with grid lines.
whitegrid White background with grid lines.
dark Dark background without prominent grid lines.
white Clean white background.
ticks White background with axis tick marks.

10. Creating a Histogram Using Seaborn

Seaborn provides the histplot() function to visualise the distribution of numerical data.

Example: Student Marks Distribution

sns.histplot(
    data=df,
    x='Marks',
    bins=5,
    kde=True,
    color='skyblue'
)

plt.title('Student Marks Distribution')
plt.xlabel('Marks')
plt.ylabel('Frequency')
plt.show()

Explanation

  • data=df specifies the DataFrame.
  • x='Marks' selects the numerical column.
  • bins=5 controls the number of intervals.
  • kde=True adds a Kernel Density Estimate curve.
  • color sets the chart colour.
KDE is a smoothed estimate of the data distribution. It helps understand the overall shape of numerical values.

11. Creating a Scatter Plot Using Seaborn

Seaborn's scatterplot() function displays the relationship between two numerical variables.

Example: Study Hours vs Marks

sns.scatterplot(
    data=df,
    x='StudyHours',
    y='Marks',
    hue='Department',
    s=100
)

plt.title('Study Hours vs Student Marks')
plt.xlabel('Study Hours')
plt.ylabel('Marks')
plt.show()

Explanation

  • x='StudyHours' selects the X-axis variable.
  • y='Marks' selects the Y-axis variable.
  • hue='Department' uses different colours for department categories.
  • s=100 controls marker size.

12. Creating a Box Plot Using Seaborn

A box plot helps analyse the distribution of numerical data. It displays quartiles, median and potential outliers.

Example

sns.boxplot(
    data=df,
    x='Department',
    y='Marks'
)

plt.title('Department-wise Marks Distribution')
plt.xlabel('Department')
plt.ylabel('Marks')
plt.xticks(rotation=20)
plt.tight_layout()
plt.show()

This chart compares student marks across different departments.

A box plot can help identify departments with different score distributions and unusual observations.

13. Creating a Count Plot Using Seaborn

A count plot displays the number of observations in each category. It is useful for analysing categorical columns.

Example: Department-wise Student Count

sns.countplot(
    data=df,
    x='Department'
)

plt.title('Students by Department')
plt.xlabel('Department')
plt.ylabel('Student Count')
plt.xticks(rotation=20)
plt.tight_layout()
plt.show()

Horizontal Count Plot

sns.countplot(
    data=df,
    y='Department'
)

plt.title('Department-wise Student Count')
plt.xlabel('Student Count')
plt.ylabel('Department')
plt.show()

14. Creating a Correlation Heatmap

A heatmap represents numerical values using different colours. It is commonly used to visualise correlation between numerical columns.

Step 1: Calculate Correlation

correlation = df[
    ['StudyHours', 'Marks', 'Attendance']
].corr()

print(correlation)

Step 2: Create Heatmap

plt.figure(figsize=(8, 6))

sns.heatmap(
    correlation,
    annot=True,
    cmap='coolwarm',
    fmt='.2f',
    linewidths=0.5
)

plt.title('Student Performance Correlation')
plt.show()

Important Parameters

  • annot=True displays numerical values inside cells.
  • cmap selects the colour map.
  • fmt='.2f' formats values to two decimal places.
  • linewidths controls the spacing between cells.

Understanding Correlation

Correlation Value Interpretation
Close to +1 Strong positive linear relationship.
Close to -1 Strong negative linear relationship.
Close to 0 Weak or no linear relationship.
Important: Correlation describes association, not causation. A high correlation does not prove that one variable causes the other to change.

15. Creating a Pair Plot

A pair plot displays relationships between multiple numerical variables in a dataset. It creates scatter plots for pairs of variables and distribution plots along the diagonal.

Example

sns.pairplot(
    df,
    vars=['StudyHours', 'Marks', 'Attendance'],
    hue='Department'
)

plt.show()

Explanation

  • df supplies the dataset.
  • vars selects numerical columns.
  • hue='Department' differentiates categories using colours.
  • sns.pairplot() generates multiple related plots.
Pair plots are useful during exploratory data analysis when we want to understand relationships between several numerical variables at once.

16. Creating Multiple Charts Using Subplots

Matplotlib allows us to create multiple charts inside a single figure. This is useful for comparing different visualisations in one report.

Example

fig, axes = plt.subplots(
    2,
    2,
    figsize=(12, 9)
)

# Line chart
axes[0, 0].plot(
    df['Student'],
    df['Marks'],
    marker='o'
)
axes[0, 0].set_title('Student Marks')

# Bar chart
axes[0, 1].bar(
    df['Student'],
    df['Marks']
)
axes[0, 1].set_title('Marks Comparison')

# Histogram
axes[1, 0].hist(
    df['Marks'],
    bins=5,
    edgecolor='black'
)
axes[1, 0].set_title('Marks Distribution')

# Scatter plot
axes[1, 1].scatter(
    df['StudyHours'],
    df['Marks']
)
axes[1, 1].set_title('Study Hours vs Marks')

plt.tight_layout()
plt.show()

Explanation

  • plt.subplots(2, 2) creates two rows and two columns of charts.
  • axes[0, 0] accesses the first chart.
  • axes[0, 1] accesses the second chart.
  • axes[1, 0] accesses the third chart.
  • axes[1, 1] accesses the fourth chart.
  • plt.tight_layout() adjusts spacing between charts.

17. Saving Visualisations

After creating charts, we can save them as image files for reports, presentations and documentation.

Save a Matplotlib Chart

plt.figure(figsize=(8, 5))

plt.plot(
    df['Student'],
    df['Marks'],
    marker='o'
)

plt.title('Student Marks Report')
plt.xticks(rotation=45)

plt.tight_layout()

plt.savefig(
    'student_marks.png',
    dpi=300,
    bbox_inches='tight'
)

plt.show()

Save a Seaborn Chart

plt.figure(figsize=(8, 5))

sns.boxplot(
    data=df,
    x='Department',
    y='Marks'
)

plt.title('Department-wise Marks')

plt.tight_layout()

plt.savefig(
    'department_marks.png',
    dpi=300,
    bbox_inches='tight'
)

plt.show()

The savefig() method saves the current Matplotlib figure. Since Seaborn uses Matplotlib for rendering, the same method can also save Seaborn charts.

18. Complete Practical Example – Student Data Visualisation

Let us combine Pandas, Matplotlib and Seaborn in one practical example. We will analyse student marks, study hours, attendance and department information.

Complete Python Program

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Configure Seaborn
sns.set_theme(style='whitegrid')

# Create dataset
data = {
    'Student': [
        'Amit', 'Rahul', 'Priya', 'Sneha',
        'Omkar', 'Neha', 'Kiran', 'Pooja',
        'Rohit', 'Anjali'
    ],
    'StudyHours': [2, 3, 4, 5, 6, 7, 3, 8, 5, 9],
    'Marks': [45, 50, 55, 65, 70, 75, 52, 85, 68, 92],
    'Attendance': [65, 70, 75, 80, 85, 90, 72, 95, 82, 98],
    'Department': [
        'Computer', 'IT', 'Computer', 'IT',
        'Computer', 'Mechanical', 'IT',
        'Computer', 'Mechanical', 'Computer'
    ]
}

# Create DataFrame
df = pd.DataFrame(data)

print(df)

# 1. Histogram
plt.figure(figsize=(8, 5))

sns.histplot(
    data=df,
    x='Marks',
    bins=5,
    kde=True
)

plt.title('Student Marks Distribution')
plt.show()

# 2. Scatter Plot
plt.figure(figsize=(8, 5))

sns.scatterplot(
    data=df,
    x='StudyHours',
    y='Marks',
    hue='Department',
    s=100
)

plt.title('Study Hours vs Marks')
plt.show()

# 3. Box Plot
plt.figure(figsize=(8, 5))

sns.boxplot(
    data=df,
    x='Department',
    y='Marks'
)

plt.title('Department-wise Marks')
plt.xticks(rotation=20)
plt.tight_layout()
plt.show()

# 4. Correlation Heatmap
correlation = df[
    ['StudyHours', 'Marks', 'Attendance']
].corr()

plt.figure(figsize=(8, 6))

sns.heatmap(
    correlation,
    annot=True,
    cmap='coolwarm',
    fmt='.2f'
)

plt.title('Student Data Correlation')
plt.show()

# 5. Department Count Plot
plt.figure(figsize=(8, 5))

sns.countplot(
    data=df,
    x='Department'
)

plt.title('Department-wise Student Count')
plt.xticks(rotation=20)
plt.tight_layout()
plt.show()

# Save the final dataset
df.to_csv('student_visualization_data.csv', index=False)

print("Data visualization completed successfully!")

What This Program Does

  • Creates a student performance DataFrame.
  • Displays marks distribution using a histogram.
  • Analyses study hours and marks using a scatter plot.
  • Compares department-wise marks using a box plot.
  • Displays numerical relationships using a correlation heatmap.
  • Counts students in each department.
  • Exports the dataset into a CSV file.

19. Best Practices for Matplotlib and Seaborn

  • Choose the correct chart based on the analysis requirement.
  • Use clear chart titles and axis labels.
  • Keep chart colours consistent and meaningful.
  • Use legends when multiple categories are displayed.
  • Choose appropriate figure dimensions.
  • Use tight_layout() to avoid overlapping labels.
  • Check missing values before plotting.
  • Use heatmaps to explore numerical correlations.
  • Use box plots to investigate data spread and potential outliers.
  • Save important charts with suitable image resolution.
  • Avoid unnecessary chart elements and misleading scales.

Summary

Matplotlib and Seaborn are important Python libraries used for creating data visualisations. Matplotlib provides detailed control over charts, while Seaborn simplifies statistical visualisation and works with Pandas DataFrames.

In this lesson, we learned how to install and import both libraries, understand Figure and Axes, create line and bar charts, customise chart elements, and work with Seaborn histograms, scatter plots, box plots and count plots.

We also explored correlation heatmaps, pair plots, multiple subplots and saving charts as image files. Finally, we combined Pandas, Matplotlib and Seaborn in a complete student performance analysis example.

These libraries are widely used in exploratory data analysis, business reporting, scientific research and machine learning projects to understand and communicate information effectively.