Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

Pandas Installation and Environment Setup

Set up Python, install Pandas, configure your development environment, and run your first Pandas program.

1. Learning Objectives

After completing this lesson, you will be able to:

  • Check whether Python is installed.
  • Check the installed Python and pip versions.
  • Install Pandas using pip.
  • Understand how a Python virtual environment works.
  • Prepare VS Code or Jupyter Notebook for Pandas.
  • Import Pandas and run a basic DataFrame program.
  • Recognize and troubleshoot common installation errors.

2. Requirements for Pandas

Before using Pandas at CIIT Training Institute, make sure Python is installed and that you can run Python commands in your terminal or development environment.

Requirement Purpose
Python Runs Python programs and libraries.
pip Installs Python packages.
Pandas Provides data analysis functionality.
VS Code or Jupyter Lets you write and execute Python code.
Important: Install Pandas into the same Python environment that your editor or notebook uses. If the environments differ, Python may report that Pandas is missing.

3. Check Python Installation

Open Command Prompt or the VS Code terminal on Windows. Run the following command to check your Python installation.

python --version

On some Windows installations, this command may be used instead:

py --version

Example output:

Python 3.12.0

Your version number may differ. The output above is only an example; it is not a claim about your computer's installed version.

Check pip

python -m pip --version

Example output:

pip 24.0 from ... (python 3.12)

The exact path and version depend on your Python installation.

If Python is not recognized: Python may not be installed, or its executable may not be available on PATH. Check your installation and reopen the terminal before trying again.

4. Install Pandas Using pip

Once Python is available, install Pandas using Python's package installer. Run this command in the terminal, not inside a normal Python script.

python -m pip install pandas

If your Windows setup uses the py launcher, you can also try:

py -m pip install pandas

pip downloads Pandas and the required dependencies for the selected Python environment.

Verify the Installation

python -c "import pandas as pd; print(pd.__version__)"

Example output:

2.2.3

The version shown is an example only. Your installed version may be different.

Remember: Use python -m pip instead of just pip when you want to make it clearer which Python installation is receiving the package.

5. Create a Virtual Environment

A virtual environment keeps the packages for one project separate from packages used by other projects. This is helpful when students at CIIT Training Institute work on multiple Python projects with different requirements.

Step 1: Create a Project Folder

mkdir CIITPandasPractice
cd CIITPandasPractice

Step 2: Create the Environment

python -m venv .venv

Step 3: Activate It on Windows

.venv\Scripts\activate

After activation, the terminal prompt often shows (.venv).

Step 4: Install Pandas Inside the Environment

python -m pip install pandas

Step 5: Verify the Environment

python -c "import pandas as pd; print(pd.__version__)"
Create project folder Create .venv Activate environment Install and verify Pandas

Virtual environment workflow: create, activate, install, and verify.

6. Set Up Pandas in VS Code

Visual Studio Code is a code editor that can run Python programs when the Python extension and a suitable Python interpreter are configured.

Setup Steps

  1. Install Python if it is not already installed.
  2. Open VS Code.
  3. Install the Python extension from Microsoft.
  4. Open your CIITPandasPractice folder.
  5. Create a file named first_pandas.py.
  6. Choose the correct Python interpreter.
  7. If using a virtual environment, select its Python interpreter.
  8. Write and run the example program.

First VS Code Program

import pandas as pd

students = pd.DataFrame({
    "Name": ["Samadhan", "Pradnya", "Samiksha"],
    "Course": ["Python", "Pandas", "Data Analysis"]
})

print(students)

Expected Output:

       Name         Course
0  Samadhan         Python
1   Pradnya         Pandas
2  Samiksha  Data Analysis

The example uses student names for a hypothetical classroom dataset. The course values are sample data.

7. Set Up Pandas in Jupyter Notebook

Jupyter Notebook allows you to run code in individual cells and see results immediately. It is useful for experimenting with datasets and explaining results during classroom demonstrations.

Install Jupyter

python -m pip install notebook

Start Jupyter Notebook

jupyter notebook

Your browser should open the Jupyter interface. If it does not, check the terminal output for the local address supplied by Jupyter.

Run a Notebook Cell

import pandas as pd

marks = pd.DataFrame({
    "Student": ["Samadhan", "Pradnya", "Samiksha"],
    "Marks": [78, 91, 85]
})

marks

In a notebook, putting the DataFrame variable at the end of a cell displays its table representation.

Tip: VS Code can also work with Jupyter notebooks when the appropriate Python and Jupyter extensions are installed.

8. Verify Your Python Environment

Use this short checklist whenever Pandas fails to import.

Check Command or Action Expected Result
Python python --version Python version is displayed.
pip python -m pip --version pip version and location are displayed.
Pandas python -c "import pandas as pd; print(pd.__version__)" Pandas version is displayed.
Editor interpreter Check the selected interpreter in VS Code. It matches the environment where Pandas is installed.

9. Common Installation Problems

Problem 1: No Module Named pandas

Possible cause: Pandas is not installed in the active Python environment.

python -m pip install pandas

Also verify that VS Code or Jupyter is using the environment in which you installed Pandas.

Problem 2: Python Is Not Recognized

Check whether Python is installed and available on PATH. On Windows, try py --version as well.

Problem 3: Permission Error

A virtual environment is often a good solution because it keeps project packages separate from system-wide packages.

Problem 4: Installation Seems Successful but Import Fails

Check the Python executable and pip location. They may refer to different installations. Re-select the correct interpreter and repeat the verification command.

10. Practice Exercise

CIIT Training Institute: Student Records

Create a DataFrame containing the names Samadhan, Pradnya, and Samiksha. Add a Marks column with any three sample scores.

Complete these tasks:

  1. Print the DataFrame.
  2. Display the Pandas version.
  3. Calculate the average marks.
  4. Display the student with the highest marks.
  5. Run the same program in your selected environment.

Starter Code

import pandas as pd

students = pd.DataFrame({
    "Name": ["Samadhan", "Pradnya", "Samiksha"],
    "Marks": [80, 92, 86]
})

print("Pandas version:", pd.__version__)
print(students)
print("Average marks:", students["Marks"].mean())

top_student = students.loc[students["Marks"].idxmax()]
print("Top student:", top_student["Name"])

Expected Output:

Pandas version: [your installed version]
       Name  Marks
0  Samadhan     80
1   Pradnya     92
2  Samiksha     86

Average marks: 86.0
Top student: Pradnya

The version placeholder will be replaced by the version installed on your computer. The sample marks produce the displayed average and top student.

Summary

You learned how to check Python and pip, install Pandas, create and activate a virtual environment, configure VS Code or Jupyter Notebook, and verify that Pandas is working.

You also created a DataFrame with student names and practiced basic analysis. You are now ready to explore Pandas Series, DataFrames, indexes, rows, columns, and dimensions in detail.