Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

Selection and Indexing in Pandas

Learn how to select individual columns, access specific rows, retrieve cells, and manage DataFrame indexes using Pandas.

1. What Is Selection and Indexing?

Pandas DataFrames contain rows and columns. In practical projects, we rarely need the entire dataset every time. Sometimes we need only one column, sometimes a few rows, and sometimes one particular cell.

Selection means choosing the data we need. Indexing means accessing data using labels or positions. Pandas provides tools such as column names, loc[], iloc[], and index-management methods.

Example: Student Marks Dataset

import pandas as pd

df = pd.DataFrame({
    "Name": ["Samadhan", "Pradnya", "Samiksha", "Amit"],
    "Age": [21, 22, 20, 23],
    "City": ["Pune", "Mumbai", "Pune", "Nashik"],
    "Marks": [85, 92, 78, 88]
})

print(df)

Expected output:

Index Name Age City Marks
0Samadhan21Pune85
1Pradnya22Mumbai92
2Samiksha20Pune78
3Amit23Nashik88

2. Understand the DataFrame Structure

A DataFrame has row labels (index) and column labels. You can use these labels or numerical positions to locate data.

Index Name Age City Marks 0 Samadhan 21 Pune 85 1 Pradnya 22 Mumbai 92 2 Samiksha 20 Pune 78 Row labels Column labels Cell values

Figure 1: A DataFrame uses row indexes and column labels to organize data.

3. Topics Covered in This Module

Study these three topics in sequence. Each topic has its own detailed lesson and executable examples.

4. Basic Selection Examples

Example 1: Select One Column

print(df["Name"])

Expected output:

0    Samadhan
1     Pradnya
2    Samiksha
3        Amit
Name: Name, dtype: object

Selecting one column with a single pair of brackets returns a Pandas Series.

Example 2: Select Multiple Columns

print(df[["Name", "Marks"]])

Expected output:

       Name  Marks
0  Samadhan     85
1   Pradnya     92
2  Samiksha     78
3      Amit     88

Selecting multiple columns requires a list of column names inside the outer brackets. The result is a DataFrame.

Example 3: Select a Row by Position

print(df.iloc[1])

Expected output:

Name     Pradnya
Age           22
City      Mumbai
Marks         92
Name: 1, dtype: object
Remember: df["Name"] selects a column, while df.iloc[1] selects the second row by position.

5. loc[] vs iloc[] at a Glance

The most important difference is how each method identifies a row or column. loc[] uses labels; iloc[] uses integer positions.

loc[] Uses index labels Label = "student_2" Select by label iloc[] Uses integer positions Position = 1 Select by position df.loc["student_2"] df.iloc[1]

Figure 2: Use labels with loc[] and zero-based positions with iloc[].

Important: iloc[1] means the second row because Python positions start at zero. But loc[1] looks for the row whose index label is 1. These can refer to different rows when custom index labels are used.

6. Quick Reference Table

Code Purpose Typical result
df["Name"] Select one column Series
df[["Name", "Marks"]] Select multiple columns DataFrame
df.loc[1] Select row with index label 1 Series for a single row
df.iloc[1] Select row at position 1 Series for a single row
df.loc[1, "Marks"] Select a cell using row label and column label Scalar value
df.iloc[1, 3] Select a cell using row and column positions Scalar value
df.set_index("Name") Set Name as the index DataFrame
df.reset_index() Restore the default integer index DataFrame

7. Practice Exercises

Exercise 1: Select Columns

  1. Display only the Name column.
  2. Display the Name and City columns.
  3. Display the Age and Marks columns.

Exercise 2: Select Rows and Cells

  1. Select the first row using iloc[].
  2. Select the row at position 2.
  3. Retrieve Samadhan's marks using the default index.
  4. Retrieve Pradnya's city using row and column labels.

Hints:

df.iloc[0]
df.iloc[2]
df.loc[0, "Marks"]
df.loc[1, "City"]

Exercise 3: Manage the Index

  1. Set Name as the index.
  2. Select the row labelled Samadhan.
  3. Reset the index and inspect the result.
indexed_df = df.set_index("Name")
print(indexed_df.loc["Samadhan"])

restored_df = indexed_df.reset_index()
print(restored_df)

8. Revision Questions

  1. What is the difference between a Series and a DataFrame?
  2. How do you select two columns from a DataFrame?
  3. What is the difference between loc[] and iloc[]?
  4. Why does iloc[0] select the first row?
  5. What does set_index() do?
  6. When would you use reset_index()?

Summary

Selection and indexing help you retrieve the required data from a Pandas DataFrame. Column names are used to select columns, loc[] accesses data by labels, and iloc[] accesses data by integer positions.

Index-management methods help organize rows using meaningful labels. Practise these methods before moving on to more advanced DataFrame operations.