Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

DataFrame Dimensions

Learn how to find the number of rows, columns, dimensions, and total elements in a Pandas DataFrame using shape, ndim, size, and other useful functions.

1. Learning Objectives

After completing this lesson, you will be able to:

  • Understand the dimensions of a Pandas DataFrame.
  • Find the number of rows and columns using shape.
  • Understand the ndim attribute.
  • Calculate total elements using size.
  • Count rows using the len() function.
  • Count non-missing values using count().
  • Understand how filtering and adding columns change DataFrame dimensions.

2. Understanding DataFrame Dimensions

A Pandas DataFrame is a two-dimensional data structure organized into rows and columns. Dimensions describe the structure and size of the data.

For example, a student DataFrame containing three students and three columns has three rows and three columns.

DataFrame Dimensions 3 Columns Name Age Marks Samadhan 20 85 Pradnya 21 92 Samiksha 19 88

Figure 1: This DataFrame has 3 rows and 3 columns, so its shape is (3, 3).

Remember: DataFrame shape is represented as (number of rows, number of columns).

3. Creating a Sample DataFrame

First, create a DataFrame that we will use throughout this lesson.

import pandas as pd

data = {
    "Name": ["Samadhan", "Pradnya", "Samiksha"],
    "Age": [20, 21, 19],
    "Marks": [85, 92, 88]
}

df = pd.DataFrame(data)

print(df)

Output:

       Name  Age  Marks
0  Samadhan   20     85
1   Pradnya   21     92
2  Samiksha   19     88

The DataFrame contains three student records and three columns. We can inspect these dimensions using Pandas attributes.

4. Finding Dimensions with shape

The shape attribute returns a tuple containing the number of rows and the number of columns.

print(df.shape)

Output:

(3, 3)

Finding Rows and Columns Separately

print("Rows:", df.shape[0])
print("Columns:", df.shape[1])

Output:

Rows: 3
Columns: 3
Remember: df.shape[0] returns the row count, and df.shape[1] returns the column count. Use df.shape, not df.shape().

5. Understanding ndim

The ndim attribute returns the number of dimensions of a Pandas object.

print(df.ndim)

Output:

2

A DataFrame is two-dimensional because it contains rows and columns. A Series is one-dimensional.

names = df["Name"]

print(names.ndim)

Output:

1
Pandas Object Attribute Dimensions
DataFrame df.ndim 2
Series names.ndim 1

6. Finding Total Elements with size

The size attribute returns the total number of elements in a DataFrame. It includes cells containing missing values.

print(df.size)

Output:

9

Our DataFrame contains three rows and three columns. Therefore, the total number of elements is:

Total elements = Rows × Columns

3 × 3 = 9

print(df.shape[0] * df.shape[1])
print(df.size)

Output:

9
9

7. Counting Rows with len()

The built-in len() function returns the number of rows in a DataFrame.

print(len(df))

Output:

3

Even though the DataFrame contains nine elements, len(df) returns three because there are three rows.

Tip: Use len(df) when you need the number of records in a DataFrame.

8. Counting Non-Missing Values with count()

The count() method counts non-missing values separately for each column by default.

print(df.count())

Output:

Name     3
Age      3
Marks    3
dtype: int64

Working with Missing Values

df_missing = pd.DataFrame({
    "Name": ["Samadhan", "Pradnya", "Samiksha"],
    "Marks": [85, None, 88]
})

print(df_missing.shape)
print(df_missing.size)
print(df_missing.count())

Output:

(3, 2)
6
Name     3
Marks    2
dtype: int64
Important: shape and size include cells containing missing values. count() excludes missing values when counting each column.

9. Comparing Dimension Properties

Property / Function Purpose Example Result
df.shape Returns rows and columns. (3, 3)
df.shape[0] Returns the number of rows. 3
df.shape[1] Returns the number of columns. 3
df.ndim Returns the number of dimensions. 2
df.size Returns total elements. 9
len(df) Returns the number of rows. 3
df.count() Counts non-missing values per column. 3 per column

10. How DataFrame Dimensions Change

The shape of a DataFrame changes when rows or columns are added, removed, or filtered.

Adding a Column

df["City"] = ["Pune", "Mumbai", "Nashik"]

print(df.shape)

Output:

(3, 4)

Adding one column increases the column count from three to four, while the row count remains three.

Filtering Rows

top_students = df[df["Marks"] >= 90]

print(top_students.shape)

Output:

(1, 4)

Only Pradnya has marks of 90 or above. The filtered DataFrame contains one row and four columns.

Removing a Column

df_without_city = df.drop(columns=["City"])

print(df_without_city.shape)

Output:

(3, 3)
Remember: Adding or removing a column changes the column count. Filtering rows can change the row count.

11. Practical Exercise

CIIT TRAINING INSTITUTE: Student Records

Create a DataFrame containing student names and marks for Samadhan, Pradnya, and Samiksha.

Complete these tasks:

  1. Display the shape of the DataFrame.
  2. Print the number of rows and columns separately.
  3. Display the number of dimensions using ndim.
  4. Find the total number of elements using size.
  5. Count the rows using len().
  6. Display the non-missing value count using count().
  7. Add a City column and check the new shape.
  8. Filter students whose marks are at least 90.

Practice Code

import pandas as pd

data = {
    "Name": ["Samadhan", "Pradnya", "Samiksha"],
    "Marks": [85, 92, 88]
}

df = pd.DataFrame(data)

print("Shape:", df.shape)
print("Rows:", df.shape[0])
print("Columns:", df.shape[1])
print("Dimensions:", df.ndim)
print("Total elements:", df.size)
print("Number of rows:", len(df))

print("\nNon-missing values:")
print(df.count())

df["City"] = ["Pune", "Mumbai", "Nashik"]

print("\nUpdated shape:")
print(df.shape)

top_students = df[df["Marks"] >= 90]

print("\nTop students:")
print(top_students)

print("\nFiltered shape:")
print(top_students.shape)

Revision Questions

  1. What does the shape attribute return?
  2. How can you find the number of rows and columns separately?
  3. What is the purpose of ndim?
  4. How is size different from shape?
  5. What does len(df) return?
  6. Does count() include missing values?
  7. How does adding a column affect DataFrame dimensions?

Summary

In this lesson, we learned how to inspect the dimensions of a Pandas DataFrame. The shape attribute returns the number of rows and columns, while ndim returns the number of dimensions.

We used size to find the total number of elements, len() to count rows, and count() to count non-missing values in each column.

Understanding DataFrame dimensions helps us inspect datasets, validate data, and understand how filtering or modifying a DataFrame affects its structure.