Data Inspection in Pandas
Learn how to explore a dataset, inspect rows and columns, understand data types, identify missing values, and summarize data.
1. What Is Data Inspection?
Data inspection is the process of examining a dataset before performing analysis, visualization, or machine learning. It helps us understand what information is available and whether the data is suitable for our task.
For example, suppose Samadhan receives a CSV file containing student names, marks, ages, and cities. Before calculating results, he should check the records, column names, data types, and missing values.
Why do we inspect data?
- Understand the size and structure of the dataset.
- Preview records without printing the entire dataset.
- Check whether columns contain the expected data types.
- Find missing or incomplete values.
- Understand numerical values through statistical summaries.
- Identify potential data-quality problems before analysis.
head(), shape, info(), and
isna().sum(). These give you a useful first overview.
2. Create Our Sample Dataset
We will use one DataFrame throughout this lesson so that every inspection method works on the same data.
import pandas as pd
data = {
"Name": ["Samadhan", "Pradnya", "Samiksha", "Aarav", "Neha"],
"Marks": [85, 92, 88, 76, 95],
"Age": [21, 22, 20, 23, 21],
"City": ["Pune", "Mumbai", "Nashik", "Pune", "Mumbai"]
}
df = pd.DataFrame(data)
print(df)
Output
Name Marks Age City 0 Samadhan 85 21 Pune 1 Pradnya 92 22 Mumbai 2 Samiksha 88 20 Nashik 3 Aarav 76 23 Pune 4 Neha 95 21 Mumbai
Our dataset contains five student records and four columns.
| Column | Meaning | Expected data |
|---|---|---|
Name |
Student name | Text |
Marks |
Student marks | Numeric |
Age |
Student age | Numeric |
City |
Student city | Text |
3. Visual Guide: Understanding a DataFrame
A DataFrame is a two-dimensional table with rows and columns. Each row represents a record, while each column represents a variable or field.
Image 1: A DataFrame consists of row indexes, column labels, and the values stored in each cell.
4. Preview Rows with head() and tail()
When a dataset contains thousands or millions of records, printing the entire DataFrame is usually unnecessary. Pandas lets you preview selected rows.
4.1 The head() method
head() displays the first five rows by default.
You can provide a number to control how many rows are returned.
print(df.head()) print(df.head(3))
Output of df.head(3)
Name Marks Age City 0 Samadhan 85 21 Pune 1 Pradnya 92 22 Mumbai 2 Samiksha 88 20 Nashik
4.2 The tail() method
tail() displays the last five rows by default.
It is useful for checking the end of a file or dataset.
print(df.tail(2))
Output
Name Marks Age City 3 Aarav 76 23 Pune 4 Neha 95 21 Mumbai
Learn these methods in detail on the dedicated Head and Tail lesson .
5. Understand DataFrame Shape and Columns
5.1 The shape attribute
The shape attribute returns a tuple containing the
number of rows followed by the number of columns.
print(df.shape)
Output
(5, 4)
This means the DataFrame has five rows and four columns.
Notice that shape is an attribute, so we use
df.shape, not df.shape().
5.2 Count rows and columns separately
rows = df.shape[0]
columns = df.shape[1]
print("Rows:", rows)
print("Columns:", columns)
Output
Rows: 5 Columns: 4
5.3 Inspect column names
print(df.columns)
This returns the column labels: Name,
Marks, Age, and City.
5.4 Check the index
print(df.index)
The default index is a range of labels starting at 0 and ending before 5. The exact representation may vary by Pandas version.
6. Inspect Data Types with dtypes
A DataFrame can contain different kinds of information. For example, student names are text, whereas marks and ages are numeric values.
print(df.dtypes)
Typical output
Name object Marks int64 Age int64 City object dtype: object
The exact dtype names can differ depending on the data and
Pandas version. In this example, Name and
City are commonly represented as object,
while Marks and Age are integers.
| Data type | Meaning | Typical example |
|---|---|---|
int64 |
Integer values | 85, 92, 76 |
float64 |
Decimal or floating-point values | 85.5, 92.0 |
object |
Often used for text or mixed Python objects | "Pune", "Mumbai" |
bool |
Boolean values | True, False |
datetime64 |
Date and time values | 2026-01-15 |
7. Get Complete Structural Information with info()
The info() method provides a compact summary of
the DataFrame. It reports the index range, number of columns,
column names, non-null counts, data types, and memory usage.
df.info()
Example output
<class 'pandas.core.frame.DataFrame'> RangeIndex: 5 entries, 0 to 4 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Name 5 non-null object 1 Marks 5 non-null int64 2 Age 5 non-null int64 3 City 5 non-null object dtypes: int64(2), object(2)
This is a simplified illustration of the output. The actual display may include different spacing, memory information, and additional details depending on the installed version.
What should you check?
- Entries: How many rows are present?
- Columns: Are the expected fields available?
- Non-Null Count: Are any values missing?
- Dtype: Are columns stored using appropriate types?
- Memory Usage: How much memory does the DataFrame use?
Read the Data Information and Types lesson for more examples.
8. Visual Guide: The Data Inspection Workflow
A useful inspection workflow moves from a quick preview to structural checks, missing-value checks, and statistical summaries. The workflow helps you decide what to investigate next.
Image 2: Follow a consistent inspection sequence before analyzing or transforming your dataset.
9. Check Missing Values
Missing values are common in real-world datasets. A missing mark, age, or city can affect calculations and reports.
Pandas provides isna() to identify missing values
and sum() to count them.
data = {
"Name": ["Samadhan", "Pradnya", "Samiksha", "Aarav"],
"Marks": [85, None, 88, 76],
"City": ["Pune", "Mumbai", None, "Pune"]
}
df_missing = pd.DataFrame(data)
print(df_missing.isna())
print(df_missing.isna().sum())
Output of df_missing.isna().sum()
Name 0 Marks 1 City 1 dtype: int64
This output indicates that the Marks column has
one missing value and the City column has one
missing value.
Continue with the Missing Values lesson to learn about handling incomplete data.
10. Descriptive Statistics with describe()
The describe() method calculates useful summary
statistics for numerical columns by default.
print(df.describe())
Example output
Marks Age count 5.000000 5.000000 mean 87.200000 21.400000 std 7.395945 1.140175 min 76.000000 20.000000 25% 85.000000 21.000000 50% 88.000000 21.000000 75% 92.000000 22.000000 max 95.000000 23.000000
The values above are illustrative of the sample dataset. Depending on formatting and data types, Pandas may display slightly different decimal precision.
| Statistic | Meaning |
|---|---|
count |
Number of non-missing values. |
mean |
Arithmetic average. |
std |
Standard deviation, a measure of spread. |
min |
Smallest observed value. |
25% |
First quartile. |
50% |
Median, or second quartile. |
75% |
Third quartile. |
max |
Largest observed value. |
You can learn the details in the Descriptive Statistics lesson .
11. Inspect Text and Numerical Columns Separately
Sometimes you want to inspect only numerical columns, or you want to summarize text columns such as student names and cities.
11.1 Numerical columns
print(df.select_dtypes(include="number").describe())
This selects columns with numerical data types before generating descriptive statistics.
11.2 Text columns
print(df.describe(include="object"))
For text columns, the summary can include the number of non-missing values, unique values, the most frequent value, and its frequency.
include="all" when you want Pandas to attempt a
summary across all column types.
print(df.describe(include="all"))
12. Complete Data Inspection Example
The following example combines the most useful inspection statements into one simple workflow.
import pandas as pd
df = pd.DataFrame({
"Name": ["Samadhan", "Pradnya", "Samiksha", "Aarav", "Neha"],
"Marks": [85, 92, 88, 76, 95],
"Age": [21, 22, 20, 23, 21],
"City": ["Pune", "Mumbai", "Nashik", "Pune", "Mumbai"]
})
# 1. Preview records
print(df.head())
# 2. Check dataset dimensions
print("Shape:", df.shape)
# 3. Check column names
print("Columns:", df.columns.tolist())
# 4. Check data types
print(df.dtypes)
# 5. Inspect structure
df.info()
# 6. Count missing values
print(df.isna().sum())
# 7. Generate statistics
print(df.describe())
13. Common Mistakes to Avoid
Mistake 1: Calling attributes as methods
# Incorrect df.shape() # Correct df.shape
Mistake 2: Confusing missing values with zero
A missing value does not mean the value is zero. Inspect
missing data explicitly with df.isna().sum().
Mistake 3: Assuming all numeric-looking columns are numeric
Values imported from files may be stored as text. Check
df.dtypes and investigate unexpected types.
Mistake 4: Printing a huge dataset
Use head(), tail(), or select
specific columns instead of printing every record.
Mistake 5: Treating inspection as data cleaning
Inspection helps reveal problems; it does not automatically fix incorrect values or remove duplicates.
14. Hands-on Practice Exercise
Samiksha is working with a student dataset. Complete the following tasks using Pandas.
import pandas as pd
df = pd.DataFrame({
"Name": ["Samadhan", "Pradnya", "Samiksha", "Aarav"],
"Marks": [85, None, 88, 76],
"Age": [21, 22, 20, 23],
"City": ["Pune", "Mumbai", None, "Pune"]
})
Tasks
- Display the first two records.
- Display the last two records.
- Print the number of rows and columns.
- Display all column names.
- Inspect the data types and structure.
- Count the missing values in each column.
- Calculate descriptive statistics for numerical columns.
Suggested solution
print(df.head(2))
print(df.tail(2))
print("Shape:", df.shape)
print("Columns:", df.columns.tolist())
print(df.dtypes)
df.info()
print(df.isna().sum())
print(df.describe())
15. Revision Questions
- What is data inspection, and why is it useful?
- What is the difference between
head()andtail()? - What does
df.shapereturn? - How do you display all column names?
- What information does
df.dtypesprovide? - What is the purpose of
df.info()? - How can you count missing values in every column?
- What is the purpose of
df.describe()? - How can you inspect numerical columns separately?
- Does data inspection automatically modify the dataset?
Lesson Summary
Data inspection is an essential first step in a Pandas workflow. It helps you understand the contents, structure, types, and quality of a dataset before analysis.
Use head() and tail() to preview rows,
shape to check dimensions, columns to
inspect labels, dtypes and info() to
understand structure, isna().sum() to count missing
values, and describe() to summarize data.
After completing this overview, study each topic in detail: Head and Tail, Data Information and Types, and Descriptive Statistics.