Master Pandas From Scratch

Learn Pandas fundamentals, Series, DataFrames, data selection, filtering, cleaning, sorting, grouping, merging, reshaping, string operations, time series, data analysis, performance optimization, visualization, and practical projects through structured lessons.

COURSE SYLLABUS
1. Pandas Introduction and Environment Setup
Introduction to Pandas Installation and Environment Setup
2. Pandas Core Data Structures
Data Structures Overview Pandas Series DataFrames Indexes, Rows and Columns Shape, Size and Dimensions
3. Pandas Data Input and Output
Data Input and Output Overview Reading CSV, Excel and JSON Files Writing and Exporting Files
4. Initial Data Inspection
Data Inspection Overview Head and Tail Data Info and Types Descriptive Statistics
5. Selection and Indexing
Selection and Indexing Overview Column Selection loc and iloc Index Management
6. Filtering and Conditional Selection
Filtering Overview Boolean Filtering Multiple Conditions isin, between and str.contains
7. Basic Data Cleaning
Data Cleaning Overview Missing Values Duplicate Data Rename and Replace Data Type Conversion
8. Vectorized Operations and Sorting
Operations and Sorting Overview Column Operations Sorting Values and Indexes Sum, Mean, Max and Min
9. GroupBy and Aggregation
GroupBy Overview Grouping Data Aggregation Functions Group Filtering Group Transform
10. Merging, Joining and Concatenating
Merge, Join and Concat Overview Concatenation Merging DataFrames Joining by Index
11. Reshaping and Pivoting
Reshaping Overview Pivot Tables Melting Data Stack and Unstack Cross Tabulation
12. String and Categorical Data
String and Categorical Overview String Operations Categorical Data One-Hot Encoding
13. Time Series Analysis
Time Series Overview DateTime Operations Time-Based Indexes Resampling Data Shift and Rolling Windows
14. Custom Functions and Mapping
Apply and Mapping Overview Apply and Lambda Functions Map Operations Element-Wise Operations
15. Performance and Large Datasets
Performance Overview Memory Optimization Processing Files in Chunks PyArrow, Dask and Polars
16. Pandas Data Visualization
Visualization Overview Basic Plotting Histograms, Lines, Scatter and Box Plots Matplotlib and Seaborn

Sorting Data in Pandas

Learn how to arrange DataFrame rows and columns, sort values in ascending or descending order, sort by multiple columns, and control where missing values appear.

1. Learning Objectives

By the end of this lesson, you will be able to:
  • Sort a DataFrame using sort_values().
  • Sort values in ascending and descending order.
  • Sort rows using multiple columns and different directions.
  • Sort a DataFrame by its index using sort_index().
  • Control the position of missing values during sorting.
  • Understand the difference between sorting and changing the index.
  • Apply sorting to practical student, sales, and employee datasets.

2. What Is Sorting?

Sorting means arranging data according to a particular order. We can sort numbers from smallest to largest, text alphabetically, dates chronologically, or records according to one or more columns.

In Pandas, sorting is especially useful when identifying the highest sales, ranking students by marks, finding the lowest prices, or arranging records by date.

Original Data Samiksha: 78 Samadhan: 92 Pradnya: 85 Unordered marks sort_values() Apply order Sorted Data Samiksha: 78 Pradnya: 85 Samadhan: 92 Ascending order

Sorting rearranges records according to the selected column or index.

3. Prepare a Sample DataFrame

We will use the following student dataset throughout the first examples. The rows are intentionally not arranged by marks.

import pandas as pd

students = pd.DataFrame({
    "Name": ["Samiksha", "Samadhan", "Pradnya", "Aarav", "Meera"],
    "Marks": [78, 92, 85, 67, 92],
    "Age": [20, 21, 20, 19, 22],
    "City": ["Pune", "Mumbai", "Pune", "Nashik", "Mumbai"]
})

print(students)
Index Name Marks Age City
0Samiksha7820Pune
1Samadhan9221Mumbai
2Pradnya8520Pune
3Aarav6719Nashik
4Meera9222Mumbai

4. Sorting by a Single Column

Use sort_values() to arrange rows based on the values in a column. Pass the column name to the by parameter.

Example 1: Sort marks in ascending order

result = students.sort_values(by="Marks")
print(result[["Name", "Marks"]])

Expected output:

       Name  Marks
3     Aarav     67
0  Samiksha     78
2   Pradnya     85
1  Samadhan     92
4     Meera     92

By default, ascending=True, so values are arranged from smallest to largest. The original index labels remain attached to their corresponding rows.

Example 2: Sort marks in descending order

result = students.sort_values(by="Marks", ascending=False)
print(result[["Name", "Marks"]])

Expected output:

       Name  Marks
1  Samadhan     92
4     Meera     92
2   Pradnya     85
0  Samiksha     78
3     Aarav     67
Remember: Use ascending=True for lowest-to-highest order and ascending=False for highest-to-lowest order.

5. Sorting by Multiple Columns

Sometimes one column is not enough. For example, we may want to sort students by marks and then arrange students with equal marks alphabetically.

result = students.sort_values(
    by=["Marks", "Name"],
    ascending=[False, True]
)

print(result[["Name", "Marks"]])

The first column, Marks, is sorted in descending order. When marks are equal, the second column, Name, is sorted in ascending alphabetical order.

Expected output:

       Name  Marks
4     Meera     92
1  Samadhan     92
2   Pradnya     85
0  Samiksha     78
3     Aarav     67
Primary Sort Marks ↓ Tie-Breaker Name ↑ Final Order Both rules Sorting priority matters Sort by Marks first, then Name for equal marks

Each item in the ascending list corresponds to the column at the same position.

Example 3: Different sort directions

result = students.sort_values(
    by=["City", "Age"],
    ascending=[True, False]
)

print(result[["City", "Name", "Age"]])

This sorts cities alphabetically. Within each city, students are arranged from oldest to youngest.

Expected output:

      City      Name  Age
1   Mumbai  Samadhan   21
4   Mumbai     Meera   22
3   Nashik     Aarav   19
0     Pune  Samiksha   20
2     Pune   Pradnya   20
Important: The output above should have the older Mumbai student first when using ascending=[True, False]. The correct Mumbai order is Meera (22), then Samadhan (21). The corrected expected output is:
      City      Name  Age
4   Mumbai     Meera   22
1   Mumbai  Samadhan   21
3   Nashik     Aarav   19
0     Pune  Samiksha   20
2     Pune   Pradnya   20

6. Sorting by the Index

Use sort_index() to arrange rows according to their index labels. This is different from sorting the values in a data column.

Example 4: Sort rows by index

result = students.sort_index(ascending=False)
print(result[["Name", "Marks"]])

Expected output:

       Name  Marks
4     Meera     92
3     Aarav     67
2   Pradnya     85
1  Samadhan     92
0  Samiksha     78

The row order follows the index labels from 4 down to 0. The marks themselves are not sorted.

Example 5: Sort column labels

result = students.sort_index(axis=1)
print(result.columns.tolist())

Expected output:

['Age', 'City', 'Marks', 'Name']

The parameter axis=1 sorts column labels. The default axis=0 sorts row index labels.

7. Resetting the Index After Sorting

Sorting rows usually preserves their original index labels. If you want the result to have a fresh sequence of index values, use reset_index().

result = students.sort_values(
    by="Marks",
    ascending=False
).reset_index(drop=True)

print(result[["Name", "Marks"]])

Expected output:

       Name  Marks
0  Samadhan     92
1     Meera     92
2   Pradnya     85
3  Samiksha     78
4     Aarav     67

The drop=True argument prevents the old index from becoming an additional column.

Useful pattern: Sort the data first, then reset the index when a clean, consecutive index is required for display or export.

8. Handling Missing Values During Sorting

A column may contain missing values represented by NaN. By default, sort_values() places missing values last. Use na_position to choose their position.

scores = pd.DataFrame({
    "Name": ["Samadhan", "Pradnya", "Samiksha", "Meera"],
    "Marks": [92, None, 78, 85]
})

print(scores.sort_values(by="Marks"))

Expected output:

       Name  Marks
2  Samiksha   78.0
3     Meera   85.0
0  Samadhan   92.0
1   Pradnya    NaN

Place missing values first

result = scores.sort_values(
    by="Marks",
    na_position="first"
)

print(result)

The row for Pradnya appears first because her marks are missing. The remaining rows are sorted in ascending order.

Missing values with descending order

result = scores.sort_values(
    by="Marks",
    ascending=False,
    na_position="last"
)

Use na_position="last" or na_position="first" explicitly when you want the placement to be clear in your code.

9. Sorting Text Values

Text columns can be sorted alphabetically. The exact ordering of text can depend on capitalization, string content, and the sorting rules used by the underlying system.

result = students.sort_values(by="Name")
print(result[["Name", "City"]])

Expected output:

       Name     City
3     Aarav   Nashik
4     Meera   Mumbai
2   Pradnya     Pune
1  Samadhan   Mumbai
0  Samiksha     Pune

This is useful for alphabetically arranging customer lists, product names, employee names, and other text-based records.

10. Sorting Dates

Convert date text into Pandas datetime values before sorting when you need chronological order.

orders = pd.DataFrame({
    "Order": ["A101", "A102", "A103"],
    "OrderDate": ["2025-03-15", "2025-01-10", "2025-02-20"]
})

orders["OrderDate"] = pd.to_datetime(orders["OrderDate"])

result = orders.sort_values(by="OrderDate")
print(result)

Expected output:

  Order  OrderDate
1  A102 2025-01-10
2  A103 2025-02-20
0  A101 2025-03-15
Common mistake: Sorting dates stored as text may not produce chronological order when formats are inconsistent. Convert the column with pd.to_datetime() first.

11. Sorting Without Modifying the Original DataFrame

By default, sort_values() returns a sorted DataFrame and does not reorder the original DataFrame in place.

sorted_students = students.sort_values(
    by="Marks",
    ascending=False
)

print("Original DataFrame:")
print(students[["Name", "Marks"]])

print("Sorted DataFrame:")
print(sorted_students[["Name", "Marks"]])

The original students DataFrame retains its initial row order, while sorted_students contains the sorted result.

If you want to keep the sorted result as your working dataset, assign it back to the original variable:

students = students.sort_values(
    by="Marks",
    ascending=False
)
Best practice: Prefer assigning the result to a variable when you want to make the next steps in your data-analysis workflow clear.

12. Practical Example: Sort Product Sales

Imagine that Pradnya is preparing a sales report. She needs to find the best-selling products and rank products within each category.

sales = pd.DataFrame({
    "Product": ["Laptop", "Mouse", "Keyboard", "Monitor", "Webcam"],
    "Category": ["Electronics", "Accessories", "Accessories",
                 "Electronics", "Accessories"],
    "UnitsSold": [25, 80, 60, 30, 45],
    "Revenue": [125000, 16000, 30000, 90000, 22500]
})

top_products = sales.sort_values(
    by="Revenue",
    ascending=False
)

print(top_products[["Product", "Revenue"]])

Expected output:

    Product  Revenue
0    Laptop   125000
3   Monitor    90000
2  Keyboard    30000
4    Webcam    22500
1     Mouse    16000

To sort by category alphabetically and then revenue from highest to lowest within each category:

result = sales.sort_values(
    by=["Category", "Revenue"],
    ascending=[True, False]
)

print(result[["Category", "Product", "Revenue"]])

Expected output:

      Category   Product  Revenue
1  Accessories     Mouse    16000
2  Accessories  Keyboard    30000
4  Accessories    Webcam    22500
0  Electronics    Laptop   125000
3  Electronics   Monitor    90000
Correction to the ordering rule: With ascending=[True, False], revenue within each category must be descending. The correct expected output is:
      Category   Product  Revenue
2  Accessories  Keyboard    30000
4  Accessories    Webcam    22500
1  Accessories     Mouse    16000
0  Electronics    Laptop   125000
3  Electronics   Monitor    90000

13. Common Sorting Parameters

Parameter Purpose Example
by Selects the column or columns used for sorting. by="Marks"
ascending Controls ascending or descending order. ascending=False
na_position Controls where missing values appear. na_position="first"
axis Chooses the sorting axis where supported. axis=0
ignore_index Creates a new consecutive index in the sorted result. ignore_index=True
kind Selects a sorting algorithm; algorithm support depends on the operation. kind="stable"
Version note: Available parameters and their behavior can depend on your Pandas version. Consult the documentation for the version installed in your environment when using less common options.

14. Common Mistakes and Their Solutions

Mistake Why it is a problem Solution
Using sort() on a DataFrame It is not the standard DataFrame sorting method. Use sort_values() or sort_index().
Forgetting ascending=False The result is in ascending order by default. Set ascending=False for descending order.
Passing multiple columns but one Boolean value The same direction applies to every selected column. Use a Boolean list such as [True, False] when directions differ.
Expecting a fresh index automatically Original index labels are generally preserved. Use reset_index(drop=True) or ignore_index=True.
Sorting dates as inconsistent text Text order may not match chronological order. Convert the column with pd.to_datetime().
Assuming the original DataFrame changes Sorting normally returns a result rather than modifying the original. Assign the result back if needed.

15. Revision Questions

  1. What is the difference between sort_values() and sort_index()?
  2. How can you sort marks from highest to lowest?
  3. How do you sort by two columns with different sort directions?
  4. What does na_position="first" do?
  5. Why might you use reset_index(drop=True) after sorting?
  6. Does sort_values() normally modify the original DataFrame?
  7. How can you sort column labels alphabetically?

16. Hands-On Exercises

Exercise 1: Student Ranking

Create a DataFrame with the names Samadhan, Pradnya, and Samiksha and marks 88, 95, and 82 respectively.

  • Sort students by marks in ascending order.
  • Sort students by marks in descending order.
  • Reset the index after descending sorting.

Exercise 2: Employee Records

Create an employee DataFrame with columns Department, Name, and Salary.

  • Sort departments alphabetically.
  • Within each department, sort salaries from highest to lowest.

Exercise 3: Missing Values

Create a DataFrame with product names and prices, including one missing price. Sort the products by price with missing values first, and then with missing values last.

Exercise 4: Date Sorting

Create a DataFrame containing at least four dates in mixed chronological order. Convert the date column to datetime and sort it from oldest to newest.

17. Quick Knowledge Check

Question 1

Which method sorts a DataFrame by the values in a column?

  • A. sort_index()
  • B. sort_values()
  • C. order_rows()

Answer: B. sort_values()

Question 2

Which argument sorts marks from highest to lowest?

  • A. ascending=True
  • B. descending=True
  • C. ascending=False

Answer: C. ascending=False

Question 3

Which method sorts rows according to their index labels?

  • A. sort_index()
  • B. sort_values()
  • C. reset_index()

Answer: A. sort_index()

Question 4

How can missing values be placed first during sorting?

  • A. missing="first"
  • B. na_position="first"
  • C. sort_na=True

Answer: B. na_position="first"

Summary

Pandas provides flexible sorting methods for organizing data. Use sort_values() to sort by column values and sort_index() to sort by index labels. The ascending parameter controls the direction, and a list of directions supports multi-column sorting.

You can also control missing-value placement, sort dates after converting them to datetime, and create a clean index after sorting. These techniques help prepare datasets for reporting, analysis, and visualization.

Practice sorting by one column first, then progress to multiple columns and missing-value handling.