Master NumPy From Scratch

Learn NumPy fundamentals, arrays, indexing, slicing, data types, array operations, broadcasting, mathematical functions, statistics, random numbers, linear algebra, file handling, and practical projects through structured lessons.

COURSE SYLLABUS
1. NumPy Introduction and Installation
Introduction to NumPy NumPy Installation and Setup NumPy Environment and IDE Setup First NumPy Program
2. NumPy Arrays and Array Creation
NumPy Array Fundamentals Array Creation Functions Zeros, Ones and Identity Arrays Range and Linspace Arrays
3. NumPy Array Indexing and Slicing
One-Dimensional Array Indexing Array Slicing Techniques Two-Dimensional Array Indexing Boolean and Fancy Indexing
4. NumPy Data Types and Attributes
NumPy Data Types Array Shape, Size and Dimensions Data Type Conversion and Casting Array Inspection and Attributes
5. NumPy Array Shape and Reshaping
Reshape and Flatten Arrays Transpose and Axis Operations Stacking and Splitting Arrays Expand and Squeeze Dimensions
6. NumPy Array Operations and Broadcasting
Arithmetic Array Operations Comparison and Logical Operations Broadcasting Fundamentals Universal Functions and Vectorization
7. NumPy Mathematical and Statistical Functions
Mathematical Functions Aggregate and Statistical Functions Minimum, Maximum and Percentiles Rounding and Trigonometric Functions
8. NumPy Filtering and Sorting
Array Filtering and Conditions Sorting and Searching Arrays Unique Values and Set Operations Where, Select and Nonzero Functions
9. NumPy Copies and Views
Array Copies and Views Shallow and Deep Copy Concepts Memory Layout and Contiguity Array Iteration and Memory Efficiency
10. NumPy Random Number Generation
Random Generator Fundamentals Random Integers and Sampling Probability Distributions Random Seeds and Simulation
11. NumPy Linear Algebra
Vectors and Matrices Matrix Multiplication and Dot Product Determinants and Matrix Inverse Eigenvalues and Linear Equations
12. NumPy File Input and Output
Importing and Exporting Text Files CSV Data with NumPy Binary Files and NPY Format NPZ Files and Data Persistence
13. NumPy Practical Projects
Student Marks Analysis Project Sales Data Analysis Project Random Simulation Project Image Array Processing Project
14. NumPy Practice and Assessment
NumPy Beginner Practice NumPy Intermediate Exercises NumPy Advanced Challenges Final NumPy Assessment

CSV Data with NumPy

Learn how to read, process, analyse, and export CSV data using NumPy. Explore delimiters, headers, column selection, row-wise and column-wise calculations, filtering numerical records, and saving processed results to CSV files.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand the structure of CSV files.
  • Load numerical CSV data using np.loadtxt().
  • Handle CSV headers and delimiters.
  • Inspect rows, columns, and array shapes.
  • Calculate totals, averages, minimums, and maximums.
  • Filter numerical data using conditions.
  • Export processed data using np.savetxt().
  • Understand when genfromtxt() is useful for missing values.

2. What Is a CSV File?

CSV stands for Comma-Separated Values. It is a text-based file format in which values are commonly separated by commas. CSV files are often used to exchange tabular data between spreadsheets, databases, and programming applications.

Example file named marks.csv:

Python,NumPy,Statistics 80,85,90 75,92,88 90,87,95

The first row contains column headings. Each following row contains numerical marks for one student.

Python NumPy Statistics 80 85 90 75 92 88 90 87 95 A CSV file represents rows and columns using text values.

CSV records can be loaded into NumPy for numerical analysis.

3. Loading CSV Data with np.loadtxt()

Use np.loadtxt() to load a CSV file when its data is consistently numerical and can be represented in a regular array.

import numpy as np data = np.loadtxt( "marks.csv", delimiter=",", skiprows=1 ) print(data) print("Shape:", data.shape)

Output:

[[80. 85. 90.] [75. 92. 88.] [90. 87. 95.]] Shape: (3, 3)

The delimiter argument specifies commas, while skiprows=1 skips the textual header. The resulting array contains three rows and three columns.

Loading Values as Integers

import numpy as np data = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) print(data) print("Data type:", data.dtype)

Output:

[[80 85 90] [75 92 88] [90 87 95]] Data type: int64
Tip: Use dtype=int for whole-number data. Use floating-point types when the values contain decimals. The exact integer dtype displayed may depend on your platform.

4. Understanding Rows, Columns, and Shape

After loading CSV data, use the shape attribute to identify how many rows and columns are present.

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) print("Number of rows:", marks.shape[0]) print("Number of columns:", marks.shape[1]) print("Total values:", marks.size)

Output:

Number of rows: 3 Number of columns: 3 Total values: 9

In this dataset, each row represents one student and each column represents a subject.

Accessing a Row

first_student = marks[0] print(first_student)

Output:

[80 85 90]

Accessing a Column

python_marks = marks[:, 0] print(python_marks)

Output:

[80 75 90]
Remember: NumPy indexing starts at zero. The first row is index 0 and the first column is index 0.

5. Selecting Specific Columns

You can select a column by using a colon for all rows and the column index for the desired subject.

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) python_marks = marks[:, 0] numpy_marks = marks[:, 1] statistics_marks = marks[:, 2] print("Python:", python_marks) print("NumPy:", numpy_marks) print("Statistics:", statistics_marks)

Output:

Python: [80 75 90] NumPy: [85 92 87] Statistics: [90 88 95]

This technique is useful when you want to analyse one subject or one numerical field independently.

6. Calculating Row-Wise Totals and Averages

NumPy aggregation functions can calculate totals and averages for each row. Use axis=1 to perform calculations across columns within each row.

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) totals = np.sum(marks, axis=1) averages = np.mean(marks, axis=1) print("Totals:", totals) print("Averages:", averages)

Output:

Totals: [255 255 272] Averages: [85. 85. 90.66666667]

The first row has a total of 255 and an average of 85. The second row also has a total of 255 and an average of 85. The third row has a total of 272 and an average of approximately 90.67.

Remember: axis=1 operates across each row. axis=0 operates down each column.

7. Calculating Column-Wise Statistics

To calculate statistics for each subject across all students, use axis=0.

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) subject_totals = np.sum(marks, axis=0) subject_averages = np.mean(marks, axis=0) subject_minimums = np.min(marks, axis=0) subject_maximums = np.max(marks, axis=0) print("Subject totals:", subject_totals) print("Subject averages:", subject_averages) print("Minimum marks:", subject_minimums) print("Maximum marks:", subject_maximums)

Output:

Subject totals: [245 264 273] Subject averages: [81.66666667 88. 91. ] Minimum marks: [75 85 88] Maximum marks: [90 92 95]

Each result contains one value for each subject column: Python, NumPy, and Statistics.

8. Filtering CSV Data

Once CSV data is loaded into a NumPy array, Boolean indexing can select rows that satisfy a condition.

Select Students with Average Marks at Least 85

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) averages = np.mean(marks, axis=1) selected_rows = marks[averages >= 85] print(selected_rows)

Output:

[[80 85 90] [75 92 88] [90 87 95]]

Every row in this example has an average of at least 85, so all three rows are selected.

Select Rows with Python Marks of at Least 80

python_marks = marks[:, 0] selected_rows = marks[python_marks >= 80] print(selected_rows)

Output:

[[80 85 90] [90 87 95]]
Tip: Boolean indexing is useful for filtering numerical records without writing a manual loop.

9. Saving Processed Data to CSV

After analysing data, you can export the processed results to a new CSV file using np.savetxt().

import numpy as np marks = np.loadtxt( "marks.csv", delimiter=",", skiprows=1, dtype=int ) averages = np.mean(marks, axis=1) np.savetxt( "student_averages.csv", averages, delimiter=",", fmt="%.2f", header="Average", comments="" ) print("Averages exported successfully.")

Output:

Averages exported successfully.

The resulting file contains:

Average 85.00 85.00 90.67

The fmt="%.2f" argument writes averages to two decimal places. The header provides a column label for the exported data.

10. Handling Missing Values with np.genfromtxt()

Real-world CSV files may contain missing numerical values. np.loadtxt() is best suited to clean, consistently formatted numerical data. For certain missing-value situations, np.genfromtxt() provides additional options.

Example file named measurements.csv:

10,20,30 40,,60 70,80,90

Loading Missing Values as NaN

import numpy as np data = np.genfromtxt( "measurements.csv", delimiter=",", filling_values=np.nan ) print(data)

Output:

[[10. 20. 30.] [40. nan 60.] [70. 80. 90.]]

The missing value is represented by NaN, which stands for Not a Number.

Calculating an Average While Ignoring NaN

column_averages = np.nanmean(data, axis=0) print(column_averages)

Output:

[40. 50. 60.]

np.nanmean() calculates averages while ignoring NaN values. This is useful when a missing value should not be treated as a valid numerical measurement.

Important: Missing data should be handled according to the meaning of the dataset. Replacing missing values with zero may change the results and should not be done automatically.

11. Practical Project: Student Marks Analysis

Project Description

At CIIT TRANING INSTITUTE, analyse the marks of Samadhan, Pradnya, and Samiksha. Store the marks in a CSV file, load the numerical records, calculate totals and averages, identify the highest average, and export the summary.

Complete Program

import numpy as np names = ["Samadhan", "Pradnya", "Samiksha"] marks = np.array([ [80, 85, 90], [75, 92, 88], [90, 87, 95] ]) # Save the original marks np.savetxt( "institute_marks.csv", marks, delimiter=",", fmt="%d", header="Python,NumPy,Statistics", comments="" ) # Load the marks loaded_marks = np.loadtxt( "institute_marks.csv", delimiter=",", skiprows=1, dtype=int ) # Calculate statistics totals = np.sum(loaded_marks, axis=1) averages = np.mean(loaded_marks, axis=1) for i, name in enumerate(names): print( name, "Total:", totals[i], "Average:", round(averages[i], 2) ) # Export calculated results summary = np.column_stack((totals, averages)) np.savetxt( "marks_summary.csv", summary, delimiter=",", fmt=["%d", "%.2f"], header="Total,Average", comments="" ) print("Summary exported successfully.")

Output:

Samadhan Total: 255 Average: 85.0 Pradnya Total: 255 Average: 85.0 Samiksha Total: 272 Average: 90.67 Summary exported successfully.

This project demonstrates a complete workflow: create numerical data, export it, load it, calculate statistics, and save the results in a separate CSV file.

12. Common Mistakes

Mistake 1: Forgetting the Delimiter

Specify delimiter="," when loading comma-separated data.

Mistake 2: Reading the Header as Numbers

Use skiprows=1 when the first row contains textual column headings and the remaining data is numerical.

Mistake 3: Confusing Rows and Columns

Use axis=1 for row-wise operations and axis=0 for column-wise operations.

Mistake 4: Ignoring Missing Values

Inspect the dataset and choose an appropriate missing-data strategy before calculating statistics.

Mistake 5: Using np.loadtxt() for Irregular Data

When numerical data contains missing values, consider np.genfromtxt() and configure its missing-value handling.

13. Practice Exercises

Exercise 1: Load a CSV File

Create a CSV file with three numerical columns. Load it using np.loadtxt() and display its shape.

Exercise 2: Select a Column

Load student marks and display only the NumPy marks column.

Exercise 3: Calculate Row Averages

Calculate each student's average using np.mean() with axis=1.

Exercise 4: Calculate Subject Averages

Calculate the average for each subject using axis=0.

Exercise 5: Filter Records

Select all rows in which the Python marks are at least 80.

Exercise 6: Export a Summary

Save totals and averages to a CSV file using np.savetxt(). Include a header and format averages to two decimal places.

Exercise 7: Handle Missing Values

Create a CSV file with one missing numerical value. Load it using np.genfromtxt() and calculate a NaN-aware column average.

Summary

In this lesson, you learned how to load numerical CSV data using np.loadtxt(), handle delimiters and headers, and select individual rows and columns.

You explored row-wise and column-wise statistics, Boolean filtering, exporting processed results with np.savetxt(), and handling missing values with np.genfromtxt().