Master NumPy From Scratch

Learn NumPy fundamentals, arrays, indexing, slicing, data types, array operations, broadcasting, mathematical functions, statistics, random numbers, linear algebra, file handling, and practical projects through structured lessons.

COURSE SYLLABUS
1. NumPy Introduction and Installation
Introduction to NumPy NumPy Installation and Setup NumPy Environment and IDE Setup First NumPy Program
2. NumPy Arrays and Array Creation
NumPy Array Fundamentals Array Creation Functions Zeros, Ones and Identity Arrays Range and Linspace Arrays
3. NumPy Array Indexing and Slicing
One-Dimensional Array Indexing Array Slicing Techniques Two-Dimensional Array Indexing Boolean and Fancy Indexing
4. NumPy Data Types and Attributes
NumPy Data Types Array Shape, Size and Dimensions Data Type Conversion and Casting Array Inspection and Attributes
5. NumPy Array Shape and Reshaping
Reshape and Flatten Arrays Transpose and Axis Operations Stacking and Splitting Arrays Expand and Squeeze Dimensions
6. NumPy Array Operations and Broadcasting
Arithmetic Array Operations Comparison and Logical Operations Broadcasting Fundamentals Universal Functions and Vectorization
7. NumPy Mathematical and Statistical Functions
Mathematical Functions Aggregate and Statistical Functions Minimum, Maximum and Percentiles Rounding and Trigonometric Functions
8. NumPy Filtering and Sorting
Array Filtering and Conditions Sorting and Searching Arrays Unique Values and Set Operations Where, Select and Nonzero Functions
9. NumPy Copies and Views
Array Copies and Views Shallow and Deep Copy Concepts Memory Layout and Contiguity Array Iteration and Memory Efficiency
10. NumPy Random Number Generation
Random Generator Fundamentals Random Integers and Sampling Probability Distributions Random Seeds and Simulation
11. NumPy Linear Algebra
Vectors and Matrices Matrix Multiplication and Dot Product Determinants and Matrix Inverse Eigenvalues and Linear Equations
12. NumPy File Input and Output
Importing and Exporting Text Files CSV Data with NumPy Binary Files and NPY Format NPZ Files and Data Persistence
13. NumPy Practical Projects
Student Marks Analysis Project Sales Data Analysis Project Random Simulation Project Image Array Processing Project
14. NumPy Practice and Assessment
NumPy Beginner Practice NumPy Intermediate Exercises NumPy Advanced Challenges Final NumPy Assessment

Binary Files and NPY Format

Learn how to save NumPy arrays in binary format using np.save(), load them using np.load(), and preserve array shape, data type, and numerical values. Explore the NPY file format, compare binary storage with text files, and practise saving and restoring numerical datasets.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand binary files and the NPY format.
  • Save NumPy arrays using np.save().
  • Load saved arrays using np.load().
  • Preserve array shape and data type.
  • Understand the difference between text and binary storage.
  • Save and load multidimensional arrays.
  • Inspect saved array information.
  • Handle common file path and loading errors.

2. What Is a Binary File?

A binary file stores information as bytes rather than as human-readable text characters. Many numerical applications use binary formats because they can preserve data types and array structure without converting every value into text.

NumPy provides the NPY format specifically for saving a single NumPy array. It stores information needed to reconstruct the array, including its shape and data type, together with its numerical data.

Feature Text File NPY File
Format Human-readable characters NumPy binary format
Typical functions np.loadtxt(), np.savetxt() np.load(), np.save()
Shape and dtype May need to be inferred or specified Stored in the file format
Human readability Usually readable in a text editor Not intended for direct text editing
Remember: Use NPY when you want to preserve a NumPy array for later use in Python without manually reconstructing its dimensions and numerical type.

3. Saving an Array Using np.save()

The np.save() function saves a NumPy array to a binary NPY file. If the filename does not already end in .npy, NumPy normally appends that extension.

import numpy as np data = np.array([10, 20, 30, 40, 50]) np.save("numbers.npy", data) print("Array saved successfully.")

Output:

Array saved successfully.

The program creates a file named numbers.npy in the current working directory. The array is stored in NumPy's binary format.

Saving a Two-Dimensional Array

import numpy as np marks = np.array([ [80, 85, 90], [75, 92, 88], [90, 87, 95] ]) np.save("marks.npy", marks) print("Marks array saved.")

Output:

Marks array saved.

NumPy stores the entire two-dimensional array, including its shape and numerical data type.

4. Loading an Array Using np.load()

Use np.load() to restore an array from an NPY file. The loaded array contains the values stored in the file and retains its stored shape and dtype.

import numpy as np data = np.array([10, 20, 30, 40, 50]) np.save("numbers.npy", data) loaded_data = np.load("numbers.npy") print(loaded_data) print("Shape:", loaded_data.shape) print("Data type:", loaded_data.dtype)

Output:

[10 20 30 40 50] Shape: (5,) Data type: int64

The displayed integer dtype can vary by platform. The important point is that the saved array's numerical type is preserved by the NPY format.

Tip: The file must exist at the path supplied to np.load(). Save the array first or load an existing NPY file.

5. Preserving Array Shape and Data Type

A useful feature of NPY files is that the stored array can be reconstructed with its original dimensions and dtype.

import numpy as np data = np.array([ [1.5, 2.5, 3.5], [4.5, 5.5, 6.5] ], dtype=np.float32) np.save("decimal_data.npy", data) loaded_data = np.load("decimal_data.npy") print(loaded_data) print("Original shape:", data.shape) print("Loaded shape:", loaded_data.shape) print("Original dtype:", data.dtype) print("Loaded dtype:", loaded_data.dtype)

Output:

[[1.5 2.5 3.5] [4.5 5.5 6.5]] Original shape: (2, 3) Loaded shape: (2, 3) Original dtype: float32 Loaded dtype: float32

The loaded array has the same shape and dtype as the original array.

6. Saving Different Data Types

NumPy arrays can contain different numerical data types, including integers and floating-point values. NPY files preserve the array's dtype when saved and loaded.

Integer Array

import numpy as np integer_data = np.array([10, 20, 30], dtype=np.int32) np.save("integer_data.npy", integer_data) loaded_integer = np.load("integer_data.npy") print(loaded_integer) print(loaded_integer.dtype)

Output:

[10 20 30] int32

Floating-Point Array

import numpy as np float_data = np.array([1.25, 2.50, 3.75], dtype=np.float64) np.save("float_data.npy", float_data) loaded_float = np.load("float_data.npy") print(loaded_float) print(loaded_float.dtype)

Output:

[1.25 2.5 3.75] float64
Remember: The NPY format preserves the array's dtype. It does not automatically convert every saved array to integers or floating-point numbers.

7. Saving Three-Dimensional Arrays

NPY supports arrays with more than two dimensions. This makes it useful for multidimensional numerical datasets, such as image batches, scientific measurements, and simulation results.

import numpy as np data = np.arange(24).reshape(2, 3, 4) np.save("three_dimensional.npy", data) loaded_data = np.load("three_dimensional.npy") print("Shape:", loaded_data.shape) print("Dimensions:", loaded_data.ndim) print(loaded_data)

Output:

Shape: (2, 3, 4) Dimensions: 3 [[[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] [[12 13 14 15] [16 17 18 19] [20 21 22 23]]]

The array has two blocks, each containing three rows and four columns. Its shape remains (2, 3, 4) after loading.

8. Comparing Text Files and NPY Files

Text files and NPY files both store data, but they are useful for different situations.

Requirement Recommended Approach
Human-readable numerical data TXT or CSV
Preserving a NumPy array's shape and dtype NPY
Sharing data with spreadsheet software CSV
Saving one array for later NumPy use NPY
Inspecting values with a text editor TXT or CSV
Important: NPY files are binary files and are not intended to be edited manually in a text editor. Use np.load() to read them and NumPy functions to modify their array contents.

9. Checking Whether the Saved Array Matches

After saving and loading an array, you can compare the original and restored arrays to confirm that the values are preserved.

import numpy as np original = np.array([ [10, 20, 30], [40, 50, 60] ]) np.save("verification.npy", original) loaded = np.load("verification.npy") print("Arrays are equal:", np.array_equal(original, loaded)) print("Original shape:", original.shape) print("Loaded shape:", loaded.shape)

Output:

Arrays are equal: True Original shape: (2, 3) Loaded shape: (2, 3)

np.array_equal() checks whether the arrays have the same shape and equal values. It is a useful check for arrays containing ordinary integer values.

10. Loading Arrays Safely

Only load NPY files from trusted sources. NumPy's np.load() has an allow_pickle option that controls loading object arrays using Python pickle serialization.

import numpy as np data = np.array([10, 20, 30]) np.save("safe_numbers.npy", data) loaded_data = np.load( "safe_numbers.npy", allow_pickle=False ) print(loaded_data)

Output:

[10 20 30]

For ordinary numerical arrays, allow_pickle=False is a suitable choice. Object arrays that require pickle may not load with this setting.

Security note: Avoid enabling pickle loading for files from untrusted sources. Pickle-based files can contain data that executes code during deserialization.

11. Handling File Errors

A file operation can fail if the file is missing or the path is incorrect. You can handle expected errors with try and except.

import numpy as np try: data = np.load("unknown_array.npy", allow_pickle=False) print(data) except FileNotFoundError: print("The NPY file was not found.") except ValueError: print("The file could not be loaded with these settings.")

If the file does not exist, the program displays a meaningful message instead of terminating with an unhandled exception.

Tip: Check the filename, file extension, and working directory when a file cannot be found.

12. Practical Project: Student Marks Storage

Project Description

At CIIT TRANING INSTITUTE, save the numerical marks of Samadhan, Pradnya, and Samiksha in an NPY file. Load the file later and calculate each student's total and average.

Complete Program

import numpy as np names = ["Samadhan", "Pradnya", "Samiksha"] marks = np.array([ [80, 85, 90], [75, 92, 88], [90, 87, 95] ], dtype=np.int32) # Save the array np.save("institute_marks.npy", marks) # Load the saved array loaded_marks = np.load( "institute_marks.npy", allow_pickle=False ) # Calculate totals and averages totals = np.sum(loaded_marks, axis=1) averages = np.mean(loaded_marks, axis=1) for i, name in enumerate(names): print(name) print("Total:", totals[i]) print("Average:", round(averages[i], 2)) print("Array shape:", loaded_marks.shape) print("Data type:", loaded_marks.dtype)

Output:

Samadhan Total: 255 Average: 85.0 Pradnya Total: 255 Average: 85.0 Samiksha Total: 272 Average: 90.67 Array shape: (3, 3) Data type: int32

This project demonstrates how NPY files can preserve numerical data for later analysis without recreating the original array manually.

13. Common Mistakes

Mistake 1: Loading the Wrong Filename

Confirm that the filename and path used by np.load() match the location where the array was saved.

Mistake 2: Confusing NPY with CSV

Use np.load() for NPY files. Use suitable text-loading functions for CSV and other numerical text files.

Mistake 3: Expecting NPY to Be Human-Readable

NPY is a binary format. Inspect its contents by loading the file into NumPy and printing the array.

Mistake 4: Forgetting the Array Shape

Check the shape and dimensions after loading if the subsequent calculation expects a particular structure.

Mistake 5: Loading Untrusted Pickle Data

Prefer allow_pickle=False for ordinary numerical arrays and load files only from trusted sources.

14. Practice Exercises

Exercise 1: Save a One-Dimensional Array

Create an array containing five integers. Save it as an NPY file and load it again.

Exercise 2: Preserve the Data Type

Create an array with dtype=float32. Save and reload it, then compare the original and loaded dtypes.

Exercise 3: Save a Two-Dimensional Array

Create a 3 × 3 matrix and save it using np.save(). Check the loaded array's shape.

Exercise 4: Save a Three-Dimensional Array

Create an array using np.arange() and reshape it into three dimensions. Save and load it, then inspect ndim.

Exercise 5: Verify Stored Values

Use np.array_equal() to verify that the loaded array matches the original array.

Exercise 6: Student Marks Storage

Save the marks of Samadhan, Pradnya, and Samiksha in an NPY file. Load them and calculate row-wise averages.

Summary

In this lesson, you learned how to save NumPy arrays in binary format using np.save() and restore them using np.load().

You explored how NPY files preserve array shape and data type, worked with multidimensional arrays, compared binary files with text files, and practised storing and reloading student marks for numerical analysis.