Master NumPy From Scratch

Learn NumPy fundamentals, arrays, indexing, slicing, data types, array operations, broadcasting, mathematical functions, statistics, random numbers, linear algebra, file handling, and practical projects through structured lessons.

COURSE SYLLABUS
1. NumPy Introduction and Installation
Introduction to NumPy NumPy Installation and Setup NumPy Environment and IDE Setup First NumPy Program
2. NumPy Arrays and Array Creation
NumPy Array Fundamentals Array Creation Functions Zeros, Ones and Identity Arrays Range and Linspace Arrays
3. NumPy Array Indexing and Slicing
One-Dimensional Array Indexing Array Slicing Techniques Two-Dimensional Array Indexing Boolean and Fancy Indexing
4. NumPy Data Types and Attributes
NumPy Data Types Array Shape, Size and Dimensions Data Type Conversion and Casting Array Inspection and Attributes
5. NumPy Array Shape and Reshaping
Reshape and Flatten Arrays Transpose and Axis Operations Stacking and Splitting Arrays Expand and Squeeze Dimensions
6. NumPy Array Operations and Broadcasting
Arithmetic Array Operations Comparison and Logical Operations Broadcasting Fundamentals Universal Functions and Vectorization
7. NumPy Mathematical and Statistical Functions
Mathematical Functions Aggregate and Statistical Functions Minimum, Maximum and Percentiles Rounding and Trigonometric Functions
8. NumPy Filtering and Sorting
Array Filtering and Conditions Sorting and Searching Arrays Unique Values and Set Operations Where, Select and Nonzero Functions
9. NumPy Copies and Views
Array Copies and Views Shallow and Deep Copy Concepts Memory Layout and Contiguity Array Iteration and Memory Efficiency
10. NumPy Random Number Generation
Random Generator Fundamentals Random Integers and Sampling Probability Distributions Random Seeds and Simulation
11. NumPy Linear Algebra
Vectors and Matrices Matrix Multiplication and Dot Product Determinants and Matrix Inverse Eigenvalues and Linear Equations
12. NumPy File Input and Output
Importing and Exporting Text Files CSV Data with NumPy Binary Files and NPY Format NPZ Files and Data Persistence
13. NumPy Practical Projects
Student Marks Analysis Project Sales Data Analysis Project Random Simulation Project Image Array Processing Project
14. NumPy Practice and Assessment
NumPy Beginner Practice NumPy Intermediate Exercises NumPy Advanced Challenges Final NumPy Assessment

NPZ Files and Data Persistence

Learn how to save multiple NumPy arrays in a single compressed or uncompressed NPZ archive. Explore np.savez(), np.savez_compressed(), np.load(), named arrays, data persistence, and practical techniques for storing and restoring numerical datasets.

1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand data persistence and NPZ files.
  • Save multiple arrays using np.savez().
  • Compress arrays using np.savez_compressed().
  • Load arrays from an NPZ archive using np.load().
  • Access saved arrays by their names.
  • Compare NPY and NPZ file formats.
  • Inspect archive contents and array shapes.
  • Apply persistent storage to a practical data analysis project.

2. What Is Data Persistence?

Data persistence means storing information so that it remains available after a program finishes running. Instead of creating the same arrays every time, you can save them to files and load them whenever they are needed.

NumPy provides NPY files for individual arrays and NPZ files for archives containing multiple arrays.

Format Purpose NumPy Functions
NPY Stores one NumPy array. np.save(), np.load()
NPZ Stores multiple arrays in one archive. np.savez(), np.load()
Compressed NPZ Stores multiple arrays using ZIP compression. np.savez_compressed(), np.load()
Remember: Use NPY when saving one array. Use NPZ when related arrays should be stored together in a single archive.

3. Saving Multiple Arrays with np.savez()

The np.savez() function saves multiple arrays into one NPZ archive. You can provide named arrays using keyword arguments, making them easier to identify when loading the file.

import numpy as np marks = np.array([80, 85, 90]) attendance = np.array([90, 95, 92]) np.savez( "student_data.npz", marks=marks, attendance=attendance ) print("Arrays saved successfully.")

Output:

Arrays saved successfully.

The file student_data.npz contains two named arrays: marks and attendance.

marks [80, 85, 90] attendance [90, 95, 92] NPZ Archive student_data.npz Multiple arrays np.load() Restore arrays One archive can contain several named arrays.

NPZ archives group related NumPy arrays in one file.

4. Loading Arrays from an NPZ File

Use np.load() to open an NPZ archive. The returned object behaves like a dictionary of arrays, allowing you to access individual arrays using their stored names.

import numpy as np data = np.load("student_data.npz", allow_pickle=False) marks = data["marks"] attendance = data["attendance"] print("Marks:", marks) print("Attendance:", attendance) data.close()

Output:

Marks: [80 85 90] Attendance: [90 95 92]

The names inside square brackets must match the names used when the arrays were saved.

Tip: Close an NPZ archive after you finish using it, or use a with statement to manage it automatically.

Using a Context Manager

import numpy as np with np.load("student_data.npz", allow_pickle=False) as data: marks = data["marks"] attendance = data["attendance"] print("Marks:", marks) print("Attendance:", attendance)

The with statement closes the archive automatically after the block finishes.

5. Inspecting the Contents of an NPZ Archive

You can inspect the array names stored in an NPZ archive using its files attribute.

import numpy as np with np.load("student_data.npz", allow_pickle=False) as data: print("Available arrays:", data.files) for name in data.files: array = data[name] print("Name:", name) print("Shape:", array.shape) print("Data type:", array.dtype) print()

Output:

Available arrays: ['marks', 'attendance'] Name: marks Shape: (3,) Data type: int64 Name: attendance Shape: (3,) Data type: int64

The exact integer dtype can depend on the platform. The files attribute helps you discover which arrays are available before accessing them.

6. Saving Arrays with np.savez_compressed()

The np.savez_compressed() function stores multiple arrays in a compressed NPZ archive. Compression can reduce file size, particularly when the data contains patterns that compress well.

import numpy as np measurements = np.arange(10000) temperatures = np.linspace(20, 35, 10000) np.savez_compressed( "measurements_compressed.npz", measurements=measurements, temperatures=temperatures ) print("Compressed archive saved.")

Output:

Compressed archive saved.

The resulting archive contains both arrays in a compressed form. The actual size reduction depends on the data.

Important: Compression does not guarantee that every file will become smaller. Random or already compressed data may provide little size reduction.

7. Comparing np.savez() and np.savez_compressed()

Feature np.savez() np.savez_compressed()
Archive type NPZ archive without compression Compressed NPZ archive
File size Often larger May be smaller, depending on data
Saving speed Generally avoids compression work Uses extra work to compress data
Loading np.load() np.load()
Best suited for Fast storage when compression is unnecessary Reducing storage space when useful

Both functions create NPZ archives and can be loaded using the same np.load() workflow.

8. Saving Arrays with Automatic Names

You can also pass arrays to np.savez() as positional arguments. NumPy assigns names such as arr_0 and arr_1.

import numpy as np first = np.array([10, 20, 30]) second = np.array([40, 50, 60]) np.savez("automatic_names.npz", first, second) with np.load("automatic_names.npz", allow_pickle=False) as data: print("Names:", data.files) print("First array:", data["arr_0"]) print("Second array:", data["arr_1"])

Output:

Names: ['arr_0', 'arr_1'] First array: [10 20 30] Second array: [40 50 60]
Best practice: Use named arrays such as marks=marks and attendance=attendance when the data has a clear purpose. Meaningful names make code easier to read.

9. Updating and Re-Saving an Archive

NPZ archives are not designed for in-place editing like a normal mutable dictionary. A common workflow is to load the required arrays, make changes, and save a new archive.

import numpy as np # Create the original archive np.savez( "original_data.npz", marks=np.array([80, 85, 90]), attendance=np.array([90, 95, 92]) ) # Load arrays and create an updated version with np.load("original_data.npz", allow_pickle=False) as data: marks = data["marks"] attendance = data["attendance"] updated_marks = marks + 2 np.savez( "updated_data.npz", marks=updated_marks, attendance=attendance ) print("Updated archive saved.")

Output:

Updated archive saved.

The new archive contains the updated marks while preserving the attendance array. The original archive remains unchanged.

10. Working with Multidimensional Arrays

An NPZ archive can store arrays of different shapes and dimensions, as long as each saved array is individually supported by NumPy.

import numpy as np vector = np.array([1, 2, 3]) matrix = np.array([ [10, 20], [30, 40] ]) cube = np.arange(24).reshape(2, 3, 4) np.savez( "multiple_dimensions.npz", vector=vector, matrix=matrix, cube=cube ) with np.load("multiple_dimensions.npz", allow_pickle=False) as data: print("Vector shape:", data["vector"].shape) print("Matrix shape:", data["matrix"].shape) print("Cube shape:", data["cube"].shape)

Output:

Vector shape: (3,) Matrix shape: (2, 2) Cube shape: (2, 3, 4)

Each array keeps its own shape and dtype in the archive. You do not need to combine all arrays into a single rectangular array before saving them.

11. Handling Missing Files and Incorrect Keys

When loading an archive, the file may not exist or the requested array name may be incorrect. Handle these situations to make your program more reliable.

Handling a Missing Archive

import numpy as np try: with np.load("missing_data.npz", allow_pickle=False) as data: print(data.files) except FileNotFoundError: print("The NPZ file was not found.")

Output:

The NPZ file was not found.

Checking an Array Name Before Accessing It

import numpy as np with np.load("student_data.npz", allow_pickle=False) as data: if "marks" in data.files: print(data["marks"]) else: print("Marks array is not available.")

Checking data.files before accessing an array prevents accidental requests for names that do not exist.

Security note: Load only trusted files. Keep allow_pickle=False for ordinary numerical arrays unless you specifically need object-array loading and trust the source.

12. Practical Project: Student Data Persistence

Project Description

At CIIT TRANING INSTITUTE, save marks and attendance for Samadhan, Pradnya, and Samiksha in a single NPZ archive. Load the data later, calculate totals and averages, and export a compressed archive of the results.

Complete Program

import numpy as np names = ["Samadhan", "Pradnya", "Samiksha"] marks = np.array([ [80, 85, 90], [75, 92, 88], [90, 87, 95] ], dtype=np.int32) attendance = np.array([90, 95, 92], dtype=np.int32) # Save related arrays in one archive np.savez( "institute_data.npz", marks=marks, attendance=attendance ) # Load the stored arrays with np.load("institute_data.npz", allow_pickle=False) as data: loaded_marks = data["marks"] loaded_attendance = data["attendance"] # Analyse the marks totals = np.sum(loaded_marks, axis=1) averages = np.mean(loaded_marks, axis=1) for i, name in enumerate(names): print(name) print("Total:", totals[i]) print("Average:", round(averages[i], 2)) print("Attendance:", loaded_attendance[i]) print() # Save the calculated results in a compressed archive np.savez_compressed( "institute_summary.npz", totals=totals, averages=averages, attendance=loaded_attendance ) print("Compressed summary saved.")

Output:

Samadhan Total: 255 Average: 85.0 Attendance: 90 Pradnya Total: 255 Average: 85.0 Attendance: 95 Samiksha Total: 272 Average: 90.67 Attendance: 92 Compressed summary saved.

This project demonstrates how related datasets can be saved together, restored later, analysed, and persisted again in a compressed archive.

13. Common Mistakes

Mistake 1: Using the Wrong Function

Use np.savez() for multiple arrays and np.save() for a single array in NPY format.

Mistake 2: Requesting an Incorrect Array Name

Check the archive's files attribute and use the exact name assigned when saving the array.

Mistake 3: Forgetting the File Extension

NumPy normally appends .npz to string filenames that do not already end in .npz when saving an archive.

Mistake 4: Expecting Every Archive to Shrink

Compression effectiveness depends on the data. Compare actual file sizes if storage efficiency matters.

Mistake 5: Loading Untrusted Object Data

Keep allow_pickle=False for ordinary numerical archives and avoid loading pickle data from unknown sources.

14. Practice Exercises

Exercise 1: Save Multiple Arrays

Create two NumPy arrays and save them in one NPZ archive using np.savez().

Exercise 2: Load Named Arrays

Load the archive and print each array using its stored name.

Exercise 3: Inspect Archive Contents

Display data.files, the shape of each array, and each array's dtype.

Exercise 4: Create a Compressed Archive

Use np.savez_compressed() to save multiple arrays. Compare the resulting file size with an uncompressed archive.

Exercise 5: Update Stored Data

Load an archive, increase one array's values, and save the updated arrays to a new NPZ file.

Exercise 6: Student Data Persistence

Save marks and attendance for Samadhan, Pradnya, and Samiksha. Load the archive and calculate each student's average marks.

Summary

In this lesson, you learned how to store multiple NumPy arrays in an NPZ archive using np.savez() and how to create compressed archives using np.savez_compressed().

You explored loading named arrays with np.load(), inspecting archive contents, working with arrays of different shapes, and saving numerical results for future use.