NPZ Files and Data Persistence
Learn how to save multiple NumPy arrays in a single compressed or uncompressed NPZ archive. Explore np.savez(), np.savez_compressed(), np.load(), named arrays, data persistence, and practical techniques for storing and restoring numerical datasets.
1. Learning Objectives
By the end of this lesson, you will be able to:
- Understand data persistence and NPZ files.
- Save multiple arrays using np.savez().
- Compress arrays using np.savez_compressed().
- Load arrays from an NPZ archive using np.load().
- Access saved arrays by their names.
- Compare NPY and NPZ file formats.
- Inspect archive contents and array shapes.
- Apply persistent storage to a practical data analysis project.
2. What Is Data Persistence?
Data persistence means storing information so that it remains available after a program finishes running. Instead of creating the same arrays every time, you can save them to files and load them whenever they are needed.
NumPy provides NPY files for individual arrays and NPZ files for archives containing multiple arrays.
| Format | Purpose | NumPy Functions |
|---|---|---|
| NPY | Stores one NumPy array. | np.save(), np.load() |
| NPZ | Stores multiple arrays in one archive. | np.savez(), np.load() |
| Compressed NPZ | Stores multiple arrays using ZIP compression. | np.savez_compressed(), np.load() |
3. Saving Multiple Arrays with np.savez()
The np.savez() function saves multiple arrays into one NPZ archive. You can provide named arrays using keyword arguments, making them easier to identify when loading the file.
Output:
The file student_data.npz contains two named
arrays: marks and attendance.
NPZ archives group related NumPy arrays in one file.
4. Loading Arrays from an NPZ File
Use np.load() to open an NPZ archive. The returned object behaves like a dictionary of arrays, allowing you to access individual arrays using their stored names.
Output:
The names inside square brackets must match the names used when the arrays were saved.
Using a Context Manager
The with statement closes the archive automatically after the block finishes.
5. Inspecting the Contents of an NPZ Archive
You can inspect the array names stored in an NPZ archive using its files attribute.
Output:
The exact integer dtype can depend on the platform. The files attribute helps you discover which arrays are available before accessing them.
6. Saving Arrays with np.savez_compressed()
The np.savez_compressed() function stores multiple arrays in a compressed NPZ archive. Compression can reduce file size, particularly when the data contains patterns that compress well.
Output:
The resulting archive contains both arrays in a compressed form. The actual size reduction depends on the data.
7. Comparing np.savez() and np.savez_compressed()
| Feature | np.savez() | np.savez_compressed() |
|---|---|---|
| Archive type | NPZ archive without compression | Compressed NPZ archive |
| File size | Often larger | May be smaller, depending on data |
| Saving speed | Generally avoids compression work | Uses extra work to compress data |
| Loading | np.load() |
np.load() |
| Best suited for | Fast storage when compression is unnecessary | Reducing storage space when useful |
Both functions create NPZ archives and can be loaded using the same np.load() workflow.
8. Saving Arrays with Automatic Names
You can also pass arrays to np.savez() as positional arguments. NumPy assigns names such as arr_0 and arr_1.
Output:
9. Updating and Re-Saving an Archive
NPZ archives are not designed for in-place editing like a normal mutable dictionary. A common workflow is to load the required arrays, make changes, and save a new archive.
Output:
The new archive contains the updated marks while preserving the attendance array. The original archive remains unchanged.
10. Working with Multidimensional Arrays
An NPZ archive can store arrays of different shapes and dimensions, as long as each saved array is individually supported by NumPy.
Output:
Each array keeps its own shape and dtype in the archive. You do not need to combine all arrays into a single rectangular array before saving them.
11. Handling Missing Files and Incorrect Keys
When loading an archive, the file may not exist or the requested array name may be incorrect. Handle these situations to make your program more reliable.
Handling a Missing Archive
Output:
Checking an Array Name Before Accessing It
Checking data.files before accessing an array prevents accidental requests for names that do not exist.
12. Practical Project: Student Data Persistence
Project Description
At CIIT TRANING INSTITUTE, save marks and attendance for Samadhan, Pradnya, and Samiksha in a single NPZ archive. Load the data later, calculate totals and averages, and export a compressed archive of the results.
Complete Program
Output:
This project demonstrates how related datasets can be saved together, restored later, analysed, and persisted again in a compressed archive.
13. Common Mistakes
Mistake 1: Using the Wrong Function
Use np.savez() for multiple arrays and np.save() for a single array in NPY format.
Mistake 2: Requesting an Incorrect Array Name
Check the archive's files attribute and use the exact name assigned when saving the array.
Mistake 3: Forgetting the File Extension
NumPy normally appends .npz to string filenames that do not already end in .npz when saving an archive.
Mistake 4: Expecting Every Archive to Shrink
Compression effectiveness depends on the data. Compare actual file sizes if storage efficiency matters.
Mistake 5: Loading Untrusted Object Data
Keep allow_pickle=False for ordinary numerical archives and avoid loading pickle data from unknown sources.
14. Practice Exercises
Exercise 1: Save Multiple Arrays
Create two NumPy arrays and save them in one NPZ archive using np.savez().
Exercise 2: Load Named Arrays
Load the archive and print each array using its stored name.
Exercise 3: Inspect Archive Contents
Display data.files, the shape of each array, and each array's dtype.
Exercise 4: Create a Compressed Archive
Use np.savez_compressed() to save multiple arrays. Compare the resulting file size with an uncompressed archive.
Exercise 5: Update Stored Data
Load an archive, increase one array's values, and save the updated arrays to a new NPZ file.
Exercise 6: Student Data Persistence
Save marks and attendance for Samadhan, Pradnya, and Samiksha. Load the archive and calculate each student's average marks.
Summary
In this lesson, you learned how to store multiple NumPy
arrays in an NPZ archive using np.savez()
and how to create compressed archives using
np.savez_compressed().
You explored loading named arrays with np.load(),
inspecting archive contents, working with arrays of different
shapes, and saving numerical results for future use.