Binary Files and NPY Format
Learn how to save NumPy arrays in binary format using np.save(), load them using np.load(), and preserve array shape, data type, and numerical values. Explore the NPY file format, compare binary storage with text files, and practise saving and restoring numerical datasets.
1. Learning Objectives
By the end of this lesson, you will be able to:
- Understand binary files and the NPY format.
- Save NumPy arrays using np.save().
- Load saved arrays using np.load().
- Preserve array shape and data type.
- Understand the difference between text and binary storage.
- Save and load multidimensional arrays.
- Inspect saved array information.
- Handle common file path and loading errors.
2. What Is a Binary File?
A binary file stores information as bytes rather than as human-readable text characters. Many numerical applications use binary formats because they can preserve data types and array structure without converting every value into text.
NumPy provides the NPY format specifically for saving a single NumPy array. It stores information needed to reconstruct the array, including its shape and data type, together with its numerical data.
| Feature | Text File | NPY File |
|---|---|---|
| Format | Human-readable characters | NumPy binary format |
| Typical functions | np.loadtxt(), np.savetxt() |
np.load(), np.save() |
| Shape and dtype | May need to be inferred or specified | Stored in the file format |
| Human readability | Usually readable in a text editor | Not intended for direct text editing |
3. Saving an Array Using np.save()
The np.save() function saves a NumPy array to a binary NPY file. If the filename does not already end in .npy, NumPy normally appends that extension.
Output:
The program creates a file named numbers.npy
in the current working directory. The array is stored in
NumPy's binary format.
Saving a Two-Dimensional Array
Output:
NumPy stores the entire two-dimensional array, including its shape and numerical data type.
4. Loading an Array Using np.load()
Use np.load() to restore an array from an NPY file. The loaded array contains the values stored in the file and retains its stored shape and dtype.
Output:
The displayed integer dtype can vary by platform. The important point is that the saved array's numerical type is preserved by the NPY format.
5. Preserving Array Shape and Data Type
A useful feature of NPY files is that the stored array can be reconstructed with its original dimensions and dtype.
Output:
The loaded array has the same shape and dtype as the original array.
6. Saving Different Data Types
NumPy arrays can contain different numerical data types, including integers and floating-point values. NPY files preserve the array's dtype when saved and loaded.
Integer Array
Output:
Floating-Point Array
Output:
7. Saving Three-Dimensional Arrays
NPY supports arrays with more than two dimensions. This makes it useful for multidimensional numerical datasets, such as image batches, scientific measurements, and simulation results.
Output:
The array has two blocks, each containing three rows and four columns. Its shape remains (2, 3, 4) after loading.
8. Comparing Text Files and NPY Files
Text files and NPY files both store data, but they are useful for different situations.
| Requirement | Recommended Approach |
|---|---|
| Human-readable numerical data | TXT or CSV |
| Preserving a NumPy array's shape and dtype | NPY |
| Sharing data with spreadsheet software | CSV |
| Saving one array for later NumPy use | NPY |
| Inspecting values with a text editor | TXT or CSV |
9. Checking Whether the Saved Array Matches
After saving and loading an array, you can compare the original and restored arrays to confirm that the values are preserved.
Output:
np.array_equal() checks whether the arrays have the same shape and equal values. It is a useful check for arrays containing ordinary integer values.
10. Loading Arrays Safely
Only load NPY files from trusted sources. NumPy's np.load() has an allow_pickle option that controls loading object arrays using Python pickle serialization.
Output:
For ordinary numerical arrays, allow_pickle=False is a suitable choice. Object arrays that require pickle may not load with this setting.
11. Handling File Errors
A file operation can fail if the file is missing or the path is incorrect. You can handle expected errors with try and except.
If the file does not exist, the program displays a meaningful message instead of terminating with an unhandled exception.
12. Practical Project: Student Marks Storage
Project Description
At CIIT TRANING INSTITUTE, save the numerical marks of Samadhan, Pradnya, and Samiksha in an NPY file. Load the file later and calculate each student's total and average.
Complete Program
Output:
This project demonstrates how NPY files can preserve numerical data for later analysis without recreating the original array manually.
13. Common Mistakes
Mistake 1: Loading the Wrong Filename
Confirm that the filename and path used by np.load() match the location where the array was saved.
Mistake 2: Confusing NPY with CSV
Use np.load() for NPY files. Use suitable text-loading functions for CSV and other numerical text files.
Mistake 3: Expecting NPY to Be Human-Readable
NPY is a binary format. Inspect its contents by loading the file into NumPy and printing the array.
Mistake 4: Forgetting the Array Shape
Check the shape and dimensions after loading if the subsequent calculation expects a particular structure.
Mistake 5: Loading Untrusted Pickle Data
Prefer allow_pickle=False for ordinary numerical arrays and load files only from trusted sources.
14. Practice Exercises
Exercise 1: Save a One-Dimensional Array
Create an array containing five integers. Save it as an NPY file and load it again.
Exercise 2: Preserve the Data Type
Create an array with dtype=float32. Save and reload it, then compare the original and loaded dtypes.
Exercise 3: Save a Two-Dimensional Array
Create a 3 × 3 matrix and save it using np.save(). Check the loaded array's shape.
Exercise 4: Save a Three-Dimensional Array
Create an array using np.arange() and reshape it into three dimensions. Save and load it, then inspect ndim.
Exercise 5: Verify Stored Values
Use np.array_equal() to verify that the loaded array matches the original array.
Exercise 6: Student Marks Storage
Save the marks of Samadhan, Pradnya, and Samiksha in an NPY file. Load them and calculate row-wise averages.
Summary
In this lesson, you learned how to save NumPy arrays
in binary format using np.save() and
restore them using np.load().
You explored how NPY files preserve array shape and data type, worked with multidimensional arrays, compared binary files with text files, and practised storing and reloading student marks for numerical analysis.