Master NumPy From Scratch

Learn NumPy fundamentals, arrays, indexing, slicing, data types, array operations, broadcasting, mathematical functions, statistics, random numbers, linear algebra, file handling, and practical projects through structured lessons.

COURSE SYLLABUS
1. NumPy Introduction and Installation
Introduction to NumPy NumPy Installation and Setup NumPy Environment and IDE Setup First NumPy Program
2. NumPy Arrays and Array Creation
NumPy Array Fundamentals Array Creation Functions Zeros, Ones and Identity Arrays Range and Linspace Arrays
3. NumPy Array Indexing and Slicing
One-Dimensional Array Indexing Array Slicing Techniques Two-Dimensional Array Indexing Boolean and Fancy Indexing
4. NumPy Data Types and Attributes
NumPy Data Types Array Shape, Size and Dimensions Data Type Conversion and Casting Array Inspection and Attributes
5. NumPy Array Shape and Reshaping
Reshape and Flatten Arrays Transpose and Axis Operations Stacking and Splitting Arrays Expand and Squeeze Dimensions
6. NumPy Array Operations and Broadcasting
Arithmetic Array Operations Comparison and Logical Operations Broadcasting Fundamentals Universal Functions and Vectorization
7. NumPy Mathematical and Statistical Functions
Mathematical Functions Aggregate and Statistical Functions Minimum, Maximum and Percentiles Rounding and Trigonometric Functions
8. NumPy Filtering and Sorting
Array Filtering and Conditions Sorting and Searching Arrays Unique Values and Set Operations Where, Select and Nonzero Functions
9. NumPy Copies and Views
Array Copies and Views Shallow and Deep Copy Concepts Memory Layout and Contiguity Array Iteration and Memory Efficiency
10. NumPy Random Number Generation
Random Generator Fundamentals Random Integers and Sampling Probability Distributions Random Seeds and Simulation
11. NumPy Linear Algebra
Vectors and Matrices Matrix Multiplication and Dot Product Determinants and Matrix Inverse Eigenvalues and Linear Equations
12. NumPy File Input and Output
Importing and Exporting Text Files CSV Data with NumPy Binary Files and NPY Format NPZ Files and Data Persistence
13. NumPy Practical Projects
Student Marks Analysis Project Sales Data Analysis Project Random Simulation Project Image Array Processing Project
14. NumPy Practice and Assessment
NumPy Beginner Practice NumPy Intermediate Exercises NumPy Advanced Challenges Final NumPy Assessment

NumPy Advanced Challenges

Welcome to the advanced NumPy challenges at CIIT TRANING INSTITUTE. This lesson focuses on solving complex numerical problems using multidimensional arrays, advanced indexing, broadcasting, aggregation, vectorization, linear algebra, and random simulations. Each challenge includes a problem statement, a working solution, and its expected output.

1. Advanced Multidimensional Indexing

NumPy allows you to select elements from multidimensional arrays using row and column indices, slices, and index arrays. These techniques are useful when working with large datasets and selecting specific observations.

Challenge 1: Extract Specific Elements

Given a 3 × 4 matrix, extract the elements at positions (row 0, column 1), (row 1, column 2), and (row 2, column 3).

import numpy as np data = np.array([ [10, 20, 30, 40], [50, 60, 70, 80], [90, 15, 25, 35] ]) rows = np.array([0, 1, 2]) columns = np.array([1, 2, 3]) selected = data[rows, columns] print(selected)

Expected output:

[20 70 35]

When index arrays are used together, NumPy selects corresponding row-column pairs rather than every possible combination.

Challenge 2: Select Every Other Column

data = np.arange(1, 21).reshape(4, 5) result = data[:, ::2] print(result)

Expected output:

[[ 1 3 5] [ 6 8 10] [11 13 15] [16 18 20]]

The slice ::2 selects every second column, starting from column index zero.

2. Advanced Boolean Masking

Boolean masking can combine several conditions to identify records that meet multiple requirements. Use parentheses around each condition when combining comparisons with & or |.

Challenge 3: Filter Values Within a Range

Select values greater than or equal to 20 and less than 60.

values = np.array([10, 20, 35, 50, 60, 75]) mask = (values >= 20) & (values < 60) print(values[mask])

Expected output:

[20 35 50]

Challenge 4: Find Students Who Meet Two Requirements

Select students whose average mark is at least 80 and whose attendance is at least 75%.

students = np.array(["Samadhan", "Pradnya", "Samiksha"]) averages = np.array([78, 92, 85]) attendance = np.array([90, 80, 65]) mask = (averages >= 80) & (attendance >= 75) print(students[mask])

Expected output:

['Pradnya']

Only Pradnya meets both conditions. Samiksha's average is high enough, but her attendance does not meet the threshold.

3. Broadcasting with Higher-Dimensional Arrays

Broadcasting is especially useful when applying different adjustments to rows or columns. Before performing an operation, compare the shapes of the arrays to make sure they are compatible.

Challenge 5: Normalize Values by Column

Subtract the mean of each column from every value in that column.

data = np.array([ [10.0, 20.0, 30.0], [20.0, 30.0, 40.0], [30.0, 40.0, 50.0] ]) column_means = data.mean(axis=0) centered = data - column_means print("Column means:", column_means) print("Centered data:") print(centered)

Expected output:

Column means: [20. 30. 40.] Centered data: [[-10. -10. -10.] [ 0. 0. 0.] [ 10. 10. 10.]]

The shape of column_means is (3,). Broadcasting aligns it with the three columns of the original matrix.

Challenge 6: Apply Row-wise Scaling

data = np.array([ [2.0, 4.0, 6.0], [10.0, 20.0, 30.0] ]) factors = np.array([10.0, 0.1]) scaled = data * factors[:, np.newaxis] print(scaled)

Expected output:

[[20. 40. 60.] [ 1. 2. 3.]]

factors[:, np.newaxis] converts the one-dimensional array into a column with shape (2, 1), allowing each row to be multiplied by its own scaling factor.

4. Advanced Aggregation and Axis Control

Aggregation functions can operate along one or more axes. They help summarize datasets, identify unusual values, and compare rows or columns.

Challenge 7: Find the Maximum Value in Each Row

data = np.array([ [12, 45, 23], [67, 34, 89], [21, 56, 43] ]) row_maximums = data.max(axis=1) column_minimums = data.min(axis=0) print("Row maximums:", row_maximums) print("Column minimums:", column_minimums)

Expected output:

Row maximums: [45 89 56] Column minimums: [12 34 23]

Challenge 8: Calculate Weighted Scores

Calculate each student's weighted score using weights of 20%, 30%, and 50% for three assessments.

students = np.array(["Samadhan", "Pradnya", "Samiksha"]) marks = np.array([ [70, 80, 90], [85, 90, 95], [60, 75, 80] ]) weights = np.array([0.2, 0.3, 0.5]) weighted_scores = marks @ weights print(np.round(weighted_scores, 2))

Expected output:

[83. 91.5 74.5]

Matrix multiplication combines each assessment score with its corresponding weight. The weights sum to 1, so the result is a weighted average.

5. Sorting, Ranking, and Top-N Selection

Sorting functions can order values or produce the indices needed to rank records. Use np.argsort() when you need to preserve the relationship between scores and their associated names.

Challenge 9: Find the Top Two Scores

students = np.array(["Samadhan", "Pradnya", "Samiksha"]) scores = np.array([84, 95, 89]) top_indices = np.argsort(scores)[-2:][::-1] print(students[top_indices]) print(scores[top_indices])

Expected output:

['Pradnya' 'Samiksha'] [95 89]

np.argsort(scores) returns the indices that would sort the scores in ascending order. Slicing selects the last two indices, and [::-1] reverses them into descending order.

Challenge 10: Find the Index of the Largest Element

values = np.array([18, 42, 27, 65, 39]) index = np.argmax(values) print("Index:", index) print("Maximum:", values[index])

Expected output:

Index: 3 Maximum: 65

6. Vectorization and Conditional Transformation

Vectorized expressions can transform an entire dataset in one operation. They reduce repetitive code and often run more efficiently than explicit Python loops for numerical calculations.

Challenge 11: Apply Different Transformations

Increase values below 50 by 10 and reduce values of 50 or more by 5.

values = np.array([20, 40, 50, 70, 90]) result = np.where(values < 50, values + 10, values - 5) print(result)

Expected output:

[30 50 45 65 85]

Challenge 12: Calculate Element-wise Differences

actual = np.array([100, 120, 150, 180]) predicted = np.array([95, 125, 145, 170]) errors = actual - predicted absolute_errors = np.abs(errors) print("Errors:", errors) print("Absolute errors:", absolute_errors) print("Mean absolute error:", absolute_errors.mean())

Expected output:

Errors: [ 5 -5 5 10] Absolute errors: [ 5 5 5 10] Mean absolute error: 6.25

This pattern is useful for comparing actual measurements with predictions or expected values.

7. Advanced Reshaping and Splitting

Reshaping, stacking, and splitting help reorganize data before analysis. These operations are useful when combining datasets or separating training and testing records.

Challenge 13: Split an Array into Three Parts

data = np.arange(12) first, second, third = np.split(data, 3) print(first) print(second) print(third)

Expected output:

[0 1 2 3] [4 5 6 7] [ 8 9 10 11]

np.split(data, 3) divides the array into three equal sections. The number of sections must divide the array length evenly.

Challenge 14: Combine Arrays Vertically

first = np.array([[1, 2], [3, 4]]) second = np.array([[5, 6]]) combined = np.vstack((first, second)) print(combined) print("Shape:", combined.shape)

Expected output:

[[1 2] [3 4] [5 6]] Shape: (3, 2)

Vertical stacking adds rows. The arrays must have compatible column dimensions.

8. Random Sampling and Reproducibility

NumPy's random generator can produce repeatable results when initialized with the same seed. This is important for testing, experiments, and simulations.

Challenge 15: Generate Repeatable Random Data

rng = np.random.default_rng(42) sample = rng.integers(1, 11, size=5) print(sample)

The generator produces five integers in the range 1 through 10, because the upper bound of integers() is excluded. Using the same seed and the same sequence of generator calls makes the result reproducible in the same NumPy environment.

Best practice: Prefer np.random.default_rng(seed) for new code instead of relying on the legacy global random state.

9. Linear Algebra Challenge

NumPy provides matrix multiplication, determinants, and solutions to systems of linear equations through numpy.linalg.

Challenge 16: Solve a System of Equations

Solve the following system:
2x + y = 5
x + 3y = 6

A = np.array([ [2.0, 1.0], [1.0, 3.0] ]) b = np.array([5.0, 6.0]) solution = np.linalg.solve(A, b) print(np.round(solution, 2))

Expected output:

[1.8 1.4]

The returned array contains the solution values for x and y. Use np.linalg.solve() when solving a square linear system with a unique solution, rather than calculating the inverse explicitly.

10. Final Advanced Project: Sales Data Analysis

In this project, analyze sales for three products across four months. Calculate total sales per product, monthly totals, identify the best-selling product, and find the month with the highest overall sales.

Complete Project Solution

import numpy as np products = np.array(["Product A", "Product B", "Product C"]) sales = np.array([ [120, 150, 180, 200], [100, 130, 160, 190], [140, 145, 155, 170] ]) product_totals = sales.sum(axis=1) monthly_totals = sales.sum(axis=0) best_product_index = np.argmax(product_totals) best_month_index = np.argmax(monthly_totals) print("Product totals:", product_totals) print("Monthly totals:", monthly_totals) print("Best-selling product:", products[best_product_index]) print("Highest-sales month index:", best_month_index)

Expected output:

Product totals: [650 580 610] Monthly totals: [360 425 495 560] Best-selling product: Product A Highest-sales month index: 3

The monthly index is zero-based, so index 3 refers to the fourth month. The project combines aggregation, indexing, and maximum-value selection into a practical data analysis workflow.

11. Advanced Practice Questions

Question 1: Replace Values Above a Threshold

Replace every value greater than 100 with 100.

values = np.array([45, 120, 80, 150, 95]) result = np.minimum(values, 100) print(result) # [ 45 100 80 100 95]

Question 2: Calculate Row-wise Standard Deviations

data = np.array([ [10, 20, 30], [20, 20, 20] ]) print(data.std(axis=1)) # [8.16496581 0. ]

NumPy calculates the population standard deviation by default, using ddof=0.

Question 3: Find All Values Above the Mean

values = np.array([10, 20, 30, 40, 50]) mean_value = values.mean() print(values[values > mean_value]) # [40 50]

Question 4: Calculate a Matrix Product

A = np.array([[1, 2], [3, 4]]) B = np.array([[5, 6], [7, 8]]) print(A @ B) # [[19 22] # [43 50]]

Question 5: Count Occurrences of Each Value

values = np.array([1, 2, 2, 3, 3, 3]) unique_values, counts = np.unique(values, return_counts=True) print(unique_values) print(counts) # [1 2 3] # [1 2 3]

12. Performance and Best Practices

  • Prefer vectorized NumPy operations over unnecessary Python loops.
  • Check array shapes before using broadcasting or matrix multiplication.
  • Choose suitable data types to balance precision and memory usage.
  • Use Boolean masks to filter records efficiently.
  • Use np.random.default_rng() for modern random-number generation.
  • Use np.linalg.solve() for suitable linear systems instead of explicitly calculating an inverse.
  • Remember that basic slices usually return views, while advanced indexing generally returns copies.
  • Validate input dimensions and handle empty arrays when building reusable functions.
Important: Vectorization can improve performance, but not every NumPy expression is automatically faster for every workload. For large datasets, measure execution time and memory usage using realistic inputs.

Summary

In this lesson, you practiced advanced indexing, Boolean masking, broadcasting, axis-based aggregation, weighted calculations, sorting, vectorized transformations, array reshaping, random sampling, and linear algebra using NumPy.

Apply these techniques to practical datasets such as student marks, sales reports, scientific measurements, and numerical simulations. Complete the challenges independently and test your solutions with different input shapes and values before attempting the final assessment.