Chapter 7: Matrix Inverses and Determinant Fundamentals

Basic Concepts of Invertible Matrices

Definition of Matrix Inverse

For an $n \times n$ square matrix $A$, if there exists a matrix $B$ such that: $AB = BA = I_n$

then $A$ is called an invertible matrix (or non-singular matrix), and $B$ is the inverse matrix of $A$, denoted as $A^{-1}$.

Important Properties:

1. The inverse matrix is unique
2. $(A^{-1})^{-1} = A$
3. $(AB)^{-1} = B^{-1}A^{-1}$
4. $(A^T)^{-1} = (A^{-1})^T$

import numpy as np
import matplotlib.pyplot as plt
from scipy.linalg import inv, det

# Example of an invertible matrix
A = np.array([[2, 1],
              [1, 1]])

print("Matrix A:")
print(A)

# Calculate the inverse matrix
A_inv = inv(A)
print("\nInverse of A:")
print(A_inv)

# Verify AA^(-1) = I
identity_check = A @ A_inv
print("\nAA^(-1) =")
print(identity_check)

# Verify A^(-1)A = I
identity_check2 = A_inv @ A
print("\nA^(-1)A =")
print(identity_check2)

Criteria for Invertibility

A square matrix $A$ is invertible if and only if the following conditions are equivalent:

1. The determinant of $A$: $\det(A) \neq 0$
2. The rank of $A$ equals $n$ (full rank)
3. The column vectors of $A$ are linearly independent
4. The row vectors of $A$ are linearly independent
5. The homogeneous system $A\mathbf{x} = \mathbf{0}$ has only the zero solution
6. For any $\mathbf{b}$, the system $A\mathbf{x} = \mathbf{b}$ has a unique solution

def check_invertibility(A):
    """
    Check if a matrix is invertible
    """
    print(f"Matrix A:")
    print(A)

    # Calculate determinant
    det_A = det(A)
    print(f"\ndet(A) = {det_A:.6f}")

    # Calculate rank
    rank_A = np.linalg.matrix_rank(A)
    n = A.shape[0]
    print(f"rank(A) = {rank_A}, matrix size = {n}×{n}")

    # Determine invertibility
    if abs(det_A) > 1e-10:
        print("Matrix is invertible")
        try:
            A_inv = inv(A)
            print("Inverse matrix:")
            print(A_inv)
        except:
            print("Numerical computation error")
    else:
        print("Matrix is not invertible (singular matrix)")

# Test invertible matrix
print("Test 1: Invertible matrix")
A1 = np.array([[1, 2], [3, 4]])
check_invertibility(A1)

print("\n" + "="*50 + "\n")

# Test non-invertible matrix
print("Test 2: Non-invertible matrix")
A2 = np.array([[1, 2], [2, 4]])
check_invertibility(A2)

Methods for Finding Inverse Matrices

Method 1: Gauss-Jordan Elimination

Transform the augmented matrix $[A|I]$ into $[I|A^{-1}]$ through row operations:

def matrix_inverse_gauss_jordan(A):
    """
    Find inverse matrix using Gauss-Jordan elimination
    """
    n = A.shape[0]
    if A.shape[0] != A.shape[1]:
        raise ValueError("Matrix must be square")

    # Construct augmented matrix [A|I]
    augmented = np.column_stack([A.astype(float), np.eye(n)])
    print("Augmented matrix [A|I]:")
    print(augmented)
    print()

    # Gauss-Jordan elimination
    for i in range(n):
        # Find pivot
        max_row = i
        for k in range(i+1, n):
            if abs(augmented[k, i]) > abs(augmented[max_row, i]):
                max_row = k

        # Swap rows
        if max_row != i:
            augmented[i], augmented[max_row] = augmented[max_row].copy(), augmented[i].copy()
            print(f"Swap row {i+1} and row {max_row+1}")

        # Check if pivot is zero
        if abs(augmented[i, i]) < 1e-10:
            raise ValueError("Matrix is not invertible")

        # Make pivot 1
        pivot = augmented[i, i]
        augmented[i] = augmented[i] / pivot
        print(f"Row {i+1} divided by {pivot:.3f}")
        print(augmented)
        print()

        # Eliminate other elements in this column
        for j in range(n):
            if j != i and abs(augmented[j, i]) > 1e-10:
                factor = augmented[j, i]
                augmented[j] = augmented[j] - factor * augmented[i]
                print(f"Row {j+1} <- Row {j+1} - {factor:.3f} * Row {i+1}")

    print("Final result [I|A^(-1)]:")
    print(augmented)

    # Extract inverse matrix
    A_inv = augmented[:, n:]
    return A_inv

# Example
A_example = np.array([[1, 2], [3, 4]])
print("Finding inverse of matrix:")
print(A_example)
print()

A_inv_manual = matrix_inverse_gauss_jordan(A_example)
print("\nCalculated inverse matrix:")
print(A_inv_manual)

# Verification
print("\nVerify AA^(-1):")
print(A_example @ A_inv_manual)

Method 2: Adjugate Matrix Method

For a $2 \times 2$ matrix: $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$

The inverse matrix is: $A^{-1} = \frac{1}{\det(A)} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$

def inverse_2x2(A):
    """
    Formula for inverse of 2×2 matrix
    """
    if A.shape != (2, 2):
        raise ValueError("Only applicable to 2×2 matrices")

    a, b = A[0, 0], A[0, 1]
    c, d = A[1, 0], A[1, 1]

    det_A = a*d - b*c
    if abs(det_A) < 1e-10:
        raise ValueError("Matrix is not invertible")

    A_inv = np.array([[d, -b], [-c, a]]) / det_A
    return A_inv

# Test 2×2 matrix inverse formula
A_2x2 = np.array([[3, 1], [2, 1]])
print("2×2 matrix:")
print(A_2x2)

A_inv_formula = inverse_2x2(A_2x2)
print("\nInverse calculated using formula:")
print(A_inv_formula)

A_inv_numpy = inv(A_2x2)
print("\nInverse calculated by NumPy:")
print(A_inv_numpy)

print("\nDifference:")
print(np.abs(A_inv_formula - A_inv_numpy))

Definition and Properties of Determinants

Definition of Determinants

Determinant of 2×2 matrix: $\det\begin{bmatrix} a & b \\ c & d \end{bmatrix} = ad - bc$

Determinant of 3×3 matrix (Sarrus’ rule): $\det\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix} = aei + bfg + cdh - ceg - afh - bdi$

def det_2x2(A):
    """Calculate determinant of 2×2 matrix"""
    return A[0,0]*A[1,1] - A[0,1]*A[1,0]

def det_3x3(A):
    """Calculate determinant of 3×3 matrix (Sarrus' rule)"""
    a, b, c = A[0, :]
    d, e, f = A[1, :]
    g, h, i = A[2, :]

    positive = a*e*i + b*f*g + c*d*h
    negative = c*e*g + a*f*h + b*d*i

    return positive - negative

# Test determinant calculation
print("2×2 matrix determinant calculation:")
A_2 = np.array([[3, 1], [2, 4]])
print(f"Matrix: \n{A_2}")
print(f"Manual calculation: {det_2x2(A_2)}")
print(f"NumPy calculation: {det(A_2)}")

print("\n3×3 matrix determinant calculation:")
A_3 = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 10]])
print(f"Matrix: \n{A_3}")
print(f"Sarrus' rule: {det_3x3(A_3)}")
print(f"NumPy calculation: {det(A_3)}")

Geometric Meaning of Determinants

The absolute value of the determinant represents the volume of the parallelepiped formed by the column vectors (or row vectors) of the matrix:

• 2×2 matrix: area of parallelogram
• 3×3 matrix: volume of parallelepiped

def visualize_determinant_2d():
    """Visualize geometric meaning of 2D determinants"""
    fig, axes = plt.subplots(1, 2, figsize=(12, 5))

    # Example 1: Positive determinant
    v1 = np.array([3, 1])
    v2 = np.array([1, 2])
    A1 = np.column_stack([v1, v2])
    det1 = det(A1)

    # Draw vectors and parallelogram
    axes[0].arrow(0, 0, v1[0], v1[1], head_width=0.1, head_length=0.1,
                  fc='red', ec='red', linewidth=2, label=f'v1 = {v1}')
    axes[0].arrow(0, 0, v2[0], v2[1], head_width=0.1, head_length=0.1,
                  fc='blue', ec='blue', linewidth=2, label=f'v2 = {v2}')

    # Vertices of parallelogram
    parallelogram1 = np.array([[0, 0], [v1[0], v1[1]],
                              [v1[0]+v2[0], v1[1]+v2[1]], [v2[0], v2[1]], [0, 0]])
    axes[0].plot(parallelogram1[:, 0], parallelogram1[:, 1], 'g--', alpha=0.7)
    axes[0].fill(parallelogram1[:-1, 0], parallelogram1[:-1, 1], alpha=0.3, color='green')

    axes[0].set_title(f'det = {det1:.2f} (area)')
    axes[0].grid(True, alpha=0.3)
    axes[0].legend()
    axes[0].set_aspect('equal')
    axes[0].set_xlim(-1, 5)
    axes[0].set_ylim(-1, 4)

    # Example 2: Negative determinant (reversed vector order)
    v3 = np.array([1, 2])
    v4 = np.array([3, 1])
    A2 = np.column_stack([v3, v4])
    det2 = det(A2)

    axes[1].arrow(0, 0, v3[0], v3[1], head_width=0.1, head_length=0.1,
                  fc='red', ec='red', linewidth=2, label=f'v1 = {v3}')
    axes[1].arrow(0, 0, v4[0], v4[1], head_width=0.1, head_length=0.1,
                  fc='blue', ec='blue', linewidth=2, label=f'v2 = {v4}')

    parallelogram2 = np.array([[0, 0], [v3[0], v3[1]],
                              [v3[0]+v4[0], v3[1]+v4[1]], [v4[0], v4[1]], [0, 0]])
    axes[1].plot(parallelogram2[:, 0], parallelogram2[:, 1], 'g--', alpha=0.7)
    axes[1].fill(parallelogram2[:-1, 0], parallelogram2[:-1, 1], alpha=0.3, color='orange')

    axes[1].set_title(f'det = {det2:.2f} (area, opposite direction)')
    axes[1].grid(True, alpha=0.3)
    axes[1].legend()
    axes[1].set_aspect('equal')
    axes[1].set_xlim(-1, 5)
    axes[1].set_ylim(-1, 4)

    plt.tight_layout()
    plt.show()

visualize_determinant_2d()

Important Properties of Determinants

1. Transpose invariance: $\det(A^T) = \det(A)$
2. Multilinearity: Linear with respect to each row (column)
3. Antisymmetry: Swapping two rows (columns) changes the sign of the determinant
4. Product property: $\det(AB) = \det(A)\det(B)$

def demonstrate_determinant_properties():
    """Demonstrate properties of determinants"""
    A = np.array([[1, 2], [3, 4]])
    B = np.array([[2, 1], [1, 3]])

    print("Matrix A:")
    print(A)
    print("Matrix B:")
    print(B)
    print()

    # Property 1: Transpose invariance
    print("Property 1: Transpose invariance")
    print(f"det(A) = {det(A):.6f}")
    print(f"det(A^T) = {det(A.T):.6f}")
    print(f"Equal? {np.isclose(det(A), det(A.T))}")
    print()

    # Property 2: Product property
    print("Property 2: Product property")
    AB = A @ B
    print(f"det(A) = {det(A):.6f}")
    print(f"det(B) = {det(B):.6f}")
    print(f"det(AB) = {det(AB):.6f}")
    print(f"det(A) * det(B) = {det(A) * det(B):.6f}")
    print(f"Equal? {np.isclose(det(AB), det(A) * det(B))}")
    print()

    # Property 3: Antisymmetry (swap rows)
    print("Property 3: Antisymmetry")
    A_swapped = A[[1, 0], :]  # Swap row 1 and row 2
    print("Matrix after swapping rows:")
    print(A_swapped)
    print(f"Original determinant: {det(A):.6f}")
    print(f"After swapping rows: {det(A_swapped):.6f}")
    print(f"Sign changed? {np.isclose(det(A), -det(A_swapped))}")

demonstrate_determinant_properties()

Determinants and Linear Transformations

Determinant as a Measure of Transformation

The determinant describes the scaling effect of a linear transformation on area/volume:

def visualize_linear_transformation():
    """Visualize linear transformations and determinants"""
    fig, axes = plt.subplots(2, 2, figsize=(12, 10))

    # Original unit square
    unit_square = np.array([[0, 1, 1, 0, 0],
                           [0, 0, 1, 1, 0]])

    # Transformation matrices
    transformations = [
        np.array([[2, 0], [0, 1]]),      # Horizontal stretch
        np.array([[1, 0.5], [0, 1]]),    # Shear transformation
        np.array([[1, 0], [0, 2]]),      # Vertical stretch
        np.array([[0, 1], [1, 0]])       # Reflection
    ]

    titles = ['Horizontal stretch (det=2)', 'Shear transformation (det=1)',
              'Vertical stretch (det=2)', 'Reflection (det=-1)']

    for i, (T, title) in enumerate(zip(transformations, titles)):
        row, col = i // 2, i % 2
        ax = axes[row, col]

        # Transformed shape
        transformed_square = T @ unit_square

        # Draw original and transformed shapes
        ax.plot(unit_square[0], unit_square[1], 'b-', linewidth=2,
                label='Original', alpha=0.7)
        ax.fill(unit_square[0], unit_square[1], alpha=0.3, color='blue')

        ax.plot(transformed_square[0], transformed_square[1], 'r-',
                linewidth=2, label='Transformed')
        ax.fill(transformed_square[0], transformed_square[1],
                alpha=0.3, color='red')

        # Calculate determinant
        det_T = det(T)
        ax.set_title(f'{title}\ndet = {det_T}')
        ax.grid(True, alpha=0.3)
        ax.legend()
        ax.set_aspect('equal')
        ax.set_xlim(-0.5, 2.5)
        ax.set_ylim(-0.5, 2.5)

    plt.tight_layout()
    plt.show()

visualize_linear_transformation()

Methods for Calculating Determinants

Minors and Cofactors

For an $n \times n$ matrix $A$:

• Minor $M_{ij}$: determinant of the $(n-1) \times (n-1)$ matrix obtained by deleting row $i$ and column $j$
• Cofactor $C_{ij} = (-1)^{i+j}M_{ij}$

Row expansion formula: $\det(A) = \sum_{j=1}^n a_{ij}C_{ij}$

def cofactor_expansion(A, row=0):
    """
    Calculate determinant by expanding along specified row
    """
    n = A.shape[0]
    if n == 1:
        return A[0, 0]
    elif n == 2:
        return A[0, 0] * A[1, 1] - A[0, 1] * A[1, 0]

    det_A = 0
    for j in range(n):
        # Construct minor matrix
        minor_matrix = np.delete(np.delete(A, row, axis=0), j, axis=1)

        # Calculate cofactor
        cofactor = (-1)**(row + j) * cofactor_expansion(minor_matrix)

        # Accumulate
        det_A += A[row, j] * cofactor

        print(f"a_{row+1}{j+1} = {A[row, j]}, cofactor = {cofactor:.3f}")

    return det_A

# Example: 3×3 matrix
A_3x3 = np.array([[1, 2, 3],
                  [4, 5, 6],
                  [7, 8, 9]])

print("3×3 matrix:")
print(A_3x3)
print("\nExpanding along first row:")
det_manual = cofactor_expansion(A_3x3, row=0)
print(f"\nManually calculated determinant: {det_manual}")
print(f"NumPy calculated determinant: {det(A_3x3)}")

Determinant of Triangular Matrices

The determinant of an upper or lower triangular matrix equals the product of its diagonal elements:

def triangular_determinant():
    """Demonstrate calculation of triangular matrix determinants"""

    # Upper triangular matrix
    upper_triangular = np.array([[2, 3, 1],
                                [0, 4, 2],
                                [0, 0, 5]])

    # Lower triangular matrix
    lower_triangular = np.array([[3, 0, 0],
                                [2, 4, 0],
                                [1, 5, 6]])

    print("Upper triangular matrix:")
    print(upper_triangular)
    det_upper = np.prod(np.diag(upper_triangular))
    print(f"Product of diagonal elements: {det_upper}")
    print(f"NumPy calculated determinant: {det(upper_triangular)}")

    print("\nLower triangular matrix:")
    print(lower_triangular)
    det_lower = np.prod(np.diag(lower_triangular))
    print(f"Product of diagonal elements: {det_lower}")
    print(f"NumPy calculated determinant: {det(lower_triangular)}")

triangular_determinant()

Application Example

Cramer’s Rule

For a system of $n$ linear equations $A\mathbf{x} = \mathbf{b}$, if $\det(A) \neq 0$, then: $x_i = \frac{\det(A_i)}{\det(A)}$

where $A_i$ is the matrix obtained by replacing the $i$-th column of $A$ with $\mathbf{b}$.

def cramers_rule(A, b):
    """
    Solve linear system using Cramer's rule
    """
    n = A.shape[0]
    det_A = det(A)

    if abs(det_A) < 1e-10:
        raise ValueError("Coefficient matrix is singular, cannot use Cramer's rule")

    print("Solving linear system using Cramer's rule")
    print(f"Determinant of coefficient matrix: {det_A}")
    print()

    x = np.zeros(n)
    for i in range(n):
        # Construct A_i
        A_i = A.copy()
        A_i[:, i] = b

        det_A_i = det(A_i)
        x[i] = det_A_i / det_A

        print(f"x_{i+1} = det(A_{i+1}) / det(A) = {det_A_i} / {det_A} = {x[i]:.6f}")

    return x

# Example
A_cramer = np.array([[2, 1, -1],
                     [1, 3, 2],
                     [3, 1, 1]])
b_cramer = np.array([8, 13, 10])

print("System of equations:")
print("2x₁ + x₂ - x₃ = 8")
print("x₁ + 3x₂ + 2x₃ = 13")
print("3x₁ + x₂ + x₃ = 10")
print()

x_cramer = cramers_rule(A_cramer, b_cramer)
print(f"\nSolution vector: {x_cramer}")

# Verification
verification = A_cramer @ x_cramer
print(f"Verification: A @ x = {verification}")
print(f"Original right-hand side: {b_cramer}")

Chapter Summary

graph TD
    A[Matrix Inverse] --> B[Definition: AA⁻¹ = I]
    A --> C[Method: Gauss-Jordan elimination]
    A --> D[Property: Uniqueness]

    E[Determinant] --> F[Geometric meaning: Volume scaling]
    E --> G[Calculation method: Cofactor expansion]
    E --> H[Properties: Multilinearity]

    B --> I[Invertibility condition]
    F --> I
    I --> J[det(A) ≠ 0]
    I --> K[Full rank]
    I --> L[Column vectors linearly independent]

    G --> M[Cramer's rule]
    D --> M

Matrix inverses and determinants are core tools in linear algebra. They not only have profound theoretical significance but also play important roles in practical applications.