Chapter 13: Quadratic Forms Theory

Haiyue

2025年10月7日

26min

Chapter 13: Quadratic Forms Theory

Learning Objectives

Understand the definition and matrix representation of quadratic forms
Master standardization methods for quadratic forms
Understand the classification of quadratic forms
Master the determination of positive definite quadratic forms
Understand the geometric meaning of quadratic forms

Basic Concepts of Quadratic Forms

Definition

A quadratic form with $n$ variables $x_1, x_2, \ldots, x_n$ is of the form: $f(x_1, x_2, \ldots, x_n) = \sum_{i=1}^n \sum_{j=1}^n a_{ij}x_ix_j$

which is a quadratic homogeneous polynomial.

Matrix Representation

A quadratic form can be written in matrix form: $f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x}$

where $A = (a_{ij})$ is a symmetric matrix.

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from scipy.linalg import eig, cholesky

def quadratic_forms_basics():
    """Basic concepts and representation of quadratic forms"""

    print("Quadratic Forms Basic Concepts Demonstration")
    print("=" * 50)

    # Two-dimensional quadratic form examples
    print("1. Two-Dimensional Quadratic Form Examples")

    # Define quadratic form matrices
    A1 = np.array([[2, 1],
                   [1, 3]], dtype=float)

    A2 = np.array([[1, 0],
                   [0, -1]], dtype=float)

    A3 = np.array([[1, 2],
                   [2, 1]], dtype=float)

    quadratic_forms = [
        (A1, "2x₁² + 2x₁x₂ + 3x₂²"),
        (A2, "x₁² - x₂²"),
        (A3, "x₁² + 4x₁x₂ + x₂²")
    ]

    fig, axes = plt.subplots(2, 3, figsize=(15, 10))

    for i, (A, formula) in enumerate(quadratic_forms):
        # Compute eigenvalues
        eigenvals, eigenvecs = eig(A)

        print(f"\nQuadratic Form {i+1}: {formula}")
        print(f"Matrix A:\n{A}")
        print(f"Eigenvalues: {eigenvals}")

        # Plot contours
        ax1 = axes[0, i]
        x = np.linspace(-3, 3, 100)
        y = np.linspace(-3, 3, 100)
        X, Y = np.meshgrid(x, y)

        # Compute quadratic form values
        Z = A[0,0]*X**2 + 2*A[0,1]*X*Y + A[1,1]*Y**2

        # Determine contour levels based on eigenvalues
        if np.all(eigenvals > 0):
            levels = [1, 4, 9, 16]
            colors = 'blue'
        elif np.all(eigenvals < 0):
            levels = [-16, -9, -4, -1]
            colors = 'red'
        else:
            levels = [-4, -1, 1, 4]
            colors = 'purple'

        contours = ax1.contour(X, Y, Z, levels=levels, colors=colors)
        ax1.clabel(contours, inline=True, fontsize=8)

        # Draw eigenvectors
        for j, (val, vec) in enumerate(zip(eigenvals, eigenvecs.T)):
            length = 1.5
            ax1.arrow(0, 0, vec[0]*length, vec[1]*length,
                     head_width=0.1, head_length=0.1, fc=f'C{j}', ec=f'C{j}',
                     linewidth=2, alpha=0.7)

        ax1.set_xlim(-3, 3)
        ax1.set_ylim(-3, 3)
        ax1.set_aspect('equal')
        ax1.grid(True, alpha=0.3)
        ax1.set_title(f'{formula}\nEigenvalues: {eigenvals[0]:.1f}, {eigenvals[1]:.1f}')

        # 3D surface plot
        ax2 = axes[1, i]
        x_3d = np.linspace(-2, 2, 30)
        y_3d = np.linspace(-2, 2, 30)
        X_3d, Y_3d = np.meshgrid(x_3d, y_3d)
        Z_3d = A[0,0]*X_3d**2 + 2*A[0,1]*X_3d*Y_3d + A[1,1]*Y_3d**2

        # Limit Z range for visualization
        Z_3d = np.clip(Z_3d, -20, 20)

        if np.all(eigenvals > 0):
            ax2.contourf(X_3d, Y_3d, Z_3d, levels=20, cmap='Blues')
            type_name = "Positive Definite"
        elif np.all(eigenvals < 0):
            ax2.contourf(X_3d, Y_3d, Z_3d, levels=20, cmap='Reds')
            type_name = "Negative Definite"
        else:
            ax2.contourf(X_3d, Y_3d, Z_3d, levels=20, cmap='RdBu')
            type_name = "Indefinite"

        ax2.set_title(f'{type_name} Quadratic Form')
        ax2.set_xlabel('x₁')
        ax2.set_ylabel('x₂')

    plt.tight_layout()
    plt.show()

quadratic_forms_basics()

Standardization of Quadratic Forms

Completing the Square Method

Transform quadratic forms into standard form by completing the square: $f(x_1, x_2, \ldots, x_n) = \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots + \lambda_n y_n^2$

Orthogonal Transformation Method

Use orthogonal diagonalization of symmetric matrices: $A = Q\Lambda Q^T$

where $Q$ is an orthogonal matrix and $\Lambda$ is a diagonal matrix.

def quadratic_form_standardization():
    """Standardization methods for quadratic forms"""

    print("Quadratic Form Standardization Demonstration")
    print("=" * 50)

    # Define three-dimensional quadratic form
    A = np.array([[1, 1, 0],
                  [1, 2, 1],
                  [0, 1, 1]], dtype=float)

    print("Original Quadratic Form Matrix:")
    print(A)

    # Corresponding quadratic form
    print("\nCorresponding Quadratic Form:")
    print("f(x₁,x₂,x₃) = x₁² + 2x₁x₂ + 2x₂² + 2x₂x₃ + x₃²")

    # Method 1: Completing the square (manual demonstration)
    print("\nMethod 1: Completing the Square")
    print("f = x₁² + 2x₁x₂ + 2x₂² + 2x₂x₃ + x₃²")
    print("  = (x₁ + x₂)² - x₂² + 2x₂² + 2x₂x₃ + x₃²")
    print("  = (x₁ + x₂)² + x₂² + 2x₂x₃ + x₃²")
    print("  = (x₁ + x₂)² + (x₂ + x₃)² - x₃² + x₃²")
    print("  = (x₁ + x₂)² + (x₂ + x₃)² + 0·x₃²")

    # Method 2: Orthogonal diagonalization
    print("\nMethod 2: Orthogonal Diagonalization")

    # Compute eigenvalues and eigenvectors
    eigenvals, eigenvecs = eig(A)

    # Sort
    idx = np.argsort(eigenvals)[::-1]
    eigenvals = eigenvals[idx]
    eigenvecs = eigenvecs[:, idx]

    # Orthogonal matrix
    Q = eigenvecs
    Lambda = np.diag(eigenvals)

    print(f"Eigenvalues: {eigenvals}")
    print(f"Orthogonal Matrix Q:\n{Q}")
    print(f"Diagonal Matrix Λ:\n{Lambda}")

    # Verify diagonalization
    reconstructed = Q @ Lambda @ Q.T
    print(f"\nVerification: Q Λ Q^T =\n{reconstructed}")
    print(f"Error: {np.linalg.norm(A - reconstructed):.2e}")

    # Standard form
    print(f"\nStandard Form: f = {eigenvals[0]:.3f}y₁² + {eigenvals[1]:.3f}y₂² + {eigenvals[2]:.3f}y₃²")

    # Coordinate transformation
    print(f"\nCoordinate Transformation: x = Qy, i.e.")
    for i in range(3):
        coeffs = Q[:, i]
        terms = []
        for j, coeff in enumerate(coeffs):
            if abs(coeff) > 1e-10:
                terms.append(f"{coeff:.3f}y_{j+1}")
        print(f"x_{i+1} = {' + '.join(terms)}")

    # Sylvester's Law of Inertia
    positive_eigenvals = np.sum(eigenvals > 1e-10)
    negative_eigenvals = np.sum(eigenvals < -1e-10)
    zero_eigenvals = np.sum(np.abs(eigenvals) <= 1e-10)

    print(f"\nSylvester's Law of Inertia:")
    print(f"Positive Index: {positive_eigenvals}")
    print(f"Negative Index: {negative_eigenvals}")
    print(f"Zero Eigenvalues: {zero_eigenvals}")

    return eigenvals, Q

eigenvals, Q = quadratic_form_standardization()

Classification of Quadratic Forms

Classification by Sign

Based on the signs of eigenvalues, quadratic forms can be classified as:

Positive Definite: All eigenvalues are positive
Negative Definite: All eigenvalues are negative
Positive Semi-definite: Eigenvalues are non-negative, at least one is zero
Negative Semi-definite: Eigenvalues are non-positive, at least one is zero
Indefinite: Has both positive and negative eigenvalues

def quadratic_form_classification():
    """Classification and properties of quadratic forms"""

    print("Quadratic Form Classification Demonstration")
    print("=" * 50)

    # Construct different types of quadratic forms
    matrices = {
        "Positive Definite": np.array([[2, 0, 0], [0, 1, 0], [0, 0, 3]]),
        "Negative Definite": np.array([[-2, 0, 0], [0, -1, 0], [0, 0, -3]]),
        "Positive Semi-definite": np.array([[1, 0, 0], [0, 2, 0], [0, 0, 0]]),
        "Negative Semi-definite": np.array([[-1, 0, 0], [0, -2, 0], [0, 0, 0]]),
        "Indefinite": np.array([[1, 0, 0], [0, -1, 0], [0, 0, 2]])
    }

    # Add non-diagonal cases
    matrices["Positive Definite (Non-diagonal)"] = np.array([[2, 1, 0], [1, 2, 1], [0, 1, 2]])
    matrices["Indefinite (Non-diagonal)"] = np.array([[1, 2, 0], [2, -1, 1], [0, 1, 1]])

    results = {}

    for name, A in matrices.items():
        eigenvals = eig(A)[0]
        eigenvals = np.real(eigenvals)  # Eigenvalues of symmetric matrices are real

        print(f"\n{name}:")
        print(f"Matrix:\n{A}")
        print(f"Eigenvalues: {eigenvals}")

        # Classification determination
        pos = np.sum(eigenvals > 1e-10)
        neg = np.sum(eigenvals < -1e-10)
        zero = np.sum(np.abs(eigenvals) <= 1e-10)

        print(f"Number of Positive Eigenvalues: {pos}")
        print(f"Number of Negative Eigenvalues: {neg}")
        print(f"Number of Zero Eigenvalues: {zero}")

        # Determine type
        if pos == len(eigenvals) and zero == 0:
            qtype = "Positive Definite"
        elif neg == len(eigenvals) and zero == 0:
            qtype = "Negative Definite"
        elif pos > 0 and neg == 0:
            qtype = "Positive Semi-definite"
        elif neg > 0 and pos == 0:
            qtype = "Negative Semi-definite"
        elif pos > 0 and neg > 0:
            qtype = "Indefinite"
        else:
            qtype = "Zero Form"

        print(f"Classification Result: {qtype}")

        results[name] = {
            'matrix': A,
            'eigenvals': eigenvals,
            'type': qtype,
            'positive': pos,
            'negative': neg,
            'zero': zero
        }

    # Visualize different types of quadratic forms (2D case)
    fig, axes = plt.subplots(2, 3, figsize=(15, 10))
    axes = axes.flatten()

    # 2D examples
    matrices_2d = {
        "Positive Definite": np.array([[2, 0], [0, 1]]),
        "Negative Definite": np.array([[-2, 0], [0, -1]]),
        "Positive Semi-definite": np.array([[1, 0], [0, 0]]),
        "Negative Semi-definite": np.array([[-1, 0], [0, 0]]),
        "Indefinite": np.array([[1, 0], [0, -1]]),
        "Positive Definite (Correlated)": np.array([[2, 1], [1, 2]])
    }

    for i, (name, A) in enumerate(matrices_2d.items()):
        ax = axes[i]

        x = np.linspace(-3, 3, 100)
        y = np.linspace(-3, 3, 100)
        X, Y = np.meshgrid(x, y)

        Z = A[0,0]*X**2 + 2*A[0,1]*X*Y + A[1,1]*Y**2

        eigenvals_2d = eig(A)[0]

        # Choose contours based on type
        if name == "Positive Definite" or name == "Positive Definite (Correlated)":
            levels = [0.5, 1, 2, 4, 8]
            colors = 'blue'
        elif name == "Negative Definite":
            levels = [-8, -4, -2, -1, -0.5]
            colors = 'red'
        elif name == "Positive Semi-definite":
            levels = [0, 0.5, 1, 2, 4]
            colors = 'green'
        elif name == "Negative Semi-definite":
            levels = [-4, -2, -1, -0.5, 0]
            colors = 'orange'
        else:  # Indefinite
            levels = [-4, -2, -1, 0, 1, 2, 4]
            colors = 'purple'

        try:
            contours = ax.contour(X, Y, Z, levels=levels, colors=colors)
            ax.clabel(contours, inline=True, fontsize=8)
        except:
            # If contour plotting fails, use filled contour
            ax.contourf(X, Y, Z, levels=50, alpha=0.7)

        ax.set_xlim(-3, 3)
        ax.set_ylim(-3, 3)
        ax.set_aspect('equal')
        ax.grid(True, alpha=0.3)
        ax.set_title(f'{name}\nλ = {eigenvals_2d[0]:.1f}, {eigenvals_2d[1]:.1f}')

    plt.tight_layout()
    plt.show()

    return results

classification_results = quadratic_form_classification()

Determination of Positive Definite Quadratic Forms

Determination Methods

Eigenvalue Criterion: All eigenvalues are positive
Principal Minor Criterion: All leading principal minors are positive
Cholesky Decomposition: Matrix can be Cholesky decomposed

def positive_definite_tests():
    """Various methods for determining positive definite quadratic forms"""

    print("Comparison of Positive Definite Quadratic Form Determination Methods")
    print("=" * 50)

    # Test matrices
    test_matrices = {
        "Positive Definite Matrix 1": np.array([[2, 1], [1, 2]]),
        "Positive Definite Matrix 2": np.array([[4, 2, 1], [2, 3, 1], [1, 1, 2]]),
        "Non-Positive Definite Matrix 1": np.array([[1, 2], [2, 1]]),
        "Non-Positive Definite Matrix 2": np.array([[1, 0, 0], [0, 0, 1], [0, 1, 0]]),
        "Boundary Case": np.array([[1, 1], [1, 1]])
    }

    for name, A in test_matrices.items():
        print(f"\n{name}:")
        print(f"Matrix A:\n{A}")

        # Method 1: Eigenvalue determination
        eigenvals = eig(A)[0]
        eigenvals = np.real(eigenvals)
        all_positive = np.all(eigenvals > 1e-10)

        print(f"\nMethod 1 - Eigenvalue Criterion:")
        print(f"Eigenvalues: {eigenvals}")
        print(f"Is Positive Definite: {all_positive}")

        # Method 2: Principal minor determination
        print(f"\nMethod 2 - Principal Minor Criterion:")
        principal_minors = []
        positive_minors = True

        for i in range(1, A.shape[0] + 1):
            minor = np.linalg.det(A[:i, :i])
            principal_minors.append(minor)
            print(f"{i}-th Leading Principal Minor: {minor:.6f}")

            if minor <= 1e-10:
                positive_minors = False

        print(f"All Principal Minors Positive: {positive_minors}")

        # Method 3: Cholesky decomposition
        print(f"\nMethod 3 - Cholesky Decomposition:")
        try:
            L = cholesky(A, lower=True)
            cholesky_success = True
            print(f"Cholesky Decomposition Successful")
            print(f"L =\n{L}")
            print(f"Verification: L L^T =\n{L @ L.T}")
        except np.linalg.LinAlgError:
            cholesky_success = False
            print("Cholesky Decomposition Failed - Matrix Not Positive Definite")

        # Summary
        print(f"\nSummary:")
        print(f"Eigenvalue Method: {'Positive Definite' if all_positive else 'Non-Positive Definite'}")
        print(f"Principal Minor Method: {'Positive Definite' if positive_minors else 'Non-Positive Definite'}")
        print(f"Cholesky Method: {'Positive Definite' if cholesky_success else 'Non-Positive Definite'}")

        consistent = (all_positive == positive_minors == cholesky_success)
        print(f"All Three Methods Consistent: {consistent}")

        print("-" * 50)

    # Visualize geometric meaning of positive definiteness
    fig, axes = plt.subplots(1, 3, figsize=(15, 5))

    # Ellipse for positive definite matrix
    A_pos = np.array([[2, 0.5], [0.5, 1]])
    eigenvals_pos, eigenvecs_pos = eig(A_pos)

    ax1 = axes[0]
    theta = np.linspace(0, 2*np.pi, 1000)
    # Unit circle
    unit_circle = np.array([np.cos(theta), np.sin(theta)])
    # Transformed ellipse
    ellipse = np.linalg.inv(np.sqrt(A_pos)) @ unit_circle

    ax1.plot(ellipse[0], ellipse[1], 'b-', linewidth=2, label='f(x) = 1')
    ax1.plot(ellipse[0]*np.sqrt(2), ellipse[1]*np.sqrt(2), 'g--', linewidth=2, label='f(x) = 2')

    # Draw eigenvectors
    for i, (val, vec) in enumerate(zip(eigenvals_pos, eigenvecs_pos.T)):
        length = 1/np.sqrt(val)
        ax1.arrow(0, 0, vec[0]*length, vec[1]*length,
                 head_width=0.05, head_length=0.05, fc=f'C{i+2}', ec=f'C{i+2}',
                 linewidth=2, label=f'Principal Axis {i+1}')

    ax1.set_xlim(-2, 2)
    ax1.set_ylim(-2, 2)
    ax1.set_aspect('equal')
    ax1.grid(True, alpha=0.3)
    ax1.legend()
    ax1.set_title('Positive Definite: Ellipse')

    # Indefinite matrix (hyperbola)
    A_indef = np.array([[1, 0], [0, -1]])

    ax2 = axes[1]
    x = np.linspace(-3, 3, 400)
    y = np.linspace(-3, 3, 400)
    X, Y = np.meshgrid(x, y)
    Z = X**2 - Y**2

    contours = ax2.contour(X, Y, Z, levels=[-2, -1, 1, 2], colors=['red', 'red', 'blue', 'blue'])
    ax2.clabel(contours, inline=True)

    ax2.set_xlim(-3, 3)
    ax2.set_ylim(-3, 3)
    ax2.set_aspect('equal')
    ax2.grid(True, alpha=0.3)
    ax2.set_title('Indefinite: Hyperbola')

    # Semi-definite matrix (parabola)
    A_semipos = np.array([[1, 0], [0, 0]])

    ax3 = axes[2]
    Z_semi = X**2

    contours_semi = ax3.contour(X, Y, Z_semi, levels=[0.5, 1, 2, 4], colors='green')
    ax3.clabel(contours_semi, inline=True)

    ax3.set_xlim(-3, 3)
    ax3.set_ylim(-3, 3)
    ax3.set_aspect('equal')
    ax3.grid(True, alpha=0.3)
    ax3.set_title('Semi-definite: Parallel Lines')

    plt.tight_layout()
    plt.show()

positive_definite_tests()

Applications of Quadratic Forms

Optimization Problems

In unconstrained optimization, the definiteness of the Hessian matrix determines the nature of critical points.

Ellipsoids and Quadric Surfaces

The quadratic form $\mathbf{x}^T A \mathbf{x} = c$ defines various quadric surfaces.

def quadratic_forms_applications():
    """Applications of quadratic forms in practical problems"""

    print("Quadratic Forms Application Examples")
    print("=" * 50)

    # Application 1: Optimization problems
    print("Application 1: Hessian Matrix in Optimization Problems")

    # Define quadratic function f(x,y) = x² + xy + 2y² - 2x - 4y
    def f(x, y):
        return x**2 + x*y + 2*y**2 - 2*x - 4*y

    # Gradient
    def grad_f(x, y):
        return np.array([2*x + y - 2, x + 4*y - 4])

    # Hessian matrix
    H = np.array([[2, 1], [1, 4]])

    print(f"Hessian Matrix:\n{H}")

    # Determine positive definiteness
    eigenvals_H = eig(H)[0]
    print(f"Hessian Eigenvalues: {eigenvals_H}")
    print(f"Positive Definiteness: {'Positive Definite' if np.all(eigenvals_H > 0) else 'Not Positive Definite'}")

    # Find critical point
    # grad_f(x, y) = 0 => [2 1; 1 4][x; y] = [2; 4]
    critical_point = np.linalg.solve(H, np.array([2, 4]))
    print(f"Critical Point: ({critical_point[0]:.3f}, {critical_point[1]:.3f})")
    print(f"Function Value: {f(critical_point[0], critical_point[1]):.3f}")

    # Visualization
    fig = plt.figure(figsize=(15, 10))

    # Function contours
    ax1 = fig.add_subplot(221)
    x = np.linspace(-2, 4, 100)
    y = np.linspace(-2, 2, 100)
    X, Y = np.meshgrid(x, y)
    Z = f(X, Y)

    contours = ax1.contour(X, Y, Z, levels=20)
    ax1.clabel(contours, inline=True, fontsize=8)
    ax1.plot(critical_point[0], critical_point[1], 'ro', markersize=10, label='Minimum Point')

    ax1.set_xlabel('x')
    ax1.set_ylabel('y')
    ax1.legend()
    ax1.grid(True, alpha=0.3)
    ax1.set_title('Quadratic Function Contours')

    # Application 2: Covariance matrix (elliptical confidence region)
    print(f"\nApplication 2: Confidence Ellipse for Multivariate Normal Distribution")

    # Covariance matrix
    Sigma = np.array([[2, 1], [1, 1.5]])
    mu = np.array([1, 0.5])

    print(f"Covariance Matrix:\n{Sigma}")
    print(f"Mean Vector: {mu}")

    # Confidence ellipse corresponds to (x-μ)^T Σ^(-1) (x-μ) = χ²
    Sigma_inv = np.linalg.inv(Sigma)
    chi2_95 = 5.991  # 95% confidence level, 2 degrees of freedom

    ax2 = fig.add_subplot(222)

    # Generate ellipse
    theta_ellipse = np.linspace(0, 2*np.pi, 1000)
    # Standard ellipse
    standard_ellipse = np.array([np.cos(theta_ellipse), np.sin(theta_ellipse)])
    # Transform to confidence ellipse
    eigenvals_sigma, eigenvecs_sigma = eig(Sigma)
    transform_matrix = eigenvecs_sigma @ np.diag(np.sqrt(eigenvals_sigma * chi2_95))
    confidence_ellipse = transform_matrix @ standard_ellipse + mu[:, np.newaxis]

    ax2.plot(confidence_ellipse[0], confidence_ellipse[1], 'b-', linewidth=2, label='95% Confidence Ellipse')
    ax2.plot(mu[0], mu[1], 'ro', markersize=8, label='Mean')

    # Generate some sample points
    np.random.seed(42)
    samples = np.random.multivariate_normal(mu, Sigma, 100)
    ax2.scatter(samples[:, 0], samples[:, 1], alpha=0.6, s=20, label='Samples')

    ax2.set_xlabel('x₁')
    ax2.set_ylabel('x₂')
    ax2.legend()
    ax2.grid(True, alpha=0.3)
    ax2.set_title('Multivariate Normal Distribution Confidence Ellipse')
    ax2.set_aspect('equal')

    # Application 3: Quadric surface classification
    print(f"\nApplication 3: Three-Dimensional Quadric Surfaces")

    # Ellipsoid
    A_ellipsoid = np.array([[1, 0, 0], [0, 2, 0], [0, 0, 3]])

    # Hyperboloid
    A_hyperboloid = np.array([[1, 0, 0], [0, 1, 0], [0, 0, -1]])

    surfaces = [
        (A_ellipsoid, "Ellipsoid", "Positive Definite"),
        (A_hyperboloid, "Hyperboloid", "Indefinite")
    ]

    for i, (A, name, type_name) in enumerate(surfaces):
        ax = fig.add_subplot(2, 2, 3+i, projection='3d')

        # Generate grid
        u = np.linspace(-1.5, 1.5, 20)
        v = np.linspace(-1.5, 1.5, 20)
        U, V = np.meshgrid(u, v)

        if name == "Ellipsoid":
            # x²/1 + y²/2 + z²/3 = 1 => z = ±√(3(1 - x² - y²/2))
            W_pos = np.sqrt(3 * np.maximum(0, 1 - U**2 - V**2/2))
            W_neg = -W_pos

            ax.plot_surface(U, V, W_pos, alpha=0.6, color='blue')
            ax.plot_surface(U, V, W_neg, alpha=0.6, color='blue')

        else:  # Hyperboloid
            # x² + y² - z² = 1 => z = ±√(x² + y² - 1)
            mask = U**2 + V**2 >= 1
            W_pos = np.zeros_like(U)
            W_neg = np.zeros_like(U)
            W_pos[mask] = np.sqrt(U[mask]**2 + V[mask]**2 - 1)
            W_neg[mask] = -W_pos[mask]

            ax.plot_surface(U, V, W_pos, alpha=0.6, color='red')
            ax.plot_surface(U, V, W_neg, alpha=0.6, color='red')

        ax.set_xlabel('x')
        ax.set_ylabel('y')
        ax.set_zlabel('z')
        ax.set_title(f'{name} ({type_name})')

    plt.tight_layout()
    plt.show()

    # Application 4: Covariance matrix in principal component analysis
    print(f"\nApplication 4: Principal Component Analysis")

    # Generate correlated data
    np.random.seed(42)
    n_samples = 200
    X1 = np.random.randn(n_samples)
    X2 = 0.8 * X1 + 0.6 * np.random.randn(n_samples)
    X = np.column_stack([X1, X2])

    # Center the data
    X_centered = X - np.mean(X, axis=0)

    # Covariance matrix
    C = np.cov(X_centered.T)
    print(f"Covariance Matrix:\n{C}")

    # Eigenvalue decomposition
    eigenvals_C, eigenvecs_C = eig(C)
    idx = np.argsort(eigenvals_C)[::-1]
    eigenvals_C = eigenvals_C[idx]
    eigenvecs_C = eigenvecs_C[:, idx]

    print(f"Principal Component Variances: {eigenvals_C}")
    print(f"Variance Explained Ratio: {eigenvals_C / np.sum(eigenvals_C)}")

    # Visualize PCA
    plt.figure(figsize=(10, 5))

    plt.subplot(1, 2, 1)
    plt.scatter(X_centered[:, 0], X_centered[:, 1], alpha=0.6)

    # Draw principal component directions
    for i, (val, vec) in enumerate(zip(eigenvals_C, eigenvecs_C.T)):
        length = 3 * np.sqrt(val)
        plt.arrow(0, 0, vec[0]*length, vec[1]*length,
                 head_width=0.1, head_length=0.1, fc=f'C{i+1}', ec=f'C{i+1}',
                 linewidth=3, label=f'PC{i+1}')

    plt.xlabel('X₁')
    plt.ylabel('X₂')
    plt.title('Principal Component Analysis')
    plt.legend()
    plt.grid(True, alpha=0.3)
    plt.axis('equal')

    # Principal component coordinates
    plt.subplot(1, 2, 2)
    X_pca = X_centered @ eigenvecs_C
    plt.scatter(X_pca[:, 0], X_pca[:, 1], alpha=0.6)
    plt.xlabel(f'PC1 ({eigenvals_C[0]/np.sum(eigenvals_C)*100:.1f}%)')
    plt.ylabel(f'PC2 ({eigenvals_C[1]/np.sum(eigenvals_C)*100:.1f}%)')
    plt.title('Principal Component Coordinate System')
    plt.grid(True, alpha=0.3)
    plt.axis('equal')

    plt.tight_layout()
    plt.show()

quadratic_forms_applications()

Chapter Summary

Core Concepts Summary

Concept	Definition	Matrix Representation	Geometric Meaning
Quadratic Form	$\sum a_{ij}x_ix_j$	$\mathbf{x}^T A \mathbf{x}$	Quadric Surface
Positive Definite	All eigenvalues > 0	$A \succ 0$	Ellipsoid
Negative Definite	All eigenvalues < 0	$A \prec 0$	Inverted Ellipsoid
Indefinite	Both positive and negative eigenvalues	-	Hyperboloid
Semi-definite	Eigenvalues ≥ 0	$A \succeq 0$	Degenerate Surface

Summary of Determination Methods

🔄 正在渲染 Mermaid 图表...

graph TD
    A[Quadratic Form] --> B{Determine Type}
    B --> C[Eigenvalue Method]
    B --> D[Principal Minor Method]
    B --> E[Cholesky Decomposition]

    C --> F[Compute Eigenvalue Signs]
    D --> G[Compute Leading Principal Minors]
    E --> H[Attempt Decomposition A=LL^T]

    F --> I[Classify by Signs]
    G --> J[Positive Definite: All > 0]
    H --> K[Success: Positive Definite]
    H --> L[Failure: Not Positive Definite]

Summary of Application Areas

Optimization Theory: Hessian matrix determines extremum properties
Statistics: Covariance matrices, confidence regions
Geometry: Quadric surface classification
Machine Learning: Principal component analysis, kernel methods
Physics: Energy functions, stability analysis

Quadratic form theory is an important application of linear algebra in analysis and geometry. It perfectly combines algebraic structure with geometric intuition, providing powerful mathematical tools for optimization problems, statistical analysis, machine learning, and other fields. Through eigenvalue analysis, we can gain deep insights into the essential characteristics and geometric properties of quadratic forms.

Argo Tutorial Series

Chapter 2: Argo CD Getting Started

Chapter 3: Argo CD Advanced Configuration

Chapter 4: Argo Workflows Fundamentals

Chapter 5: Argo Workflows Advanced Features

Argo Tutorial Series

Chapter 2: Argo CD Getting Started

Chapter 3: Argo CD Advanced Configuration

Chapter 4: Argo Workflows Fundamentals

Chapter 5: Argo Workflows Advanced Features

Chapter 13: Quadratic Forms Theory

Chapter 13: Quadratic Forms Theory

Basic Concepts of Quadratic Forms

Definition

Matrix Representation

Standardization of Quadratic Forms

Completing the Square Method

Orthogonal Transformation Method

Classification of Quadratic Forms

Classification by Sign

Determination of Positive Definite Quadratic Forms

Determination Methods

Applications of Quadratic Forms

Optimization Problems

Ellipsoids and Quadric Surfaces

Chapter Summary

Core Concepts Summary

Summary of Determination Methods

Summary of Application Areas