Chapter 1: Introduction to Linear Algebra Fundamentals

The Importance of Linear Algebra

Linear algebra is one of the core branches of modern mathematics, with important applications in data science, machine learning, computer graphics, physics, engineering, and many other fields. It provides powerful tools for handling multidimensional data and linear relationships.

Basic Concept Definitions

Scalar

A scalar is a single numerical value, usually represented by lowercase letters such as $a$, $b$, $c$, etc.

import numpy as np

# Scalar examples
scalar_a = 5
scalar_b = -2.5
scalar_c = 3.14159

print(f"Scalar a = {scalar_a}")
print(f"Scalar b = {scalar_b}")
print(f"Scalar c = {scalar_c}")

Vector

A vector is an ordered array, usually represented by bold lowercase letters or letters with arrows, such as $\mathbf{v}$ or $\vec{v}$.

# Vector examples
vector_2d = np.array([3, 4])  # 2D vector
vector_3d = np.array([1, -2, 5])  # 3D vector
vector_nd = np.array([2, -1, 0, 3, 7])  # n-dimensional vector

print(f"2D vector: {vector_2d}")
print(f"3D vector: {vector_3d}")
print(f"5D vector: {vector_nd}")

# Vector magnitude (norm)
magnitude_2d = np.linalg.norm(vector_2d)
print(f"Magnitude of 2D vector: {magnitude_2d}")

Matrix

A matrix is a two-dimensional array, usually represented by uppercase letters such as $A$, $B$, $C$, etc.

# Matrix examples
matrix_2x2 = np.array([[1, 2],
                       [3, 4]])

matrix_3x3 = np.array([[1, 0, 2],
                       [0, 1, -1],
                       [2, -1, 0]])

matrix_2x3 = np.array([[1, 2, 3],
                       [4, 5, 6]])

print("2×2 matrix:")
print(matrix_2x2)
print("\n3×3 matrix:")
print(matrix_3x3)
print("\n2×3 matrix:")
print(matrix_2x3)

# Matrix shape
print(f"\nMatrix shape: {matrix_2x3.shape}")  # (rows, columns)

The Concept of Linear Combination

Linear combination is one of the most fundamental concepts in linear algebra. Given vectors $\mathbf{v_1}, \mathbf{v_2}, ..., \mathbf{v_n}$ and scalars $c_1, c_2, ..., c_n$, then:

$\mathbf{w} = c_1\mathbf{v_1} + c_2\mathbf{v_2} + ... + c_n\mathbf{v_n}$

is called a linear combination of these vectors.

# Linear combination example
v1 = np.array([1, 2])
v2 = np.array([3, 1])

# Coefficients
c1 = 2
c2 = -1

# Calculate linear combination
linear_combination = c1 * v1 + c2 * v2
print(f"v1 = {v1}")
print(f"v2 = {v2}")
print(f"Linear combination {c1}*v1 + {c2}*v2 = {linear_combination}")

# Visualize linear combination
import matplotlib.pyplot as plt

plt.figure(figsize=(8, 6))
plt.arrow(0, 0, v1[0], v1[1], head_width=0.1, head_length=0.1, fc='blue', ec='blue', label='v1')
plt.arrow(0, 0, v2[0], v2[1], head_width=0.1, head_length=0.1, fc='red', ec='red', label='v2')
plt.arrow(0, 0, linear_combination[0], linear_combination[1],
          head_width=0.1, head_length=0.1, fc='green', ec='green', label='2v1 - v2')

plt.grid(True, alpha=0.3)
plt.axis('equal')
plt.legend()
plt.title('Linear Combination of Vectors')
plt.xlabel('x')
plt.ylabel('y')
plt.show()

Intuitive Understanding of Linear Dependence

Linearly Independent

If no vector in a set of vectors can be represented as a linear combination of other vectors, then this set of vectors is said to be linearly independent.

# Linearly independent vector set
v1 = np.array([1, 0])  # Unit vector in x-axis direction
v2 = np.array([0, 1])  # Unit vector in y-axis direction

print("Linearly independent vector set:")
print(f"v1 = {v1}")
print(f"v2 = {v2}")
print("These two vectors are linearly independent because they are not on the same line")

Linearly Dependent

If at least one vector in a set of vectors can be represented as a linear combination of other vectors, then this set of vectors is said to be linearly dependent.

# Linearly dependent vector set
v1 = np.array([2, 1])
v2 = np.array([4, 2])  # v2 = 2 * v1
v3 = np.array([6, 3])  # v3 = 3 * v1

print("Linearly dependent vector set:")
print(f"v1 = {v1}")
print(f"v2 = {v2} = 2 * v1")
print(f"v3 = {v3} = 3 * v1")
print("These three vectors are linearly dependent because they are all on the same line")

Geometric Intuition

graph TB
    A[Scalar] --> B[One-dimensional number line]
    C[2D Vector] --> D[Arrow in a plane]
    E[3D Vector] --> F[Arrow in space]
    G[Matrix] --> H[Linear transformation]

    I[Linear Combination] --> J[Vector addition and scalar multiplication]
    K[Linearly Independent] --> L[Vectors not collinear/coplanar]
    M[Linearly Dependent] --> N[Vectors collinear/coplanar]

Chapter Summary

This chapter introduced the fundamental concepts of linear algebra:

These fundamental concepts lay a solid foundation for subsequent learning of vector spaces, matrix operations, linear transformations, and other topics. Understanding their geometric meaning helps establish intuitive mathematical thinking.