Linear Algebra

Contents
Linear Equation

Linear Space

Linear Subspace

Affine Space
Topological Linear Space

Normed Linear Space

(1,0)-Tensor/Vector
Example
a1 = 1, a2 = 0, a3 = 3 or a = (1,0, 3).
Basis

Example
Orthonormal Basis
Gram–Schmidt Process
Linear Map

Antilinear Map

Semilinear Map

2-Tensor/Matrix
Properties
Example
a1 = (3,1) and a2 = (0,6)or a = ((3,1), (0,6)).
Vector-Matrix Multiplication
Left Vector Multiplication
Right Vector Multiplication
Matrix-Matrix Multiplication

Properties
∵
∵
∵
Matrix Exponential

Similarity

Column Matrix/Vector
Row Matrix/Vector
Transposition

Identity Matrix
Zero Matrix
Scalar Matrix
Diagonal

Properties
(i = j)→(j ≠ k)→ai,j bj,k = 0(j = k)→(i ≠ j)→ai,j bj,k = 0
Symmetric Matrix

or
Properties
∵
Lemma
(a+at)t=at+(at)t=at+a=a+at.
Antisymmetric/skew-symmetric

Lemma
Conjugate Transpose

Hermitian Matrix

Orthogonal Matrix

Normal Matrix

Unitary Matrix

Lower/Left Triangular Matrix
Upper/Right Triangular Matrix
Miner
Cofactor
Adjugate Matrix
Inverse Matrix
Generalised Inverse

Moore–Penrose Inverse

Trace
Properties
See also: tensor contraction.
Eigenvalues and Eigenvectors

Eigenspace


Levi-Civita Symbol

Alternatively
See
.
εi1, i2, i3 εi4, i5, i1=ei1, i4 ei2, i5 − ei1, i5 ei2, i4.
Inner Product


Frobenius Inner Product

a⋅b=ai,j bi,j=vec(a)t vec(b).
Exterior Product


Lemma
a ∧ (b ∧ c)=ai1 ei1 ∧ (εi2, i3, i4 bi2 ci3 ei4)=εi2, i3, i4 ai1 bi2 ci3 (ei1 ∧ ei4)=εi2, i3, i4 εi1, i4, i5 ai1 bi2 ci3 ei5=(e e−e e) ... ...=(a⋅c) b − (a⋅b) c
a,b:R3 ⊢(a⊗b)⋅a=(a⊗b)⋅b=0 ||a ⊗ b||=||a|| ||b||
Outer Product

Example
a=(a1, a2, a3), b=(b1, b2),a ⊗ b=((a1 b1, a1 b2), (a2 b1, a2 b2), (a3 b1, a3 b2)).
Properties
∵
t=b ⊗ a(a⊗b)ti,j=(a⊗b)j,i=aj bi=bi aj=(b ⊗ a)i,j.
∵
(a ⊗ (b+c))i,j=ai (b+c)j=ai (bj + cj)=ai bj + ai cj=(a⊗b)i,j + (a ⊗ c)i,j=(a ⊗ b+a ⊗ c)i,j.
∵
((a+b) ⊗ c)i,j=(a+b)i cj=(ai + bi) cj=ai cj + bi cj=(a ⊗ c)i,j + (b⊗c)i,j=(a ⊗ c+b ⊗ c)i,j.
∵
(c (a⊗b))i,j=c (a⊗b)i,j=c ai bj=(c ai) bj=ai (c bj)
Outer product of tensors satisfies:
(a⊗b) ⊗ c=a ⊗ (b⊗c) (associative).tr(a⊗b)=(a⊗b)i,i=ai bia ⊗Kron b=vec(b ⊗outer a)a ⊗Kron bt=a ⊗outer b
Kronecker Product

a:fn,m, b:fl,k ⊢(a⊗b)i,j≡ai,j b
Example
(a1, a2) ⊗ (b1, b2)=(a1 (b1, b2), a2 (b1, b2))=((a1 b1, a1 b2), (a2 b1, a2 b2))
Properties
(a⊗b)p (r−1) + v,q (s−1) + w=ar,s bv,w
Elementwise Product

Also called Hadamard product.
Properties
(a ∘ (b∘c))i,j=ai,j (b∘c)i,j=ai,j bi,j ci,j=(a∘b)i,j ci,j=((a∘b) ∘ c)i,j(associative),(a ∘ (b+c))i,j=ai,j (b+c)i,j=ai,j (bi,j + ci,j)=ai,j bi,j + ai,j ci,j=(a ∘ b+a ∘ c)i,j(distributive),((c a) ∘ b)i,j=(c a)i,j bi,j=c ai,j bi,j=ai,j (c b)i,j=c (ai,j bi,j),ai,j 0=0 ai,j=0 or a ∘ 0n=0n ∘ a=0.
Khatri–Rao Product

Example
a=((a1,1, a1,2), (a2,1, a2,2))a1,1=((a1,1, 1,1, a1,1, 1,2), (a1,1, 2,1, a1,1, 2,2))a1,2=((a1,2, 1,1,), (a1,2, 2,1,))a2,1=((a2,1, 1,1, a2,1, 1,2),)a2,2=((a2,2, 1,1,),)b=((b1,1, b1,2), (b2,1, b2,2))b1,1=((b1,1, 1,1,),)b1,2=((b1,2, 1,1, b1,2, 1,2),)b2,1=((b2,1, 1,1,), (b2,1, 2,1,))b2,2=((b2,2, 1,1, b2,2, 1,2), (b2,2, 2,1, b2,2, 2,2))a * b=((a1,1 ⊗ b1,1, a1,2 ⊗ b1,2),(a2,1 ⊗ b2,1, a2,2 ⊗ b2,2))
Tracy–Singh Product

a ∘ b=(ai,j ∘ b)i,j=((ai,j ⊗ bk,l)k,l)i,j
Determinant
Properties
Characteristic Polynomial

Rank

Matrix Decomposition

Eigendecomposition/Spectral Decomposition

a,q:fn,n, ci,i is eigenvalue, ci,j=0 if i ≠ j,aj,k xi,k=ci,i xi,j for each i. or a=q c q−1,
Singular Value Decomposition

Dual Space

Transpose of a linear map


Integration

Bilinear Map

Alternating Bilinear Map
Submatrix/block matrix

ai,j, k,l=(ai,j)k,l
Change of Bases

Covariance and Contravariance

Minimal Polynomial

Cayley–Hamilton Theorem

Numerical Linear Algebra
QR Decomposition
Jacobian Matrix

Hessian Matrix

Covariance and Contravariance



Gramian matrix

Vectorisation

vec:fm,n→fm nvec(a)i=ai % m,i / m
i | 0 | ⋯ | m−1 | m | ⋯ | m+m − 1 | ⋯ | (n−1) m | ⋯ | (n−1) m+m − 1 |
---|
i % m | 0 | ⋯ | m−1 | 0 | ⋯ | m−1 | ⋯ | 0 | ⋯ | 0 |
---|
i / m | 0 | ⋯ | 0 | 1 | ⋯ | 1 | ⋯ | n−1 | ⋯ | n−1 |
---|
Principle Invariants
Da Ia=1n,Da IIa=Ia 1n − a,Da IIIa=IIIa a−t.
References