Linear Algebra
Linear Equation
Linear Space
Linear Subspace
Affine Space
Topological Linear Space
Normed Linear Space
(1,0)-Tensor/Vector
Example
a1 = 1, a2 = 0, a3 = 3 or a = (1,0, 3).
Basis
Example
Orthonormal Basis
Gram–Schmidt Process
Linear Map
Antilinear Map
Semilinear Map
2-Tensor/Matrix
Properties
Example
a1 = (3,1) and a2 = (0,6)or a = ((3,1), (0,6)).
Vector-Matrix Multiplication
Left Vector Multiplication
Right Vector Multiplication
Matrix-Matrix Multiplication
Properties
∵
∵
∵
Matrix Exponential
Similarity
Column Matrix/Vector
Row Matrix/Vector
Transposition
Identity Matrix
Zero Matrix
Scalar Matrix
Diagonal
Properties
(i = j)→(j ≠ k)→ai,j bj,k = 0(j = k)→(i ≠ j)→ai,j bj,k = 0
Symmetric Matrix
or
Properties
∵
Lemma
(a+at)t=at+(at)t=at+a=a+at.
Antisymmetric/skew-symmetric
Lemma
Conjugate Transpose
Hermitian Matrix
Orthogonal Matrix
Normal Matrix
Unitary Matrix
Lower/Left Triangular Matrix
Upper/Right Triangular Matrix
Miner
Cofactor
Adjugate Matrix
Inverse Matrix
Generalised Inverse
Moore–Penrose Inverse
Trace
Properties
See also: tensor contraction.
Eigenvalues and Eigenvectors
Eigenspace
Jordan Normal Form
Levi-Civita Symbol
Alternatively
See .
εi1, i2, i3 εi4, i5, i1=ei1, i4 ei2, i5 − ei1, i5 ei2, i4.
Inner Product
Frobenius Inner Product
a⋅b=ai,j bi,j=vec(a)t vec(b).
Exterior Product
Lemma
a ∧ (b ∧ c)=ai1 ei1 ∧ (εi2, i3, i4 bi2 ci3 ei4)=εi2, i3, i4 ai1 bi2 ci3 (ei1 ∧ ei4)=εi2, i3, i4 εi1, i4, i5 ai1 bi2 ci3 ei5=(e e−e e) ... ...=(a⋅c) b − (a⋅b) c
a,b:R3 ⊢(a⊗b)⋅a=(a⊗b)⋅b=0 ||a ⊗ b||=||a|| ||b||
Outer Product
Example
a=(a1, a2, a3), b=(b1, b2),a ⊗ b=((a1 b1, a1 b2), (a2 b1, a2 b2), (a3 b1, a3 b2)).
Properties
∵
t=b ⊗ a(a⊗b)ti,j=(a⊗b)j,i=aj bi=bi aj=(b ⊗ a)i,j.
∵
(a ⊗ (b+c))i,j=ai (b+c)j=ai (bj + cj)=ai bj + ai cj=(a⊗b)i,j + (a ⊗ c)i,j=(a ⊗ b+a ⊗ c)i,j.
∵
((a+b) ⊗ c)i,j=(a+b)i cj=(ai + bi) cj=ai cj + bi cj=(a ⊗ c)i,j + (b⊗c)i,j=(a ⊗ c+b ⊗ c)i,j.
∵
(c (a⊗b))i,j=c (a⊗b)i,j=c ai bj=(c ai) bj=ai (c bj)
Outer product of tensors satisfies:
(a⊗b) ⊗ c=a ⊗ (b⊗c) (associative).tr(a⊗b)=(a⊗b)i,i=ai bia ⊗Kron b=vec(b ⊗outer a)a ⊗Kron bt=a ⊗outer b
Kronecker Product
a:fn,m, b:fl,k ⊢(a⊗b)i,j≡ai,j b
Example
(a1, a2) ⊗ (b1, b2)=(a1 (b1, b2), a2 (b1, b2))=((a1 b1, a1 b2), (a2 b1, a2 b2))
Properties
(a⊗b)p (r−1) + v,q (s−1) + w=ar,s bv,w
Elementwise Product
Also called Hadamard product.
Properties
(a ∘ (b∘c))i,j=ai,j (b∘c)i,j=ai,j bi,j ci,j=(a∘b)i,j ci,j=((a∘b) ∘ c)i,j(associative),(a ∘ (b+c))i,j=ai,j (b+c)i,j=ai,j (bi,j + ci,j)=ai,j bi,j + ai,j ci,j=(a ∘ b+a ∘ c)i,j(distributive),((c a) ∘ b)i,j=(c a)i,j bi,j=c ai,j bi,j=ai,j (c b)i,j=c (ai,j bi,j),ai,j 0=0 ai,j=0 or a ∘ 0n=0n ∘ a=0.
Khatri–Rao Product
Example
a=((a1,1, a1,2), (a2,1, a2,2))a1,1=((a1,1, 1,1, a1,1, 1,2), (a1,1, 2,1, a1,1, 2,2))a1,2=((a1,2, 1,1,), (a1,2, 2,1,))a2,1=((a2,1, 1,1, a2,1, 1,2),)a2,2=((a2,2, 1,1,),)b=((b1,1, b1,2), (b2,1, b2,2))b1,1=((b1,1, 1,1,),)b1,2=((b1,2, 1,1, b1,2, 1,2),)b2,1=((b2,1, 1,1,), (b2,1, 2,1,))b2,2=((b2,2, 1,1, b2,2, 1,2), (b2,2, 2,1, b2,2, 2,2))a * b=((a1,1 ⊗ b1,1, a1,2 ⊗ b1,2),(a2,1 ⊗ b2,1, a2,2 ⊗ b2,2))
Tracy–Singh Product
a ∘ b=(ai,j ∘ b)i,j=((ai,j ⊗ bk,l)k,l)i,j
Determinant
Properties
Characteristic Polynomial
Rank
Matrix Decomposition
Eigendecomposition/Spectral Decomposition
a,q:fn,n, ci,i is eigenvalue, ci,j=0 if i ≠ j,aj,k xi,k=ci,i xi,j for each i. or a=q c q−1,
Singular Value Decomposition
Dual Space
Transpose of a linear map
Linear Form
Integration
Bilinear Form
Antisymmetric Bilinear Form
Bilinear Map
Alternating Bilinear Map
Submatrix/block matrix
ai,j, k,l=(ai,j)k,l
Change of Bases
Covariance and Contravariance
Minimal Polynomial
Cayley–Hamilton Theorem
Householder Transformation
Numerical Linear Algebra
QR Decomposition
Jacobian Matrix
Hessian Matrix
Covariance and Contravariance
Active and passive transformation
Covariant transformation
Gramian matrix
Vectorisation
vec:fm,n→fm nvec(a)i=ai % m,i / m
i | 0 | ⋯ | m−1 | m | ⋯ | m+m − 1 | ⋯ | (n−1) m | ⋯ | (n−1) m+m − 1 |
---|
i % m | 0 | ⋯ | m−1 | 0 | ⋯ | m−1 | ⋯ | 0 | ⋯ | 0 |
---|
i / m | 0 | ⋯ | 0 | 1 | ⋯ | 1 | ⋯ | n−1 | ⋯ | n−1 |
---|
Principle Invariants
Da Ia=1n,Da IIa=Ia 1n − a,Da IIIa=IIIa a−t.
References