# Linear Algebra

## (1,0)-Tensor/Vector

$a,b:Fn c:F(a+b)i≡ai+bi,(ca)i≡cai.$

#### Example

a1 = 1, a2 = 0, a3 = 3 or a = (1,0, 3).

## Basis

$ei:Fn, ai:F,a≡(a1,⋯,an)≡ai ei,ei⋅ej≡case(i = j)(1)(0)≡(1n)i,j≡δi,j.$

#### Example

$ei1,i2 ei2,i3 ei3,i1=ei1, i3 ei3, i1=ei1, i1$

## Linear Map

$A,B⊆f,c:f,f:A→Bf(a+b)=f(a)+f(b)(additive),f(c a)=cf(a)(homogeneous).$

## Antilinear Map

$f(ax+b y)=a*f(x)+b*f(y).$

## 2-Tensor/Matrix

$a, b:Fn,m, c:f,ai,j≡(ai)j,(a = b)≡(i,j)→(ai,j = bi,j).$

#### Properties

$(a+b)i,j≡((a+b)i)j≡(ai+bi)j≡(ai)j+(bi)j≡ai,j+bi,j(additive).$$(ca)i,j≡((c a)i)j≡(c ai)j≡c (ai)j≡c ai,j(homogeneous).$$a − b≡a+(−1) b.$$(a−b)i,j≡(a+(−1) b)i,j≡ai,j+((−1) b)i,j≡ai,j+(−1) bi,j≡ai,j − bi,j.$

#### Example

a1 = (3,1) and a2 = (0,6)or a = ((3,1), (0,6)).

## Vector-Matrix Multiplication

$a:Fn,m, x:Fn, y:Fm,$

### Left Vector Multiplication

$(x a)i≡x ati.$

### Right Vector Multiplication

$(a y)i≡ai y.$

## Matrix-Matrix Multiplication

$a:Fn,m, b:Fm,l,a b:Fn,l,(a b)i,j≡ai,k bk,j≡ai⋅btj*.$

### Properties

$a (b c) = (a b) c (associative).$

$(a (b c))i,j≡ai,k (b c)k,j≡ai,k bk,l cl,j≡(a b)i,l cl,j≡((a b) c)i,j.$$(a,b) × (a b ≠ b a) (non-commutative),$$a (b+c)=a b+a c(left-distributive).$

$(a (b+c))i,j≡ai,k (b+c)k,j≡ai,k (bk,j+ck,j)≡ai,k bk,j+ai,k ck,j≡(a b)i,j+(a c)i,j≡(a b+a c)i,j.$$(a+b) c=a c+b c(right-distributive).$

$((a+b) c)i,j≡(a+b)i,k ck,j≡(ai,k+bi,k) ck,j≡ai,k ck,j+bi,k ck,j≡(a c)i,j+(b c)i,j≡(a c+b c)i,j.$

## Matrix Exponential

### Similarity

$b=p−1 a p$

### Column Matrix/Vector

$a:Fn,1,a≡((a1,1), ⋯, (an,1))$

### Row Matrix/Vector

$a:F1,n,a≡((a1,1, ⋯, a1,n),)$

### Transposition

$(at)i,j≡aj,i.$

### Identity Matrix

$1n:Fn,n,(1n)i,j≡case(i = j)(1)(0).$

### Zero Matrix

$0n:Fn,n,(0n)i,j≡0.$

### Scalar Matrix

$c:f,c 1n.$

### Diagonal

$(i ≠ j)→(ai,j = 0).$

#### Properties

$((i ≠ j)→(ai,j = 0) × (bi,j = 0))→((i ≠ j)→((a b)i,j = 0)).$

(i = j)(jk)ai,j bj,k = 0(j = k)(ij)ai,j bj,k = 0

### Symmetric Matrix

$a:Fn,n,(i,j)→(ai,j = aj,i),$

or

$a = at.$

### Properties

$((i,j)→(ai,j = aj,i) × (bi,j = bj,i))→(i,j)→((a+b)i,j = (a+b)j,i).$

$(a+b)i,j≡ai,j+bi,j = aj,i+bj,i≡(a+b)j,i$$((i,j)→(ai,j = aj,i))→(i,j)→((c a)i,j = (c a)j,i).$$((i,j)→((a b)i,j = (a b)j,i))→(a b = b a).$$((i,j)→(ai,j = aj,i))→(i,j)→(ani,j = anj,i).$$a−1=(a−1)t ≃ (a=at) × a−1 exists.$

### Lemma



(a+at)t=at+(at)t=at+a=a+at.

### Antisymmetric/skew-symmetric

$a:Fn,n,(i,j)→ai,j=−aj,i,ora=at.$$[a,b]≡a b−b a.$$[a,b]t≡(a b−b a)t≡bt at − at bt=(−b) (−a) − (−a) (−b)≡b a−a b≡−[a,b].$$exp(a)≡(n!)−1 an.$

#### Lemma

$(a−at)t=at−att=at−a=−(a−at).$

### Conjugate Transpose

$(a*)i,j≡(aj,i)*.$

### Hermitian Matrix

$a:Fn,n,(i,j)→(ai,j=(aj,i)*),or a=a*$

### Orthogonal Matrix

$a:Fn,nai,j ai,j=aj,i aj,i=(1n)i,j,or a at=at a=1n,$

### Normal Matrix

$a:Fn,nai,j (ai,j)*=(aj,i)* aj,i,or a a*=a* a.$

### Unitary Matrix

$a:Fn,nai,j (ai,j)*=(aj,i)* aj,i=(1n)i,j,or a a*=a* a=1n.$

### Lower/Left Triangular Matrix

$a:Fn,m,(i,j)→(i < j)→(ai,j=0).$

### Upper/Right Triangular Matrix

$a:Fn,m,(i,j)→(i > j)→(ai,j=0).$

## Miner

$mi,j≡det(ap ≠ i,q ≠ j).$

## Cofactor

$ci,j≡(−1)i+j mi,j.$

$a'i,j≡cj,i or a'≡ct.$

## Inverse Matrix

$(a−1)i,j≡det(a)−1 a'i,j.$

## Trace

$tr(a)≡ai,i.$

### Properties

$tr(at)=tr(a).$$tr(a b)=tr(b a).$$tr(c a+d b)=c tr(a) + d tr(b).$$tr(a bt)=a:b.$

## Eigenvalues and Eigenvectors

$a:Fn,n, x:Fn, x ≠ 0, c:f,ai,j xj=c xi,or a x=c x.$

## Levi-Civita Symbol

$εi1, i2, i3≡caseevenodd((i1, i2, i3))(1)(−1)(0).$

Alternatively

$εi1, i2, i3≡2−1 (i1 − i2) (i2 − i3) (i3 − i1).$

See .

$εi1, i2, i3 εi4, i5, i1=ei1, i4$

εi1, i2, i3 εi4, i5, i1=ei1, i4 ei2, i5ei1, i5 ei2, i4.

## Inner Product

$a⋅b≡ai bi.$

### Frobenius Inner Product

ab=ai,j bi,j=vec(a)t vec(b).

## Exterior Product

$ei1,ei2,ei3:Fn,ei1∧ei2≡εi1,i2,i3 ei3,a∧b=ai bj εi,j, k ek=εi,j, k ai bj ek$

#### Lemma

a(bc)=ai1 ei1(εi2, i3, i4 bi2 ci3 ei4)=εi2, i3, i4 ai1 bi2 ci3 (ei1ei4)=εi2, i3, i4 εi1, i4, i5 ai1 bi2 ci3 ei5=(e ee e) ... ...=(ac) b(ab) c

a,b:R3(ab)a=(ab)b=0 ||ab||=||a|| ||b||

## Outer Product

$a:Fn, b:Fm,(a⊗b)i,j≡ai bj.$$i≡(i1, ⋯, in), j≡(j1, ⋯, jm), a:Fi, b:Fj, a ⊗ b:Fi,j,(a⊗b)i,j≡ai bj.$

### Example

a=(a1, a2, a3), b=(b1, b2),ab=((a1 b1, a1 b2), (a2 b1, a2 b2), (a3 b1, a3 b2)).

### Properties

$(a⊗b)t=b ⊗ a.$

t=ba(ab)ti,j=(ab)j,i=aj bi=bi aj=(ba)i,j.

$a ⊗ (b+c)=a ⊗ b+a ⊗ c (left-distributive).$

(a(b+c))i,j=ai (b+c)j=ai (bj + cj)=ai bj + ai cj=(ab)i,j + (ac)i,j=(ab+ac)i,j.

$(a+b) ⊗ c=a ⊗ c+b ⊗ c (right-distributive).$

((a+b)c)i,j=(a+b)i cj=(ai + bi) cj=ai cj + bi cj=(ac)i,j + (bc)i,j=(ac+bc)i,j.

$c (a⊗b)=(c a) ⊗ b=a ⊗ (c b).$

(c (ab))i,j=c (ab)i,j=c ai bj=(c ai) bj=ai (c bj)

Outer product of tensors satisfies:

(ab)c=a(bc) (associative).tr(ab)=(ab)i,i=ai biaKron b=vec(bouter a)aKron bt=aouter b

## Kronecker Product

a:fn,m, b:fl,k(ab)i,jai,j b

### Example

(a1, a2)(b1, b2)=(a1 (b1, b2), a2 (b1, b2))=((a1 b1, a1 b2), (a2 b1, a2 b2))

### Properties

(ab)p (r1) + v,q (s1) + w=ar,s bv,w

## Elementwise Product

$(a∘b)i,j≡ai,jbi,j.$

### Properties

$(a∘b)i,j=ai,jbi,j=bi,jai,j=(b∘a)i,j(commutative),$

(a(bc))i,j=ai,j (bc)i,j=ai,j bi,j ci,j=(ab)i,j ci,j=((ab)c)i,j(associative),(a(b+c))i,j=ai,j (b+c)i,j=ai,j (bi,j + ci,j)=ai,j bi,j + ai,j ci,j=(ab+ac)i,j(distributive),((c a)b)i,j=(c a)i,j bi,j=c ai,j bi,j=ai,j (c b)i,j=c (ai,j bi,j),ai,j 0=0 ai,j=0 or a ∘ 0n=0na=0.

## Khatri–Rao Product

$a * b≡(ai,j ⊗ bi,j)i,j.$

### Example

a=((a1,1, a1,2), (a2,1, a2,2))a1,1=((a1,1, 1,1, a1,1, 1,2), (a1,1, 2,1, a1,1, 2,2))a1,2=((a1,2, 1,1,), (a1,2, 2,1,))a2,1=((a2,1, 1,1, a2,1, 1,2),)a2,2=((a2,2, 1,1,),)b=((b1,1, b1,2), (b2,1, b2,2))b1,1=((b1,1, 1,1,),)b1,2=((b1,2, 1,1, b1,2, 1,2),)b2,1=((b2,1, 1,1,), (b2,1, 2,1,))b2,2=((b2,2, 1,1, b2,2, 1,2), (b2,2, 2,1, b2,2, 2,2))a * b=((a1,1b1,1, a1,2b1,2),(a2,1b2,1, a2,2b2,2))

## Tracy–Singh Product

ab=(ai,jb)i,j=((ai,jbk,l)k,l)i,j

## Determinant

$det(a)≡ε* a1,i an,i.$$det(a)=ε11,⋯,1n a1,i1 an,in$

### Properties

$det(1n)=1.$$det(at)=det(a).$$det(ca)=c3 det(a).$$det(a b)=det(a) det(b).$$det(u ⊗ v)=0.$

## Characteristic Polynomial

$a:Fn,n,det(x 1n−a)$

## Matrix Decomposition

### Eigendecomposition/Spectral Decomposition

$a,q:Fn,n,(i)→aj,k xi,k=ci,i xi,j,or a=q c q−1.$

a,q:fn,n, ci,i is eigenvalue, ci,j=0 if ij,aj,k xi,k=ci,i xi,j for each i. or a=q c q1,

## Dual Space

$A⊆f,A*≡A→f.$$f,g:A*,(f+g)(a)=f(a) + g(a),(c f)(a)=c f(a).$

## Linear Form

### Integration

$I(f+g)=I(f) + I(g),I(a f)=a I(f).$

## Bilinear Form

$A,B, C⊆f, f:A × B→C,f(a+b, c)=f(a,c) + f(b,c),f(c a,b)=c f(a,b),f(a, b+c)=f(a,b) + f(a,c),f(a,c b)=c f(a,b).$

### Antisymmetric Bilinear Form

$f(a,b)=−f(a,b).$

## Bilinear Map

$A⊆f, f:A × A→ff(a+b, c)=f(a,c) + f(b,c), f(c a,b)=c f(a,b),f(a, b+c)=f(a,b) + f(a,c), f(a,c b)=c f(a,b).$

## Submatrix/block matrix

ai,j, k,l=(ai,j)k,l

## Minimal Polynomial

$a:Fn,n,μa(a)=0.$$μa(a)=0≃ det(λ 1n − a)=0≃ a x=λ x.$

## Vectorisation

vec:fm,nfm nvec(a)i=ai % m,i / m

 i i % m i / m 0 ⋯ m−1 m ⋯ m+m − 1 ⋯ (n−1) m ⋯ (n−1) m+m − 1 0 ⋯ m−1 0 ⋯ m−1 ⋯ 0 ⋯ 0 0 ⋯ 0 1 ⋯ 1 ⋯ n−1 ⋯ n−1

## Principle Invariants

$a:Fn,m,Ia≡tr(a)≡ai,i,IIa≡(tr(a)2 − tr(a2)) / 2,IIIa≡det(a),$

Da Ia=1n,Da IIa=Ia 1na,Da IIIa=IIIa a−t.