Linear Algebra Cheat Sheet By The WeSolveThem Team
Table of Contents Book Symbols ..................................................................................................................................... 4 Rank of matrix and pivots ................................................................................................................... 5 Length of a vector and the unit vector ................................................................................................ 6 Solutions of Augmented Matrices ....................................................................................................... 6 Coefficient Matrix ...................................................................................................................................................................................... 7 Unique Solution .......................................................................................................................................................................................... 7 Infinite Solution .......................................................................................................................................................................................... 7 No Solution ................................................................................................................................................................................................... 7 Solving System of Equations ............................................................................................................... 8 Gauss Jordan Augmented Matrix ........................................................................................................ 9 Row Operation Rules and Guidelines for Solve a System of Matrices .................................................. 9 Echelon Forms: EF, REF, RREF ........................................................................................................... 11 Echelon Form ........................................................................................................................................................................................... 11 Reduced Echelon Form ........................................................................................................................................................................ 11 Reduced Row Echelon Form .............................................................................................................................................................. 11 Linear Dependence ........................................................................................................................... 12 Linear combination ................................................................................................................................................................................ 12 Ex 1: Set u,v,w Linearly Dependent ................................................................................................................................................ 12 Ex 2: Set u,v,w Linearly Independent ............................................................................................................................................. 13 Ex 3: Vectors Linearly Independent ............................................................................................................................................... 13 Ex 4: Vectors Linearly D ....................................................................................................................................................................... 13 ependent ..................................................................................................................................................................................................... 13 Ex 5: Polynomials .................................................................................................................................................................................... 14 Ex 6: (M_(2x2)) ........................................................................................................................................................................................ 14 Column Space - Row Space - Null Space - Kernel ............................................................................... 15 Identify Row Space ................................................................................................................................................................................. 15 Identify Column Space .......................................................................................................................................................................... 15 Null Space (Kernel) ................................................................................................................................................................................ 16 LUD Decomposition and Elementary Matrices .................................................................................. 16 Transpose ......................................................................................................................................... 17 Symmetric matrix for A=LDU=LDL^T ................................................................................................. 18 Matrix addition and subtraction ....................................................................................................... 19 Multiply the matrices (2x2)(2x3) ....................................................................................................... 20 Matrix Multiplication (mxn)(nxp) ..................................................................................................... 20 Idempotent matrix ........................................................................................................................... 22 Rotation and Translate ..................................................................................................................... 22 Ex. 1 ............................................................................................................................................................................................................... 22 Ex. 2 ............................................................................................................................................................................................................... 23 Rotate about a point c, d .................................................................................................................. 24
2
Nilpotent matrix (eigenvalues are zero) ............................................................................................ 25 Determinant rules ............................................................................................................................ 26 Proofs .............................................................................................................................................. 26 Determinate’s of a (2x2) matrix ........................................................................................................ 27 Determinate of a (3x3) and higher matrices ...................................................................................... 28 Cofactor Expansion ................................................................................................................................................................................ 28 Vector Space, Subspace and Subset .................................................................................................. 30 Cramer’s rules .................................................................................................................................. 31 Basis coordinate vector .................................................................................................................... 32 Ex.1 ................................................................................................................................................................................................................ 32 Ex. 2 ............................................................................................................................................................................................................... 32 Adjugate of a matrix ......................................................................................................................... 33 Compute the Adjugate .......................................................................................................................................................................... 34 Inverse of a 2x2 Matrix ..................................................................................................................... 35 Inverse of 3x3 ................................................................................................................................... 36 Trace ................................................................................................................................................ 37 Cholesky Decomposition .................................................................................................................. 38 Eigenvalues ...................................................................................................................................... 39 Eigenvectors ..................................................................................................................................... 40 Diagonlize a Matrix .......................................................................................................................... 41 Singular Value Decomposition .......................................................................................................... 41 System of differential equations ....................................................................................................... 42 Linear Programming: Simplex Method .............................................................................................. 43
3
Book Symbols
Note: Some symbols may have different meanings in different courses i.e. never assume. And ∧ Or ∨ Row reduction occurred ~ Implies ⇒ Becomes ⇐ If and only if ⇔ Therefore ∴ Because ∵
4
Rank of matrix and pivots
𝟏 , 1
𝟏 0
𝟏 0 0
𝟏 , 0
𝟏
0 , 1
0 0 , 𝟏
𝑟𝑎𝑛𝑘 𝐴< = 2 𝑟𝑎𝑛𝑘 𝐴> = 2 𝑟𝑎𝑛𝑘 𝐴@ = 2
𝟏 1 1 𝟏 1 1 𝟏 0 0
1 1,
𝑟𝑎𝑛𝑘 𝐴3: = 1
𝑟𝑎𝑛𝑘 𝐴33 = 1
1 1 1 1 , 1 1 1 1 1 −𝟏 , 1 1 1 𝟏 0
𝑟𝑎𝑛𝑘 𝐴5 = 1
𝑟𝑎𝑛𝑘 𝐴8 = 1
𝟏 0 , 0
𝑟𝑎𝑛𝑘 𝐴; = 1
0 , 𝟏
0 𝟏
𝑟𝑎𝑛𝑘 𝐴9 = 1
1,
𝟏
𝟏 1 , 1 𝟏 0
𝑟𝑎𝑛𝑘 𝐴3 = 1
𝑟𝑎𝑛𝑘 𝐴6 = 1
1 1,
𝟏
1,
𝟏
1 1 , 𝟏
𝑟𝑎𝑛𝑘 𝐴36 = 1
𝑟𝑎𝑛𝑘 𝐴39 = 2
𝑟𝑎𝑛𝑘 𝐴3; = 3
Note: max rank is the smaller dimension of 𝑛×𝑚 e.g. 3×7 means that 3 is the highest possible rank. It goes with the transpose as well i.e. 7×3 still has a highest rank of 3. 1 2 1 1 1 1 𝑅1 + 𝑅2 ⇐ 𝑅2 𝟏 2 1 1 1 1 𝐴= ⇒ 𝑟𝑎𝑛𝑘 𝐴 = 2 0 0 𝟐 2 2 2 −1 −2 1 1 1 1 ~ 3 3 2 31 𝟏 0 0 −7 𝐴𝑥 = 𝑏 ⇒ 1 3 3 3 ~ 0 𝟏 0 8 , 𝑟𝑎𝑛𝑘 𝐴 = 3 𝑖. 𝑒. 𝐴 = 𝑓𝑢𝑙𝑙 𝑟𝑎𝑛𝑘 3 2 11 0 0 𝟏 7 0
5
Length of a vector and the unit vector
Given a vector 𝒙 = 𝑥 = 𝑥3 , 𝑥6 , 𝑥9 , … , 𝑥T
𝑥3 𝑥6 = 𝑥9 ⋮ 𝑥T
The length of the vector is the magnitude of the vector 𝑥36 + 𝑥66 + 𝑥96 + ⋯ + 𝑥T6
𝒙 ≡ 𝑥 =
Ex: Find the length of 1,2,3,4 1 2 1,2,3,4 = ⇒ 1,2,3,4 = 16 + 26 + 36 + 46 = 1 + 4 + 9 + 16 = 30 units 3 4 Ex: From the vector above, find its unit vector. 𝑣 𝒗 𝑣 𝒗 = ⇒ = = 1 units 𝑣 𝒗 𝑣 𝒗 1 𝒙 1 1,2,3,4 1 2 3 4 2 = = = , , , 𝒙 1 + 4 + 9 + 16 3 30 30 30 30 30 4 𝑥 = 𝑥
1 30
6
+
2 30
6
+
3 30
6
+
4 30
6
=
1 4 9 16 + + + = 30 30 30 30
30 = 1 units 30
Solutions of Augmented Matrices
Consider the basic scenario i.e. remember from algebra when you have 𝑎𝑥 + 𝑏𝑦 = 𝑐 and 𝑑𝑥 + 𝑒𝑦 = 𝑓? Remember that these two lines either lye on each other, intersect or never touch, and this means they have either a unique solution, infinite solutions, on no solution. The same goes with 𝑎𝑥 + 𝑏𝑦 + 𝑐𝑧 = 𝑑, except this is a plane.
6
For ℝ9 , consider the following system and its three possible solutions after reduction:
Coefficient Matrix
𝑎𝑥 + 𝑏𝑦 + 𝑐𝑧 = 𝑑 𝑎 𝑒𝑥 + 𝑓𝑦 + 𝑔𝑦 = ℎ ⇒ 𝑒 𝑖 𝑖𝑥 + 𝑗𝑦 + 𝑘𝑧 = 𝑙
𝑏 𝑓 𝑗
𝑐 𝑥 𝑎 𝑑 𝑔 𝑦 = ℎ ⇒ 𝑒 𝑘 𝑧 𝑖 𝑙
𝑎 The Coefficient Matrix = 𝑒 𝑖
𝑏 𝑓 𝑗
𝑏 𝑓 𝑗
𝑐 𝑑 𝑔ℎ 𝑘 𝑙
𝑐 𝑔 𝑘
Unique Solution
∗ 𝑥 1 0 0∗ ~ 0 1 0∗ ⇒ 𝑦 = ∗ ∗ 𝑧 0 0 1∗ In 2𝐷/3𝐷 here is a single point of intersection
Infinite Solution
∗ 𝑥 1 0 0∗ 0 ~ 0 1 0 ∗ ⇒ 𝑦 = ∗ +𝑠 0 0 𝑧 0 0 00 1 In 3𝐷 two planes lie on top of each other In 2𝐷 two lines lie on top of each other
No Solution
𝑥 ∗ 1 0 0∗ ~ 0 1 0∗ ⇒ 𝑦 = ∗ ∗ 0 0 0 0∗ Two planes/lines never touch
7
Solving System of Equations
𝑥6 + 𝑥; = 5 ∧ 𝑥9 − 4𝑥; = 4 I like to set up a matrix for this problem, and solve the matrix i.e. 𝑥6 + 𝑥; = 5 ⇒ 0𝑥3 + 𝑥6 + 0𝑥9 + 𝑥; = 5 𝑥9 − 4𝑥; = 4 ⇒ 0𝑥3 + 0𝑥6 + 𝑥9 − 4𝑥; = 4 0 010 1 5 ⇒ ⇒ 0 0 0 0 1 −4 4 0
0 0 1 0 0 1 0 0
0 1 −4 0
0 5 4 0
This helps to see the pivots, and identify that 𝑥3 ∧ 𝑥; are free variables. Which means you can set them equal to themselves. 0 0 0 0
0 0 1 0 0 1 0 0
0 1 −4 0
0 5 4 0
𝑥3 𝑥6
𝑥3 5 − 𝑥;
0 + 𝑥3 + 0𝑥; 5 + 0𝑥3 − 𝑥;
0 5
0 + 0𝑥1 + 𝑥4
4 0
⇒ 𝒙 = 𝑥 = 4 + 4𝑥 = = 4 + 0𝑥1 + 4𝑥4 3 4 𝑥4
𝑥4
+
1 0 0 0
𝑥3 +
0 −1 4 1
𝑥;
You can choose 𝑥3 ∧ 𝑥; = 𝑠 ∧ 𝑡 since they are free
∴𝒙=
0 5 4 0
+𝑠
1 0 0 0
+𝑡
0 −1 4 1
= 𝑠, 5 − 𝑡, 4 + 4𝑡, 𝑡
8
Gauss Jordan Augmented Matrix
2𝑥 − 𝑦 = 2 2 −1 2 𝑥 + 2𝑦 = 11 ⇒ 1 2 11 2𝑥 + 3𝑦 = 18 2 3 18
2 −1 2 1 𝑅2 ⇐ 𝑅1 − 𝑅2 ∧ 𝑅3 ⇐ 𝑅1 − 𝑅3 ⇒ 0 − 5 −10 2 2 0 −4 −16 2 −1 2 2 1 𝑅2 ⇐ − 𝑅2 ∧ 𝑅3 ⇐ − 𝑅3 ⇒ 0 1 4 5 4 0 1 4 2 −1 2 𝑅3 ⇐ 𝑅2 − 𝑅3 ⇒ 0 1 4 0 0 0 2 06 𝑅1 ⇐ 𝑅1 + 𝑅2 ⇒ 0 1 4 0 00 1 03 1 𝑥=3 3 𝑅1 ⇐ 𝑅1 ⇒ 0 1 4 ⇒ 𝐼6 𝒙 = ⇒ 𝑦 =4 4 2 0 00 ∴ 𝑥, 𝑦 = 3,4
Row Operation Rules and Guidelines for Solve a System of Matrices
Solve the system of equations rref [{-1/4,1,0,1},{1,0,1,2},{3,-1,1,2}] 1 1 − 𝑥+𝑦 =1 − 1 01 4 4 ⇒ 1 0 12 𝑥+𝑧 =2 3 −1 1 2 3𝑥 − 𝑦 + 𝑧 = 2 Always go in the following order unless a zero already exists and or a row operation makes it exist. ∗ ∗ ∗ ∗∗ ∗ 12 11 9 ∗ 1 0 0 0∗ ∗ ∗ 1 ∗ ∗ ∗∗ 1st 2 4 ∗ ∗ , 2nd 0 ∗ 10 8 , 3rd 0 1 0 0 ∗ ∗ 7 ∗ 0 0 0 0 1 0∗ 3 5 6 ∗∗ 0 ∗ ∗ 0 0 0 0 0 1∗
9
General allowed operations when solve a system (not the same for a matrix A)
1. Row swapping 𝑅1 ⇔ 𝑅2 (means swap row 1 with row 2) 3
2. Divide/multiply a Row 𝑅2 ∧ −3𝑅4 (means divide row 2 by 5 and multiply row 4 by -3)