Linear Algebra Unit 1
Matrices
Matrix Notation
Size
\(A_{m\times n}\) : m rows, n columns
Entry Notation
\(A=[a_{ij}]\) : entry \(a_{ij}\) is row \(i\), column \(j\)
Matrices equal ⇔ same size and same entries
Transpose
Definition
Rows become columns
\[
A=
\begin{bmatrix}
1&2&3\\
4&5&6
\end{bmatrix}
\qquad
A^T=
\begin{bmatrix}
1&4\\
2&5\\
3&6
\end{bmatrix}
\]
Symmetric Matrix
Definition
\[
C=C^T
\]
\[
c_{ij}=c_{ji}
\]
\[
C=
\begin{bmatrix}
1&2&3\\
2&4&5\\
3&5&6
\end{bmatrix}
\qquad
C^T=
\begin{bmatrix}
1&2&3\\
2&4&5\\
3&5&6
\end{bmatrix}
\]
Property
\[
(dC)^T=dC^T
\]
Matrix Operations
Addition
\[
A+B=[a_{ij}+b_{ij}]
\]
Scalar Multiplication
\[
cA=[ca_{ij}]
\]
Matrix Multiplication
\[
A_{m\times n}B_{n\times p}=AB_{m\times p}
\]
\[
(AB)_{ij}=\text{row }i(A)\cdot\text{column }j(B)
\]
Example
\[
A=
\begin{bmatrix}
a&b\\
c&d
\end{bmatrix}
\qquad
B=
\begin{bmatrix}
e&f\\
g&h
\end{bmatrix}
\]
\[
AB=
\begin{bmatrix}
ae+bg & af+bh\\
ce+dg & cf+dh
\end{bmatrix}
\]
\[
AB\ne BA
\]
Systems
Linear Combinations
System Form
\[
Ax=b
\]
Write \(A\) by columns
\[
A=[a_1\ a_2]
\]
Example
\[
A=
\begin{bmatrix}
1&2\\
3&4
\end{bmatrix},
\qquad
x=
\begin{bmatrix}
x_1\\
x_2
\end{bmatrix}
\]
\[
a_1=
\begin{bmatrix}
1\\
3
\end{bmatrix}
\qquad
a_2=
\begin{bmatrix}
2\\
4
\end{bmatrix}
\]
\[
Ax=
\begin{bmatrix}
x_1+2x_2\\
3x_1+4x_2
\end{bmatrix}
\]
\[
Ax=x_1a_1+x_2a_2
\]
Consistent system → \(b\) is a linear combination of the columns of \(A\)
Row Operations
Elementary Row Operations
swap rows
\[
R_i \leftrightarrow R_j
\]
multiply row by non-zero constant
\[
R_i \rightarrow cR_i
\]
add multiple of one row to another
\[
R_i \rightarrow R_i + cR_j
\]
REF Example
\[
\begin{bmatrix}
1&2&3\\
2&5&3\\
1&0&8
\end{bmatrix}
\]
\[
R_2 \rightarrow R_2 - 2R_1,\quad
R_3 \rightarrow R_3 - R_1
\]
\[
\begin{bmatrix}
1&2&3\\
0&1&-3\\
0&-2&5
\end{bmatrix}
\]
\[
R_3 \rightarrow R_3 + 2R_2
\]
\[
\text{REF}=
\begin{bmatrix}
1&2&3\\
0&1&-3\\
0&0&-1
\end{bmatrix}
\]
RREF
\[
R_3 \rightarrow -R_3
\]
\[
R_1 \rightarrow R_1 - 3R_3,\quad
R_2 \rightarrow R_2 + 3R_3
\]
\[
R_1 \rightarrow R_1 - 2R_2
\]
\[
\text{RREF}=
\begin{bmatrix}
1&0&0\\
0&1&0\\
0&0&1
\end{bmatrix}
\]
Matrix Inverse
\[
AA^{-1}=I
\]
Identity Matrix
\[
I=
\begin{bmatrix}
1&0&0\\
0&1&0\\
0&0&1
\end{bmatrix}
\]
Solving Systems
\[
Ax=b \Rightarrow x=A^{-1}b
\]
\[
AY=B \Rightarrow Y=A^{-1}B
\]
Finding the Inverse
\[
[A|I]\rightarrow[I|A^{-1}]
\]
LU Factorization
\[
A=LU
\]
Lower Triangular
\[
L=
\begin{bmatrix}
*&0&0\\
*&*&0\\
*&*&*
\end{bmatrix}
\]
Upper Triangular
\[
U=
\begin{bmatrix}
*&*&*\\
0&*&*\\
0&0&*
\end{bmatrix}
\]
Solve
\[
Ly=b
\]
\[
Ux=y
\]
Elementary Matrices
Elementary matrix = matrix obtained by performing a row operation on the identity matrix.
\[
A=
\begin{bmatrix}
1&2\\
3&4
\end{bmatrix}
\;\xrightarrow{R_1 \leftrightarrow R_2}\;
\begin{bmatrix}
3&4\\
1&2
\end{bmatrix}
\]
\[
I=
\begin{bmatrix}
1&0\\
0&1
\end{bmatrix}
\;\xrightarrow{R_1 \leftrightarrow R_2}\;
E=
\begin{bmatrix}
0&1\\
1&0
\end{bmatrix}
\]
\[
EA=
\begin{bmatrix}
3&4\\
1&2
\end{bmatrix}
\]
Markov / Steady State
Transition Matrix
\(P=[p_{ij}]\)
\(p_{ij}\) = probability of moving from state \(j\) to state \(i\)
Columns sum to 1
State Vector
\[
x=
\begin{bmatrix}
x_1\\
x_2\\
\vdots
\end{bmatrix}
\]
Update Rule
\[
x_{k+1}=Px_k
\]
\[
x_n = P^n x_0
\]
Steady State
\[
Px=x
\]
Solve
\[
(P-I)x=0
\]
Probability condition
\[
x_1+x_2+\dots+x_n=1
\]