Basic Matrix Operations (easy to use/read) • ᑌᒥᗜᕭᒥᖶᖶᒧᑎ ᒧᖴ-ᖴᒧᑎᕭᒧᐉ ᒧᖴ-ᖴᒧᑎᐅᐅᕭ

Nov 30, 2017 ⪆ 7 min read

Some Definitions

$0_{m \times n}$ denotes the $m \times n$ zero matrix, with all entries zero

$I_{n} $denotes the $ n \times n $ identity matrix, with $I_{ij}= \cases{ 1 & i = j \cr 0 & i $\ne$ j }$

For example;

\[ 0_{2 \times 3} =\left\lbrack\ \matrix{ 0 & 0 & 0 \cr 0 & 0 & 0 } \ \right\rbrack \]

and,

\[ I_{2} =\left\lbrack\ \matrix{ 1 & 0 \cr 0 & 1 }\ \ \right\rbrack \]

Transpose of Matrix

\[ \rm \bf A = \left\lbrack\ \matrix{a & b & c \cr d & e & f } \ \right\rbrack\ {⇒}\ {\rm \bf A^T} = \left\lbrack\ \matrix{ a & d \cr b & e \cr c & f } \ \right\rbrack \]

Summation of Matrices (entrywise)

\[ \left\lbrack\ \matrix{ a & b \cr c & d } \ \right\rbrack + \left\lbrack\ \matrix{ x & y \cr z & t} \ \right\rbrack = \left\lbrack\ \matrix{ a+x & b+y \cr c+z & d+t } \ \right\rbrack \]

Properties of matrix addition

commutative: $ \rm \bf A + B = B + A $
associative: $( \rm \bf A + B ) + C = A + ( B + C ) $ , so we can write as $ \rm \bf A + B + C $
$ \rm \bf A + 0 = 0 + A = A$
$ \rm \bf A −A = 0 $
$ \rm \bf ( A + B )^T = A^T + B^T $

Scalar Multiplication

$ s \left\lbrack\ \matrix{ x & y \cr z & t} \ \right\rbrack = \left\lbrack\ \matrix{ sx & sy \cr sz & st } \ \right\rbrack$

Dot Product

$a$ and $b$ are $n$ dimensional vectors;

$ a = \lbrack a_{1}, a_{2}, ⋯ , a_{n} \rbrack $

$ b = \lbrack b_{1}, b_{2}, ⋯ , b_{n} \rbrack $

Dot product of these two vectors is a scalar;

$ a \cdot b = \langle a, b \rangle = \sum\limits_{i=1}^n a_i b_i = a_1 b_1 + a_2 b_2 + ⋯ + a_n b_n $

Also; $ a \cdot b = a^T b $ , which means, same as above;

\[ a^T b = \lbrack \matrix{ a_{1} & a_{2} & ⋯ & a_{n}} \rbrack \left\lbrack\ \matrix{b_{1} \cr b_{2} \cr ⋮\cr b_{n}} \ \right\rbrack = \sum\limits_{i=1}^n a_i b_i = a_1 b_1 + a_2 b_2 + ⋯ + a_n b_n \]

Matrix - Vector Product

\[ \left \lbrack\ \matrix{ a_{11} & a_{12} & ⋯ & a_{1n} \cr a_{21} & a_{22} & ⋯ & a_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr a_{m1} & a_{m2} & ⋯ & a_{mn} } \ \right\rbrack \left\lbrack\ \matrix{ x_{1} \cr x_{2} \cr ⋮ \cr x_{n} } \ \right\rbrack = \left\lbrack\ \matrix{ a_{11}x_{1} + a_{12}x_{2} + ⋯ + a_{1n}x_{n} \cr a_{21}x_{1} + a_{22}x_{2} + ⋯ + a_{2n}x_{n} \cr ⋮ \cr a_{m1}x_{1} + a_{m2}x_{2} + ⋯ + a_{mn}x_{n} } \ \right\rbrack \]

Matrix multiplication

Matrix times matrix, namely;

$\rm \bf A_{\rm \it m \times p} B_{\rm \it p \times n} $ , outputs the matrix product $\rm \bf C_{\rm \it m \times n}$ ;

\[ \left\lbrack\ \matrix{ a_{11} & a_{12} & ⋯ & a_{1p} \cr a_{21} & a_{22} & ⋯ & a_{2p} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr a_{m1} & a_{m2} & ⋯ & a_{mp} } \ \right\rbrack \left\lbrack\ \matrix{ b_{11} & b_{12} & ⋯ & b_{1n} \cr b_{21} & b_{22} & ⋯ & b_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr b_{p1} & b_{p2} & ⋯ & b_{pm} } \ \right\rbrack = \left\lbrack\ \matrix{ c_{11} & c_{12} & ⋯ & c_{1n} \cr c_{21} & c_{22} & ⋯ & c_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr c_{m1} & c_{m2} & ⋯ & c_{mn} } \ \right\rbrack \]

The following dot product gives the value at the intersection point of that column and row for the output matrix:

\[ \lbrack \matrix{ a_{1} & a_{2} & ⋯ & a_{n}} \rbrack \left\lbrack\ \matrix{b_{1} \cr b_{2} \cr ⋮ \cr b_{n}} \ \right\rbrack = a_{1} b_{1} + a_{2} b_{2} + ⋯ + a_{n} b_{n} \]

so for the big, limitless matrices above, the result of $\rm \bf AB = C$ can be shown as;

\[\rm \bf C = \left\lbrack\ \matrix{ c_{11} = a_{row_1} \cdot b_{col_1} & c_{12} = a_{row_1} \cdot b_{col_2} & ⋯ & c_{1n} = a_{row_1} \cdot b_{col_n} \cr c_{21} = a_{row_2} \cdot b_{col_1} & c_{22} = a_{row_2} \cdot b_{col_2} & ⋯ & c_{2n} = a_{row_2} \cdot b_{col_n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr c_{m1} = a_{row_m} \cdot b_{col_1} & c_{m2} = a_{row_m} \cdot b_{col_2} & ⋯ & c_{mn} = a_{row_m} \cdot b_{col_n} } \ \right\rbrack \]

Properties of matrix multiplication

$0{\rm \bf A} = 0,$ ${\rm \bf A}0 = 0 $

here $0$ can be scalar, or a compatible matrix
$I{\rm \bf A} = {\rm \bf A},$ $ {\rm \bf A}I = {\rm \bf A} $
$\rm \bf A(BC) = (AB)C$ so we can write $\rm \bf ABC$ , but, the order must be same.
$ \alpha (\rm \bf AB) = (\alpha A ) B$ , where $\alpha$ is a scalar.
$\rm \bf A (B+C) = AB + AC, (A+B)C = AC + BC $
$\rm \bf (AB)^T = B^T A^T $

Matrix powers

$\rm \bf A$ must be $ n \times n $ sized matrix, so it can be multiplicative by itself;

$\rm \bf A^k = \underbrace{A A ⋯ A }_{k\rm\ times} $
${\rm \bf A}^0 = I$
${\rm \bf A}^k {\rm \bf A}^l = {\rm \bf A}^{k + l}$

Determinant

For a two dimensional matrix $\rm \bf A$;

\[ {\rm \bf A} = \left\lbrack\ \matrix{ a & b \cr c & d } \ \right\rbrack \]

\[ \det({\rm \bf A}) = |{\rm \bf A}| = \left| \matrix{ a & b \cr c & d } \right| = ad - bc \]

For a three dimensional matrix $\rm \bf M$;

\[ {\rm \bf M} = \left\lbrack\ \matrix{ \color{#23a493}{m_{11}} & \color{#1d4ed8}{m_{12}} & m_{13} \cr m_{21} & \color{#23a493}{m_{22}} & \color{#1d4ed8}{m_{23}} \cr \color{#1d4ed8}{m_{31}} & m_{32} & \color{#23a493}{m_{33}} } \ \right\rbrack \]

\[ \det({\rm \bf M}) = |{\rm \bf M}| = \left|\ \matrix{ \color{#23a493}{\color{#23a493}{m_{11}}} & \color{#1d4ed8}{m_{12}} & m_{13} \cr m_{21} & \color{#23a493}{m_{22}} & \color{#1d4ed8}{m_{23}} \cr \color{#1d4ed8}{m_{31}} & m_{32} & \color{#23a493}{m_{33}} }\ \right | \]

\[{ = \color{#23a493}{m_{11}} \color{#23a493}{m_{22}} \color{#23a493}{m_{33}} + \color{#1d4ed8}{m_{12}} \color{#1d4ed8}{m_{23}} \color{#1d4ed8}{m_{31}} + m_{13} m_{21} m_{32} - m_{13} \color{#23a493}{m_{22}} \color{#1d4ed8}{m_{31}} - \color{#1d4ed8}{m_{12}} m_{21} \color{#23a493}{m_{33}} - \color{#23a493}{m_{11}} \color{#1d4ed8}{m_{23}} m_{32} } \]

If the dimensions go higher, knowing how did we get above formula would help. Each element in a square matrix has its own minor. The minor is the value of the determinant of the matrix that results from crossing out the row and column of the element under consideration.

The $\color{#23a493}{m_{11}}$ ‘s minor is;

\[ \left|\ \matrix{ \color{#23a493}{m_{22}} & \color{#1d4ed8}{m_{23}} \cr m_{32} & \color{#23a493}{m_{33}} }\ \right| \]

You just take the first row and their minors to construct below;

\[ = \color{#23a493}{m_{11}} \left|\ \matrix{ \color{#23a493}{m_{22}} & \color{#1d4ed8}{m_{23}} \cr m_{32} & \color{#23a493}{m_{33}} }\ \right| - \color{#1d4ed8}{m_{12}} \left|\ \matrix{ m_{21} & \color{#1d4ed8}{m_{23}} \cr \color{#1d4ed8}{m_{31}} & \color{#23a493}{m_{33}} }\ \right| + m_{13} \left|\ \matrix{ m_{21} & \color{#23a493}{m_{22}} \cr \color{#1d4ed8}{m_{31}} & m_{32} }\ \right| \]

\[ { = \color{#23a493}{m_{11}}(\color{#23a493}{m_{22}} \color{#23a493}{m_{33}} − \color{#1d4ed8}{m_{23}} m_{32}) − \color{#1d4ed8}{m_{12}}(m_{21} \color{#23a493}{m_{33}} − \color{#1d4ed8}{m_{23}} \color{#1d4ed8}{m_{31}}) + m_{13}(m_{21} m_{32} − \color{#23a493}{m_{22}} \color{#1d4ed8}{m_{31}}) }\]

\[ { = \color{#23a493}{m_{11}} \color{#23a493}{m_{22}} \color{#23a493}{m_{33}} + \color{#1d4ed8}{m_{12}} \color{#1d4ed8}{m_{23}} \color{#1d4ed8}{m_{31}} + m_{13} m_{21} m_{32} - m_{13} \color{#23a493}{m_{22}} \color{#1d4ed8}{m_{31}} - \color{#1d4ed8}{m_{12}} m_{21} \color{#23a493}{m_{33}} - \color{#23a493}{m_{11}} \color{#1d4ed8}{m_{23}} m_{32} } \]

Matrix inverse

${\rm \bf A}$ must be $ n \times n $ sized matrix, ${\rm \bf F}$ satisfies $ {\rm \bf FA} = I $;

${\rm \bf F}$ is called the inverse of ${\rm \bf A}$, and is denoted ${{\rm \bf A}^{−1}}$, The matrix ${\rm \bf A}$ is called invertible or nonsingular.

Assuming ${\rm \bf A}$ and ${\rm \bf B}$ are invertible, $x \in \mathbf{R}^n, \alpha \ne 0 $;

$ {\rm \bf A}^{−k} = ({{\rm \bf A}^{−1}}) ^k $
$ ({{\rm \bf A}^{−1}}) ^{-1} = {\rm \bf A} $
$ ({\rm \bf AB}) ^{-1} = {\rm \bf B}^{-1}A^{-1} $
$ ({{\rm \bf A}^{T}}) ^{-1} = ({{\rm \bf A}^{−1}}) ^T$
$ I^{−1} = I $
$ (\alpha {\rm \bf A})^{−1} = (1 / \alpha ){\rm \bf A}^{−1}$
$ y = {\rm \bf A}x \Rightarrow x = {\rm \bf A}^{-1}y \Rightarrow {\rm \bf A}^{−1}y = {\rm \bf A}^{−1}{\rm \bf A}x = Ix = x$

TODO add more

Sources

Share this post on: