How do you find the inverse of #A=##((1, 2, 3), (0, 1, 4), (0,0,8))#?

Given:

#((1, 2, 3), (0, 1, 4), (0, 0, 8))#

First add an extra three columns like an identity matrix to form an augmented matrix:

#((1, 2, 3, color(blue)(1), color(blue)(0), color(blue)(0)), (0, 1, 4, color(blue)(0), color(blue)(1), color(blue)(0)), (0, 0, 8, color(blue)(0), color(blue)(0), color(blue)(1)))#

Then perform a sequence of row operations to make the left hand #3xx3# submatrix look like an identity matrix.

Subtract #2xx"row"2# from #"row"1# to get:

#((1, 0, -5, color(blue)(1), color(blue)(-2), color(blue)(0)), (0, 1, 4, color(blue)(0), color(blue)(1), color(blue)(0)), (0, 0, 8, color(blue)(0), color(blue)(0), color(blue)(1)))#

Divide #"row"3# by #8# to get:

#((1, 0, -5, color(blue)(1), color(blue)(-2), color(blue)(0)), (0, 1, 4, color(blue)(0), color(blue)(1), color(blue)(0)), (0, 0, 1, color(blue)(0), color(blue)(0), color(blue)(1/8)))#

Add #5xx"row"3# to #"row"1# to get:

#((1, 0, 0, color(blue)(1), color(blue)(-2), color(blue)(5/8)), (0, 1, 4, color(blue)(0), color(blue)(1), color(blue)(0)), (0, 0, 1, color(blue)(0), color(blue)(0), color(blue)(1/8)))#

Subtract #4xx"row"3# from #"row"2# to get:

#((1, 0, 0, color(blue)(1), color(blue)(-2), color(blue)(5/8)), (0, 1, 0, color(blue)(0), color(blue)(1), color(blue)(-1/2)), (0, 0, 1, color(blue)(0), color(blue)(0), color(blue)(1/8)))#

Now the left hand #3xx3# submatrix is an identity matrix, the right hand #3xx3# submatrix gives us our inverse matrix:

#((color(blue)(1), color(blue)(-2), color(blue)(5/8)), (color(blue)(0), color(blue)(1), color(blue)(-1/2)), (color(blue)(0), color(blue)(0), color(blue)(1/8)))#

This method works for matrices of any size.