We use,
(a-b)^2=a^2-2ab+b^2
and a^2-b^2=(a+b)(a-b)
to factorise the denominator
x^4-8x^2+16=(x^2-4)^2
=(x+2)^2(x-2)^2
So,
(5x^3+2x^2-12x-8)/(x^4-8x^2+16)=(5x^3+2x^2-12x-8)/((x+2)^2(x-2)^2)
=A/(x+2)^2+B/(x+2)+C/(x-2)^2+D/(x-2)
=(A(x-2)^2+B(x+2)(x-2)^2+C(x+2)^2+D(x-2)(x+2)^2)/((x+2)^2(x-2)^2)
So,
5x^3+2x^2-12x-8=A(x-2)^2+B(x+2)(x-2)^2+C(x+2)^2+D(x-2)(x+2)^2
Let x=2, =>, 16=16C, =>, C=1
Let x=-2, =>, -16=16A, =>, A=-1
Coefficients of x^3, 5=B+D
And -8=4A+8B+4C-8D
-8=-4+8B+4-8D
-8=8B-8D
B-D=-1
So, B=2 and D=3
int((5x^3+2x^2-12x-8)dx)/(x^4-8x^2+16)
==int(-1dx)/(x+2)^2+int(2dx)/(x+2)+int(1dx)/(x-2)^2+int(3dx)/(x-2)
=1/(x+2)+2ln(∣x+2∣)-1/(x-2)+3ln(∣x-2∣)+C
