Rewriting the denominator:
#int(3x^2+5x-1)/(x^2+2x-1)^2dx=int(3x^2+5x-1)/{(x^2+2x+1)-2}^2dx=int(3x^2+5x-1)/{(x+1)^2-2}^2dx#
Let #u=x+1#, implying that #x=u-1# and #du=dx#.
#=int(3(u-1)^2+5(u-1)-1)/(u^2-2)^2du=int(3u^2-u-3)/(u^2-2)^2du#
Now let #u=sqrt2sectheta#. This implies that #u^2-2=2sec^2theta-2=2tan^2theta# and that #du=sqrt2secthetatanthetad theta#.
#=int(3(2sec^2theta)-sqrt2sectheta-3)/(2tan^2theta)^2(sqrt2secthetatanthetad theta)#
#=sqrt2/4int(6sec^3theta-sqrt2sec^2theta-3sectheta)/tan^3thetad theta#
Multiplying through by #cos^3theta/cos^3theta#:
#=1/(2sqrt2)int(6-sqrt2costheta-3cos^2theta)/sin^3thetad theta#
#=3/sqrt2intcsc^3thetad theta-1/2intcotthetacsc^2thetad theta-3/(2sqrt2)int(1-sin^2theta)/sin^3thetad theta#
Note that #3/sqrt2-3/(2sqrt2)=3/(2sqrt2)#:
#=3/(2sqrt2)intcsc^3thetad theta-1/2intcotthetacsc^2thetad theta+3/(2sqrt2)intcscthetad theta#
Find the method for integrating #csc^3theta# here.
For the second integral, let #v=cottheta# so #dv=-csc^2thetad theta#.
The integral of #csctheta# is well known. For its derivation, see here.
#=3/(2sqrt2)(-1/2)(cotthetacsctheta+lnabs(cottheta+csctheta))+1/2intvdv+3/(2sqrt2)lnabs(csctheta-cottheta)#
Note that #1/2intvdv=1/2(v^2/2)=v^2/4=cot^2theta/4#.
#=-3/(4sqrt2)cotthetacsctheta-3/(4sqrt2)lnabs(cottheta+csctheta)+3/(4sqrt2)lnabs(csctheta-cottheta)+1/4cot^2theta#
Our substitution was #u=sqrt2sectheta#, implying that #costheta=sqrt2/u#, which is a right triangle where the side adjacent to #theta# is #sqrt2#, the hypotenuse is #u#, and the side opposite #theta# is #sqrt(u^2-2)#.
Thus, #csctheta=u/sqrt(u^2-2)# and #cottheta=sqrt2/sqrt(u^2-2)#.
Also note that #-3/(4sqrt2)lnabs(cottheta+csctheta)+3/(4sqrt2)lnabs(csctheta-cottheta)=3/(4sqrt2)lnabs((csctheta-cottheta)/(cottheta+csctheta))=3/(4sqrt2)lnabs((1-costheta)/(1+costheta))#.
#=-3/(4sqrt2)(sqrt2u)/(u^2-2)+3/(4sqrt2)lnabs((1-sqrt2/u)/(1+sqrt2/u))+1/4(2/(u^2-2))#
#=-3/4(u/(u^2-2))+1/2(1/(u^2-2))+3/(4sqrt2)lnabs((u-sqrt2)/(u+sqrt2))#
#=(-3u+2)/(4(u^2-2))+3/(4sqrt2)lnabs((u-sqrt2)/(u+sqrt2))#
With #u=x+1#:
#=(-3x-1)/(4(x^2+2x-1))+3/(4sqrt2)lnabs((x-sqrt2+1)/(x+sqrt2+1))+C#
