When we integrate by parts a function of the form:
#x^nf(x)#
we normally choose #x^n# as the integral part and #f(x)# as the differential part, so that in the resulting integral we have #x^(n-1)#
In this case however
#cos^2xdx#
is not the differential of an «easy» function, so we first reduce the degree of the trigonometric function using the identity:
#cos^2x = (1+cos(2x))/2#
#int x^2cos^2xdx = int x^2(1+cos(2x))/2dx = int x^2/2dx +int x^2/2 cos(2x)dx#
Now we can solve the first integral directly:
#int x^2/2dx = x^3/6#
and the second by parts:
#int x^2/2 cos(2x)dx = 1/4 int x^2 d(sin2x) = (x^2sin(2x))/4 -1/2 int xsin(2x)dx#
and again:
#1/2 int xsin(2x)dx = -1/4 int xd(cos2x) = -(xcos(2x))/4 + 1/4 int cos2xdx = -(xcos(2x))/4 +1/8sin2x +C#
Putting it together:
#int x^2cos^2xdx = x^3/6 +(x^2sin(2x))/4 + (xcos(2x))/4 -1/8sin2x +C#