Integrate by parts, using dx as differential factor. As:
d/dx sin^(-1)x = 1/sqrt(1-x^2)
we will end up with an algebraic function:
int sin^(-1)(sqrt(x/(a+x)))dx = xsin^(-1)(sqrt(x/(a+x)))- int x d(sin^(-1)(sqrt(x/(a+x))))
Let's calculate the derivative separately because its a bit cumbersome:
d/dx sin^(-1)(sqrt(x/(a+x))) = 1/sqrt(1-x/(x+a)) d/dx sqrt(x/(a+x))
d/dx sin^(-1)(sqrt(x/(a+x))) = 1/sqrt((x+a-x)/(x+a)) 1/(2sqrt(x/(a+x)))d/dx (x/(a+x))
d/dx sin^(-1)(sqrt(x/(a+x))) = sqrt(x+a)/sqrta 1/(2sqrt(x/(a+x)))(x+a-x)/(x+a)^2
d/dx sin^(-1)(sqrt(x/(a+x))) = (x+a)/(2sqrtasqrtx) a/(x+a)^2
d/dx sin^(-1)(sqrt(x/(a+x))) = sqrta/(2 sqrtx (x+a))
Then:
int sin^(-1)(sqrt(x/(a+x)))dx = xsin^(-1)(sqrt(x/(a+x)))- int x sqrta/(2 sqrtx (x+a))dx
int sin^(-1)(sqrt(x/(a+x)))dx = xsin^(-1)(sqrt(x/(a+x)))- sqrta/2 int sqrtx / (x+a)dx
Solve now the resulting integral by substituting:
sqrtx = t, x = t^2, dx = 2tdt:
int sqrtx / (x+a)dx = 2 int t^2/(a+t^2)dt
int sqrtx / (x+a)dx = 2 int (a + t^2-a)/(a+t^2)dt
int sqrtx / (x+a)dx = 2 int (1-a/(a+t^2))dt
using the linearity of the integral:
int sqrtx / (x+a)dx = 2 intdt -2 a int 1/(a+t^2)dt
int sqrtx / (x+a)dx = 2 t -2 sqrta int 1/(1+(t/sqrta)^2)d(t/sqrta)
int sqrtx / (x+a)dx = 2 t -2 sqrta arctan(t/sqrta)+C
and undoing the substitution:
int sqrtx / (x+a)dx = 2 sqrtx -2 sqrta arctan(sqrt(x/a))+C
Putting the partial solutions together:
int sin^(-1)(sqrt(x/(a+x)))dx = xsin^(-1)(sqrt(x/(a+x)))- sqrtasqrtx +a arctan(sqrt(x/a))+C
