#I=intsqrt(1-1/x^2)dx#
Let's try to get the integrand to resemble #sqrt(1-sin^2theta)#. To do so, let #x=csctheta#. Then, #dx=-cscthetacotthetad theta#, and:
#I=intsqrt(1-1/csc^2theta)(-cscthetacotthetad theta)#
#color(white)I=intsqrt(1-sin^2theta)(-cscthetacottheta)d theta#
#color(white)I=-intcostheta1/sinthetacostheta/sinthetad theta#
#color(white)I=-intcot^2thetad theta#
Use #cot^2theta=csc^2theta-1#:
#I=int(1-csc^2theta)d theta#
#color(white)I=theta+cottheta#
Reverse the substitution #x=csctheta#:
#I=theta+sqrt(csc^2theta-1)#
#color(white)I=csc^-1x+sqrt(x^2-1)+C#