For x in (1,+oo) substitute:
x = sect
dx = sect tant dt
with t in (0,pi/2)
so:
int (2dx)/(x^3sqrt(x^2-1)) = 2 int (sect tant dt )/(sec^3tsqrt(sec^2t-1))
Now:
sec^2t -1 = tan^2t
and as for t in (0,pi/2) the tangent is positive:
sqrt(sec^2t -1) = tant
then:
int (2dx)/(x^3sqrt(x^2-1)) = 2 int (sect tant dt )/(sec^3t tant)
int (2dx)/(x^3sqrt(x^2-1)) = 2 int dt/sec^2t
int (2dx)/(x^3sqrt(x^2-1)) = 2 int cos^2tdt
Now:
2cos^2t = 1+cos2t
so:
int (2dx)/(x^3sqrt(x^2-1)) = int (1+cos2t)dt
and using linearity:
int (2dx)/(x^3sqrt(x^2-1)) = int dt +int cos2tdt
int (2dx)/(x^3sqrt(x^2-1)) =t + 1/2sin2t+C
To undo the substitution note that:
x = sect => tant = sqrt(x^2-1)
so:
t = arctan(sqrt(x^2-1))
and using the parametric fomulas:
sin 2t = (2tant)/(1+tan^2t) = (2sqrt(x^2-1))/(1+x^2-1) = (2sqrt(x^2-1))/x^2
So:
int (2dx)/(x^3sqrt(x^2-1)) = arctan(sqrt(x^2-1)) + sqrt(x^2-1)/x^2+C
By differentiating we can see that the solution is valid also for x in (-oo,-1)