The answer is: 1/15(x^2+1)sqrt(x^2+1)(3x^2-2)+c.
If we make the substitution:
x=sinhtrArrdx=coshtdt, than:
intx^3sqrt(x^2+1)dx=intsinh^3tsqrt(sinh^2t+1)*coshtdt=
=intsinh^3tsqrt(cosh^2t)*coshtdt=intsinh^3tcosh^2tdt=
=intsinh^2tsinhtcosh^2tdt=int(cosh^2t-1)sinhtcosh^2tdt=
=int(sinhtcosh^4t-sinhtcosh^2t)dt=
=cosh^5t/5-cosh^3t/3+c=(1)
And, since cosht=sqrt(sinh^2t+1)=sqrt(x^2+1),
(1)=(sqrt(x^2+1))^5/5-(sqrt(x^2+1))^3/3+c=
=(x^2+1)^2sqrt(x^2+1)/5-(x^2+1)sqrt(x^2+1)/3+c=
=(x^2+1)sqrt(x^2+1)[(x^2+1)/5-1/3]+c=
=(x^2+1)sqrt(x^2+1)((3x^2+3-5)/15)+c=
=1/15(x^2+1)sqrt(x^2+1)(3x^2-2)+c.