Substitute x=sint, dx=costdt. As the integrand is defined only for x∈(−1,1) then t will vary accordingly in (−π2,π2) and we must keep in mind that in this interval cost is positive.
So:
∫x3√1−x2dx=∫sin3t√1−sin2tcostdt
∫x3√1−x2dx=∫sin3tcostcostdt
∫x3√1−x2dx=∫sin3tdt
Now:
∫sin3tdt=∫sin2tsintdt
∫sin3tdt=−∫(1−cos2t)d(cost)
and using the linearity of the integral:
∫sin3tdt=∫cos2td(cost)−∫d(cost)
∫sin3tdt=cos3t3−cost+C
to undo the substitution note that:
cost=√1−sin2t=√1−x2
So:
∫x3√1−x2dx=13(1−x2)32−√1−x2+C
∫x3√1−x2dx=√1−x2(13−x23−1)+C
∫x3√1−x2dx=−√1−x2(x2+23)+C
It is however probably more straightforward not to use a trigonometric substitution, in fact:
∫x3√1−x2dx=∫x22xdx2√1−x2
and we can integrate by parts:
∫x3√1−x2dx=−∫x2d(√1−x2)
∫x3√1−x2dx=−x2(√1−x2)+2∫x√1−x2dx
and finally:
∫x3√1−x2dx=−x2(√1−x2)−∫√1−x2d(1−x2)
∫x3√1−x2dx=−x2(√1−x2)−23(1−x2)32+C
∫x3√1−x2dx=√1−x2(−x2−23+23x2)+C
∫x3√1−x2dx=−√1−x2(x2+23)+C