Use the trigonometric formulas:
#cos 3x = cos(2x+x) = cos2xcosx-sin2xsinx#
#cos 3x = (cos^2x-sin^2x) cosx-2cosxsin^2x#
#cos 3x = cos^3x-3cosxsin^2x#
so:
#cos3xsin^2x = cos^3xsin^2x-3cosxsin^4x#
#cos3xsin^2x = cos^3x(1-cos^2x)-3cosxsin^4x#
#cos3xsin^2x = cos^3x- cos^5x-3cosxsin^4x#
Solve the integral separately:
#int cos^3xdx = int cosx(1-sin^2x)dx#
#int cos^3xdx = int cosxdx - int sin^2xcosxdx#
#int cos^3xdx = sinx - int sin^2xd(sinx)#
#int cos^3xdx = sinx - sin^3x/3+c_1#
then:
#int cos^5x = int cosx(1-sin^2x)^2dx#
#int cos^5x = int (1-2sin^2x+sin^4x)d(sinx)#
#int cos^5x = sinx-(2sin^3x)/3+sin^5x/5 +c_2#
and finally:
#int cosxsin^4xdx = int sin^4xd(sinx) = sin^5x/5+c_3#
Putting the partial solutions together:
#int cos3xsin^2xdx = sinx - sin^3x/3-sinx+(2sin^3x)/3-sin^5x/5-(3sin^5x)/5+C#
and simplifying:
#int cos3xsin^2xdx = sin^3x/3-(4sin^5x)/5+C#