Here you have to integrate by parts twice to "kill" your x^2x2.
You start with the first "by parts" as:
intx^2e^(-5x)dx=x^2*e^(-5x)/(-5)-inte^(-5x)/(-5)*2xdx=∫x2e−5xdx=x2⋅e−5x−5−∫e−5x−5⋅2xdx=
Which can be written as:
=x^2*e^(-5x)/(-5)+1/5inte^(-5x)*2xdx==x2⋅e−5x−5+15∫e−5x⋅2xdx=
Now you go for another integration by parts and get:
=x^2*e^(-5x)/(-5)+1/5[e^(-5x)/(-5)*2x-inte^(-5x)/(-5)*2dx]==x2⋅e−5x−5+15[e−5x−5⋅2x−∫e−5x−5⋅2dx]=
=x^2*e^(-5x)/(-5)+1/5[e^(-5x)/(-5)*2x+2/5inte^(-5x)dx]==x2⋅e−5x−5+15[e−5x−5⋅2x+25∫e−5xdx]=
=x^2*e^(-5x)/(-5)+1/5e^(-5x)/(-5)*2x+1/5*2/5e^(-5x)/(-5)==x2⋅e−5x−5+15e−5x−5⋅2x+15⋅25e−5x−5=
=e^(-5x)[-x^2/5-(2x)/25-2/125]+c=e−5x[−x25−2x25−2125]+c