First we expand that:
#cos(arctan(-2) + arccos(5/13)) =#
#cos(arctan(-2))cos(arccos(5/13)) - sin(arctan(-2))sin(arccos(5/13))#
We know that #cos(arccos(theta)) = theta# so we can rewrite that to
#(5cos(arctan(-2)))/13 - sin(arctan(-2))sin(arccos(5/13))#
We know that #sin^2(theta) = 1 - cos^2(theta)#, so
#sin^2(arccos(5/13)) = 1 - (5/13)^2 = 1 - 25/169 = 144/169#
Taking the root
#sin(arccos(5/13)) = 12/13#
It's positive because the sine is always positive in the arccosine range. It can let us rewrite the first expansion to:
#(5cos(arctan(-2)))/13 - (12sin(arctan(-2)))/13#
Now, from #sin^2(theta) + cos^2(theta) = 1#, if we divide both sides by #cos^2(theta)# we get #tan^2(theta) + 1 = 1/cos^2(theta)#. So:
#tan^2(arctan(-2)) + 1 = 1/cos^2(theta) rarr cos^2(theta) = 1/((-2)^2 + 1)#
#cos^2(theta) = 1/5 rarr cos(theta) = 1/sqrt(5) = sqrt(5)/5#
It's positive because on the range of the arctangent, the cosine is always postive. So we can rewrite it to:
#sqrt(5)/13 - (12sin(arctan(-2)))/13#
Since the tangent is negative, and the cosine positive, it means the sine is negative. Using #sin^2(theta) + cos^2(theta) =1# we have that
#sin^2(arctan(-2)) = 1 - 1/5 = 4/5#
#sin(arctan(-2)) = -2/sqrt(5) = -(2sqrt(5))/5#
Which means we can rewrite it all to:
#sqrt(5)/13 - (12(-2sqrt(5)/5))/13#
From there on it's algebra:
#(5sqrt(5))/65 + (24sqrt(5))/65 =#
#(29sqrt(5))/65#