The dot product betwwen 2 vectors is
〈veca〉.〈vecc〉=∥〈veca〉∥∥〈vecc〉∥cos(veca,vecc)⟨→a⟩.⟨→c⟩=∥⟨→a⟩∥∥⟨→c⟩∥cos(→a,→c)
cos(veca,vecc)=(〈veca〉 .〈 vecc 〉 ) /(∥〈veca〉∥∥〈vecc〉∥)cos(→a,→c)=⟨→a⟩.⟨→c⟩∥∥⟨→a⟩∥∥⟨→c⟩∥∥
If 〈veca〉=〈a_1,a_2,a_3〉⟨→a⟩=⟨a1,a2,a3⟩
and 〈vecc〉=〈c_1,c_2,c_3〉⟨→c⟩=⟨c1,c2,c3⟩
The dot product is 〈veca〉.〈vecc〉=a_1c_1+a_2c_2+a_3c_3⟨→a⟩.⟨→c⟩=a1c1+a2c2+a3c3
〈vecc〉=〈veca〉-〈vecb〉 =〈2,1,8〉-〈5,7,3〉 =〈-3,-6,5〉⟨→c⟩=⟨→a⟩−⟨→b⟩=⟨2,1,8⟩−⟨5,7,3⟩=⟨−3,−6,5⟩
〈veca〉.〈vecc〉=-6+(-6)+40=28⟨→a⟩.⟨→c⟩=−6+(−6)+40=28
∥〈veca〉∥=sqrt(a_1^2+a_2^2+a_3^2)∥∥⟨→a⟩∥=√a21+a22+a23
∥〈veca〉∥=sqrt(4+1+64)=sqrt69∥∥⟨→a⟩∥=√4+1+64=√69
∥〈vecc〉∥=sqrt(9+36+25)=sqrt70∥∥⟨→c⟩∥=√9+36+25=√70
let cos(veca,vecc)=costhetacos(→a,→c)=cosθ
so the angle
costheta=28/((sqrt69)(sqrt70))=0.403cosθ=28(√69)(√70)=0.403
theta =66.2º