We can convert each to Cartesian:
(3, (3pi)/2) rightarrow 3e^(i(3pi)/2) = 3(-i) = -3i
(9, (5pi)/8) rightarrow 9e^(i(5pi)/8) = -(9sqrt(2-sqrt(2)))/2 + i (9sqrt(2+sqrt(2)))/2
In order to do the trig functions for (5pi)/8, I used the half angle formulae and the angle addition formulae (just to be careful about signs):
(5pi)/8 = pi/2 + pi/8
cos(pi/8) = cos(1/2 (pi/4)) = sqrt( (1+cos(pi/4)) /2) = sqrt( 2 + sqrt(2))/2
sin(pi/8) = sin(1/2 (pi/4)) = sqrt( (1-cos(pi/4)) /2) = sqrt( 2 - sqrt(2))/2
cos(pi/2 + pi/8) = - sin(pi/8) = -sqrt( 2- sqrt(2))/2
sin(pi/2 + pi/8) = cos(pi/8) = sqrt(2 + sqrt(2))/2
Now, we can just apply Pythagorean theorem:
d^2 = Delta x^2 + Delta y^2
= 81/4 * (2 - sqrt(2)) + ((9sqrt(2+sqrt2))/2 + 3)^2
= 81/4 * (2 - sqrt(2)) + 81/4 * (2+sqrt2) + 9 + 2 * 3 * 9/2 *sqrt(2+sqrt2)
= 81/2 + 18/2 + 27sqrt(2+sqrt2)
d^2= 99/2 + 27sqrt(2+sqrt2)
Which means that the distance is actually
d = sqrt(99/2 + 27sqrt(2+sqrt2)) approx 9.97