Let's put the equation in a better form before we to do the substitutions:
xy = y^2 + x(xy - 5y - 7x + 35)xy=y2+x(xy−5y−7x+35)
xy = y^2 + x^2y - 5xy - 7x^2 + 35xxy=y2+x2y−5xy−7x2+35x
Substitute rcos(theta)rcos(θ) for x and rsin(theta)rsin(θ) for y:
r^2cos(theta)sin(theta) = r^2sin^2(theta) + r^3cos^2(theta)sin(theta) - 5r^2cos(theta)sin(theta) - 7r^2cos^2(theta) + 35rcos(theta)r2cos(θ)sin(θ)=r2sin2(θ)+r3cos2(θ)sin(θ)−5r2cos(θ)sin(θ)−7r2cos2(θ)+35rcos(θ)
Divide both sides by r:
rcos(theta)sin(theta) = rsin^2(theta) + r^2cos^2(theta)sin(theta) - 5rcos(theta)sin(theta) - 7rcos^2(theta) + 35cos(theta)rcos(θ)sin(θ)=rsin2(θ)+r2cos2(θ)sin(θ)−5rcos(θ)sin(θ)−7rcos2(θ)+35cos(θ)
Put in standard quadratic form:
cos^2(theta)sin(theta)r^2 + (sin^2(theta) -6cos(theta)sin(theta)- 7cos^2(theta))r + 35cos(theta) = 0cos2(θ)sin(θ)r2+(sin2(θ)−6cos(θ)sin(θ)−7cos2(θ))r+35cos(θ)=0
The middle term factors:
cos^2(theta)sin(theta)r^2 + [{sin(theta) + cos(theta)}{sin(theta)- 7cos(theta)}]r + 35cos(theta) = 0cos2(θ)sin(θ)r2+[{sin(θ)+cos(θ)}{sin(θ)−7cos(θ)}]r+35cos(θ)=0
This is a quadratic of the form ar^2 + br + car2+br+c where:
a = cos^2(theta)sin(theta)a=cos2(θ)sin(θ)
b = [{sin(theta) + cos(theta)}{sin(theta)- 7cos(theta)}]b=[{sin(θ)+cos(θ)}{sin(θ)−7cos(θ)}]
c = 35cos(theta)c=35cos(θ)
r = (-b +-sqrt(b^2 - 4(a)(c)))/(2a)r=−b±√b2−4(a)(c)2a
r = ((-[{sin(theta) + cos(theta)}{sin(theta)- 7cos(theta)}] +-sqrt([{sin(theta) + cos(theta)}{sin(theta)- 7cos(theta)}]^2 - 140cos^3(theta)sin(theta)))/(2cos^2(theta)sin(theta)))r=⎛⎜
⎜⎝−[{sin(θ)+cos(θ)}{sin(θ)−7cos(θ)}]±√[{sin(θ)+cos(θ)}{sin(θ)−7cos(θ)}]2−140cos3(θ)sin(θ)2cos2(θ)sin(θ)⎞⎟
⎟⎠