A conic equation of the type of #Ax^2+Bxy+Cy^2+Dx+Ey+F=0# is rotated by an angle #theta#, to form a new Cartesian plane with coordinates #(x',y')#, if #theta# is appropriately chosen, we can have a new equation without term #xy# i.e. of standard form.
The relation between coordinates #(x,y)# and #(x'.y')# can be expressed as
#x=x'costheta-y'sintheta# and #y=x'sintheta+y'costheta#
or #x'=xcostheta+ysintheta# and #y=-xsintheta+ycostheta#
for this we need to have #theta# given by #cot2theta=(A-C)/B#
In the given case as equation is #x^2+xy-3=0#, we have #A=1# and #B=1# and #C=0#, hence #cot2theta=1# i.e. #theta=pi/8#
Hence relation is give by #x=x'cos(pi/8)-y'sin(pi/8)# and #y=x'sin(pi/8)+y'cos(pi/8)# i.e.
#x=(x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2# and #y=(x'sqrt(2-sqrt2))/2+(y'sqrt(2+sqrt2))/2#
Hence, we get #((x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2)^2+((x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2)((x'sqrt(2-sqrt2))/2+(y'sqrt(2+sqrt2))/2)-3=0#
or #((x'^2(2+sqrt2))/4+(y'^2(2-sqrt2))/4-(2x'y'sqrt2)/4)+((x'^2sqrt2)/4-(y'^2sqrt2)/4+(x'y'(2+sqrt2))/4-(x'y'(2-sqrt2))/4)-3=0#
or #(x'^2)/4(2+2sqrt2)+(y'^2)/4(2-2sqrt2)-3=0#
The two graphs are as follows:
graph{x^2+xy-3=0 [-10, 10, -5, 5]}
and
graph{(x^2)/4(2+2sqrt2)+(y^2)/4(2-2sqrt2)-3=0 [-10, 10, -5, 5]}
