How do you rotate the axes to transform the equation `x^2+xy=3` into a new equation with no xy term and then find the angle of rotation?

A conic equation of the type of `Ax^2+Bxy+Cy^2+Dx+Ey+F=0` is rotated by an angle `theta`, to form a new Cartesian plane with coordinates `(x',y')`, if `theta` is appropriately chosen, we can have a new equation without term `xy` i.e. of standard form.
![http://philschatz.com/precalculus-book/contents/m49441.html]()
The relation between coordinates `(x,y)` and `(x'.y')` can be expressed as
`x=x'costheta-y'sintheta` and `y=x'sintheta+y'costheta`

or `x'=xcostheta+ysintheta` and `y=-xsintheta+ycostheta`

for this we need to have `theta` given by `cot2theta=(A-C)/B`

In the given case as equation is `x^2+xy-3=0`, we have `A=1` and `B=1` and `C=0`, hence `cot2theta=1` i.e. `theta=pi/8`

Hence relation is give by `x=x'cos(pi/8)-y'sin(pi/8)` and `y=x'sin(pi/8)+y'cos(pi/8)` i.e.

`x=(x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2` and `y=(x'sqrt(2-sqrt2))/2+(y'sqrt(2+sqrt2))/2`

Hence, we get `((x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2)^2+((x'sqrt(2+sqrt2))/2-(y'sqrt(2-sqrt2))/2)((x'sqrt(2-sqrt2))/2+(y'sqrt(2+sqrt2))/2)-3=0`

or `((x'^2(2+sqrt2))/4+(y'^2(2-sqrt2))/4-(2x'y'sqrt2)/4)+((x'^2sqrt2)/4-(y'^2sqrt2)/4+(x'y'(2+sqrt2))/4-(x'y'(2-sqrt2))/4)-3=0`

or `(x'^2)/4(2+2sqrt2)+(y'^2)/4(2-2sqrt2)-3=0`

The two graphs are as follows:
graph{x^2+xy-3=0 [-10, 10, -5, 5]}
and
graph{(x^2)/4(2+2sqrt2)+(y^2)/4(2-2sqrt2)-3=0 [-10, 10, -5, 5]}