We replace differentiation by $x$, y$ by multiplication on $\xi,\eta$. So $\partial_x^2 \mapsto \xi^2$ (just square); as a result senior terms like $Au_{xx}+2Bu_{xy}+ Cu_{yy}$ are replaced by quadratic form $A\xi^2+2B\xi\eta+C\eta^2$.
In the Linear Algebra you studied quadratic forms, right? And you know that
- if $AC-B^2 >0$ the canonical form is $\pm (\xi^2+\eta^2)$ (as $\pm A>0$)
- if $AC-B^2 <0$ the canonical form is $ (\xi^2-\eta^2)$,
- if $AC-B^2 =0$, but at least one of coefficients is not $0$ the canonical form is $\pm \xi^2$.