First calculate the vector orthogonal to the other 2 vectors. This is given by the cross product.
| (veci,vecj,veck), (d,e,f), (g,h,i) |
where veca=〈d,e,f〉 and vecb=〈g,h,i〉 are the 2 vectors
Here, we have veca=〈-4,-5,2〉 and vecb=〈-5,4,-5〉
Therefore,
| (veci,vecj,veck), (-4,-5,2), (-5,4,-5) |
=veci| (-5,2), (4,-5) | -vecj| (-4,2), (-5,-5) | +veck| (-4,-5), (-5,4) |
=veci((-5)*(-5)-(4)*(2))-vecj((-4)*(-5)-(-5)*(2))+veck((-4)*(4)-(-5)*(-5))
=〈17,-30,-41〉=vecc
Verification by doing 2 dot products
〈17,-30,-41〉.〈-4,-5,2〉=(17)*(-4)+(-30)*(-5)+(-41)*(2)=0
〈17,-30,-41〉.〈-5,4,-5〉=(17)*(-5)+(-30)*(4)+(-41)*(-5)=0
So,
vecc is perpendicular to veca and vecb
The unit vector is
hatc=vecc/(||vecc||)=1/sqrt(17^2+(-30)^2+(-41)^2)*〈17,-30,-41〉
=1/sqrt(2870)〈17,-30,-41〉