From the above diagram,
hat (11pi)/6ˆ11π6 is in IV quadrant where cos is positive.
cos ((11pi)/6) = cos (((11pi)/6) - 2pi) = cos -(pi/6) = cos (pi/6)cos(11π6)=cos((11π6)−2π)=cos−(π6)=cos(π6)
But cos (pi/6) = cos 60 = x = 1/2cos(π6)=cos60=x=12 :. color(red)(cos ((11pi)/6) = 1/2
hat (5pi)/4 is in III Quadrant where sin is negative.
sin ((5pi)/4) = sin (pi + (pi/4)) = - sin (pi/4) = - sin 45
But sin (pi/4) = 1/sqrt2
:. color(green)(sin ((5pi)/4) = y = -1/sqrt2
Returning to the given sum,
6 cos ((11pi)/6) - 2 sin ^2 ((5pi)/4) = 6 * (1/2) - 2 (-(1/sqrt2))^2
=> (6 * (1/2)) -(2* (1/2)) = 3 - 1 = 2