Changing to polar coordinates using the pass transformations
#{(x=r costheta),(y=rsin theta):}#
we have
#f(x,y)->f(r,theta)=r^2(7cos^2theta+4sintheta cos theta+3sin^2theta)#
and the condition
#x^2+y^2=1->r^2=1# then the transformed problem is
#{min,max}f(1,theta) = {min,max}(7cos^2theta+4sintheta cos theta+3sin^2theta)#
but after simplifications
#7cos^2theta+4sintheta cos theta+3sin^2theta=5+2(sin(2theta)+cos(2theta))#
so
#f(1,theta)=5+2(sin(2theta)+cos(2theta))#
The stationary points condition is
#(df)/(d theta) = 4(sin(2theta)+cos(2theta))=0# and this is attained at
#theta = 1/2(pm pi/4 + 2kpi)# for #k=0,1,2,cdots#
The stationary points qualification is done evaluating
#(d^2f)/(d theta^2) = -8(sin(2theta)+cos(2theta))# resulting in
#{min,max}={8 sqrt[2], -8 sqrt[2]}#
The values at those points #theta = {-pi/8, pi/8}# are
#m = 5 - 2 sqrt[2]# and #M = 5 + 2 sqrt[2]# so
#(m+M)/2=5#
