It is very tempting to start solving using angles. But I would advice you to start with distances from the origin along x and y directions (say x and y). Following steps should easily guide you to the answer.
1) Area = 1/2*(xy+ ab Sin(ArcCos([a^2+b^2-x^2-y^2]/[2ab])))
2) Using partial derivatives show x=y
3) substitute x=y in area and using first derivative of area show that
   x^2= [a^2+b^2+sqrt(2)ab]/2
4) Substituting y=x and the result from (3) Area =[a^2+b^2]/4 + ab/sqrt(2)

Edited by author 08.02.2013 15:38

1)I can compute the partial derivative of the area function.
How do you prove that x=y
2)I can compute the derivative of the area function with x=y but how do you prove that
x^2= [a^2+b^2+sqrt(2)ab]/2