I got AC when add:

if (Regex.Match(val, "[\\+\\-]?([0-9]*\\.)?[0-9]+([eE]?[\\+\\-]?[0-9]+)?", RegexOptions.Singleline).Value != val)
{
Console.WriteLine("Not a floating point number");
}

No, the correct expression is
@"^[+-]?([0-9]*\.)?[0-9]+([eE][+-]?[0-9]+)?$"
without "?" after [eE]

Your expression fails this test:
1+10
0
#

The correct answer is "Not a floating point number"