James Denton Isn't So Sure About His TV's Sexiest Man Desperate Housewives Label