With Britney Spears and L.A. Reid both off "The X Factor," Simon Cowell hopes the last remaining judge -- Demi Lovato -- will return for Season 3.

On this morning's "On Air With Ryan Seacrest," Simon said he enjoyed working with Demi, and that they'd "be crazy not to have her back."

Demi mentored the young adults last season, but failed to get an act in the finale. She's yet to confirm her return to the talent show, and is currently promoting her new hit single "Heart Attack."

Simon also dismissed rumors Michael Bolton was in the running to be a judge, and jokingly said he could "neither confirm or deny" outgoing Pope Benedict in the role.

In addition to the judges, Simon is looking for a host to replace Khloe Kardashian. Ryan Seacrest brought up the idea of guest hosting "The X Factor," an idea Simon found intriguing. However, both men were unsure whether the brass at FOX would want the "American Idol" host on another show.

"The X Factor" is holding auditions in Los Angeles today, and Simon encourages older singers and groups to try out (probably due to the success of his acts Susan Boyle and One Direction). The show, which has struggled in ratings, has yet to produce a top-selling act.

Do you want Demi to return to "The X Factor"? Vote below: