A show insider told E! News: "They were after him. But there's no way he could have done it with everything he has going on."
As well as playing Blaine Anderson in 'Glee', Darren, 25, has recently finished a stint in Broadway musical 'How to Succeed in Business Without Really Trying', and has also recently been filming movie 'Imogene', with Kristen Wiig and Annette Bening.
'X Factor' boss Simon Cowell recently admitted he would love to land a host for the show whose background is not presenting.
Outrage as St. Patrick's Day deemed too religious and renamed "O'Green Day"
Top ten words invented in Ireland - useful for St. Patrick’s Day
Explaining his decision to axe Steve, he said: "With the host, too many Brits on the show, if I'm being honest with you. One Brit allowed.
"I always wanted a boy and a girl to host the show. And I like hosts who don't kind of have a hosting background, you know. I like them when they've done acting and stuff like that. It's a different kind of dynamic."
Simon is also looking for two new judges for 'The X Factor' after Paula Abdul and Nicole Scherzinger were dropped from the panel.