The 'How I Met Your Mother' actor - who has twins Harper and Gideon, with husband David Burtka - hosted the Academy Awards for the first time last month but insists he won't take the helm again because it took up so much time.
He said: "I don't know that my family nor my soul could take it.
"It's a beast. It was fun to check off the list, but for the amount of time spent and the understandable opinionated response, I don't know that it's a delightful balance to do every year or even again."
The 41-year-old star experienced a mixed reception to his hosting stint but he insists he hasn't paid too much attention to the response.
He said: "I didn't keep up with it obsessively, but it was interesting to see just what people thought landed and didn't. realize just how much time and concession and compromise
"It's so difficult for one who's simply watching the show to and explanation has gone into almost every single thing.
"So my job was to try and keep things as light and specific to this year's set of films as possible.
"And if people are critical of that, it's a big giant platform, so I would assume that they would be."
While he isn't too worried about the audience's response, Neil is just pleased those who attended the ceremony, which took place at the Dolby Theatre in Los Angeles, seemed to have a good time.
He said: "I was glad we got through it, and I thought that those in the audience at the Dolby [Theatre] seemed to be enjoying themselves more as the show went on, when I was told that the opposite would be true.
"I was told that as the room fills, with you know it's four-fifths of the room didn't win, and you get further into the award giving they get less enthusiastic and less excited. And I felt while we were doing it that people were enjoying themselves more and more, so for that I'm happy."