|Jennifer Connelly and her family|
The 'Broken Lines' actor – who has two children, Stellan, eight, and four-month-old Agnes, with wife Jennifer Connelly, as well as raising Kai, the actress' 14-year-old son from a previous relationship – loves being a father but admits he is always nervous about the early years of his kids' lives.
He said: "I have a very small baby. Kids are great, great – but the first three years they just scare the s**t out of you."
Paul – who has been married to Jennifer for eight years – also admits he finds it "heartbreaking" when work means he has to be separated from his family for lengthy periods of time.
He said in an interview with The Guardian newspaper: "It's so weird, so difficult and occasionally heartbreaking. I have a rule that we'll never go more than two weeks without seeing each other."
The 40-year-old star also admitted he understands why people are fascinated by his marriage because it has lasted such a long time compared with other Hollywood unions.
He said: "The thing is, we've been married for eight years, which is like 50 years in Hollywood years. So I suppose it suddenly becomes really intriguing – 'How do they do it?' "
Simon Cowell hasn't married his fiancee Mezhgan Hussainy yet because he's been so busy working on 'The X Factor'.
The music mogul has launched a US version of his pop star search show this year and he admits his attention has been focused on making his new project a success rather than think about wedding plans.
He told America's Star magazine: "I’m not going to lie - everything this year has been about working on 'The X Factor'. It has probably been the busiest year I’ve ever had … Making the choice to get married is something you just have to decide when the time is right."
Simon and make-up artist Mezhgan got engaged in February 2010 and the 51-year-old star's publicist
Max Clifford recently insisted everything was great between the couple but has no idea when they will tie the knot.
He said: "You might have to wait 20 years for Simon to plan a wedding - but they are as close as they have ever been."
The US version of 'The X Factor' launched in September and features Simon and fellow judges Paula Abdul, L.A. Reid and Nicole Scherzinger looking for a new singer.