The 'Les Miserables' actor is sick of being constantly questioned about his sexuality, especially because it upsets his wife Deborra-Lee Furness.
Speaking in a joint interview with Deborra-Lee, 57, on CBS' '60 Minutes', he said: "If I was [homosexual], I would be. I don't think - to me, it's not the most interesting thing about a person anyway, but I do get frustrated for Deb, because I see Deb go, 'Ah, this is crazy.'
"On some level it's a compliment, you know, because it only happens when you've got to a certain point in your career.''
Deborra-Lee - who has been married to Hugh, 44, for 17 years and has adoptive children Oscar, 13, and Ava, seven, with him - angrily added the rumours are extremely "offensive" to their family.
Hugh has also spoken out about his constant battle with inner aggression, claiming that playing Wolverine in the 'X-Men' films and spin-off movies is a form of "therapy" for him to channel his anger.
He told the new edition of Men's Health magazine: "Nine out of 10 characteristics of Wolverine I don't share, but aggression is a primal thing and needs to be exercised in some way.
"When I was younger I was very explosive. I used to let most of it out playing rugby. I'm not as explosive now but in a sense it's still there.
"It's in our DNA ... far better to have it in a controlled violent environment. So in a way playing Wolverine is good therapy."