The 'Anger Management' star's ex-wife Denise Richards, who has temporary custody of his four-year-old twins, Bob and Max, with Brooke Mueller, was reportedly informed by the Los Angeles County Department of Children and Family Services (DCFS) that she was not allowed to take the children over to the actor's house for a visit.
A source told gossip website TMZ.com that officials told the 42-year-old actress that they were annoyed with Charlie after he claimed his sons were in danger and were not being protected by the department.
Bob and Max, who were removed from Mueller's care by the DCFS in May amid allegations her drug use had resulted in an "unsafe environment," spent their first over-night stay with her on Friday since she was released from rehab.
The 48-year-old Sheen blasted Mueller on Twitter on Friday as an "evil and pathetic uber loser" and slammed the DCFS as being "inept."
He also said: "She's as awful a parent and a person as you can possibly draw up."
The troubled socialite is reportedly seeking legal advice about Charlie's comments as everyone involved in the custody and guardianship case has been ordered not to discuss the situation.