The troubled socialite - who has twin sons Max and Bob, four, from her past relationship with the 'Anger Management' star - has worried the actor by moving out of her all-female sober living facility in San Juan Capistrano.
Now social workers are reportedly pushing for her "parental right [to] be terminated" with news she now isn't subject to random drug tests.
An insider told RadarOnline: "Brooke says she is still getting outpatient treatment at a clinic in San Jan Capistrano, but she isn't being randomly drug tested.
"Social workers are preparing a report for the judge in child dependency court and are going to recommend that Brooke's parental right be terminated, immediately."
Brooke already lost custody temporarily of Max and Bob - who are currently in the care of Charlie's other ex-wife Denise Richards - when she entered rehab and the Los Angeles County of Department of Children and Family Services are pushing for this to be permanent.
The source added: "She clearly isn't taking getting sober seriously. She claims she had to move out of the sober living house because Brooke is a celebrity and a public person and couldn't get privacy.
"This flies in the face of reason and the Los Angeles County of Department of Children and Family Services feel it's in the best interests of Bob and Max, to have Brooke's parental rights revoked."
Log in with your social accounts:
Or, log in with your IrishCentral account:
Don't have an account yet? Register now !
Join IrishCentral with your social accounts:
Already have an account ? Log in
Or, sign up for an IrishCentral account below:
Make sure we gathered the correct information from you
You already have an account on IrishCentral! Please confirm you're the owner.
Our new policy requires our users to save a first and last name. Please update your account: