A source said: "It's a pain in the backside losing your phone at the best of times. Caroline's particularly panicked because she hasn't got any of the names in her phone under pseudonyms.
"She does have a password and has since been on to her network provider to triple check she can have it all blocked."
Other famous names in the phone include Olly Murs - who supported One Direction on their US tour - and Simon Cowell.
Despite having a few phones, this was reportedly her most private one.
The source added to The Sun newspaper: "The thing is, she's not sure it has been stolen. It could be lost under her bed in her flat, or down the side of the couch. But she hasn't seen the phone for a few days. She actually has a few phones -- so she can still keep in touch with people.
"The one she has lost is the one that she would rather didn't fall into the wrong hands."
Speaking previously, Caroline admitted what was reported about her relationship with Harry was "95 per cent false".
She said: "With the whole Harry situation, about 95 per cent of what was written was false. I didn't really know where to begin. But I've developed thicker skin and become more hardened to it."