First off, a latex mold or silicone mold? Ive seen many refer to silicone as latex. Latex is not the best for a mold especially one thats going to see lots of use.
Ive made hundreds of human skulls in various materials. Rigid foam will do one of two things, bite into the silicone and not let go or break apart when attempting to demold it if your undercuts and details are anywhere near acceptably deep.
What your after is a self skinning medium firm foam if your set against first applying a nice skin coat. This sill allow you to keep some nice deep details and undercuts. But be aware soft foam can implode upon itself when degassing. This can be within minutes or a day later. Ive seen it happen with soft foams. This is why I suggest a medium firm foam and again, self skinning. The skin is important as it acts as a barrier against the silicone allowing it to release and not bite in.
Your best option is to rotocast them. You will get many parts off less material but its more work.