Oh boy, that could be a problem. The Lycra will stretch as you put it on and the latex is also flexible, thus rendering useless any conventional adhesive. I would suggest some type of rubber cement but I don't know how effective that would be when holding two opposing flexible surfaces together.
How thick are the latex pieces? I mean could you use some "sew 'n' stick" Velcro without deforming the latex parts? I mean standing proud of the suit or velcro "bulges".
I'm going to see my Devcon rep later next week, I'll see if he has any suggestions.
Good luck with this one,
Malc.