I used alcohol pads. The Luke FX saber cleaned up after just a few pads. The FX Vader took a lot more.. like 10-15.
I have also experienced similar issues with the MR ST:Star Trek: The Next Generation phaser grip and the butt plate of a P90 Mauri Airsoft. Alcohol pads worked every time and as stated, once the rubber broke down and was wiped off, the problem does not return.