Yes, you'd need something soft like rubber, be it silicone, RTV or perhaps even mold builder. But you would need to cover that over with something hard like the plaster or fiberglass so that when you make a copy using your rubber mold, you will have something to help keep it's shape.
If you use just a rubber mold, the weight of the material you're using to make your copy could possibly distort the mold, which in turn would cause your casting to be distorted.