You generally don't just screw one of them on anyway. Most are very specifically secured or in some cases (like SOCOM suppressors for M4s I shot on active duty), they're held in place with set screws.
But even if they were used like that, yeah, why would you bother not having in place when you left the plane/boat/truck to get there? That's just one more thing you might lose on the way there that could easily find it's way out of your pockets or vest/ruck.
Also, the sound is hilarious in movie, they make it seem like you won't hear anything. In real life, you can hear one an appreciable distance away. It just won't deafen you when you shoot the thing and won't being bad guys from hundreds of yards away as a normal firearm discharge would. I've heard suppressors being fired from well over a hundred of yards away, while outside, when I wasn't listening for the sound.