The Mountain had lines in season 2. At least 1 scene that I recall, possibly 2. Not a lot of screen time, but more than just the background. Plus, Ian Whyte just didn't look right. Looked too much like a super tall Serbian basketball player.
I agree on his look. My point, though, is that even though he had lines, changing the actor isn't as big a deal, given the character's prominence in the story. Next season, though, they need the right guy. It's not enough to just "pick the big guy on set" and give him 2 lines. They have to have the right look, because The Mountain features fairly prominently in the coming season, assuming it tracks to the second half of Book 2.
By this, I mean that, in Season 2, if you swapped any of the other Lannister goons in with the Mountan's character, you wouldn't dramatically change the story. Tywin tells...some goon...to go stir up trouble in the Riverlands, after giving him a stern talking to about wasting their prisoners' potential when they could be put to work. Does it matter if it's Amory Lorch, The Mountain, or some other random goon? Not really. I mean, yes, it's the Mountain who ends up doing it, but I don't think it'd dramatically change the story if the guy who says "As you command, Lord Tywin" in response to his new orders is him or any other character.
By contrast, in Season 1 and Season 4, the Mountain is prominent to the point that other characters take close note of him. At the tourney, he beheads his horse after it fails him, and after he kills the newly jumped-up knight who gave false testimony to the previous Hand or whathave you. The Hound fights him, too, and their animosity and personal history is revealed. In Season 4, as I said, he'll figure in a similarly prominent fashion.