I'm sure both rabbit and beaver felt hats will last you for many many years.
I think most people who obsess about hats the way people on the rpf obsess about movie items would say beaver felt is nicer, but that doesn't mean rabbit is bad. If bespoke makers offer rabbit felt, they must have some confidence in it -and they know the materials better than we do.
Moths are probably a bigger issue than the hat getting worn out.
Contact a bespoke hatter who offers both rabbit and beaver felt and ask about the differences.