It didn't hurt his career. He just strikes me as ungrateful.
I think a lot of Folks are misconstruing some of Freddie's recent comments. He genuinely enjoyed his time on Rebels, that much is very evident in any interview done at the time and shortly afterwards. He loved and respected the character of Kanan, that is why he has said recently he doesn't want to do anymore with Kanan, why doing Bad Batch and the voice Cameo in Rise of Skywalker were special favors to Dave, but he wasn't thrilled to do it. Not because he hates Star Wars or hates the character or thinks its below him, but rather because he feels to continue to bring him back and have cameos and Easter Eggs undermines the character and his sacrifice.
Something also to understand, that I did not until I listened to more interviews with him, is that Freddie is a writer. Most of that is for story lines and characters for Professional Wrestling, yes, but he is actually a student of Character and story. In his mind as a writer and story teller, Kanan's story is done and to just keep trying to bring him back, even though that is the standard in Star Wars, does a disservice to the character, the story, and its impact.
This is what he actually said:
I was asked to, yeah. I didn’t necessarily want to. I feel like every time you hear Kanan’s voice since Rebels ended, it really kind of dilutes his impact.
And then when pressed by the interviewer if we would see him in Ahsoka, even after that comment, is when he said he's "Too Old" to be Kanan. Which , honestly is fair. He is now older than Kanan was, and if you don't want to dilute the impact of your character, you definitely won't put up with the de-aging CGI mess.
As for ill will towards fans, he might have or had it against the vocal segment of the fandom that mocked him and Rebels at the start, but, just last weekend I saw him at DragonCon in Atlanta where he was happy to sign pictures of Kannan, take pictures with fans, and talk about his time on the show and the character. You might want to say he only did that for the money, but, its not like the guy is a regular on the convention circuit and just trying to milk it, and I saw first hand when I got his autograph for a friend who couldn't make it that he genuinely seemed happy to see the fans and hear the feedback on how much Kanan meant to other folks in the line around me.