I think what it was missing was actually something that the narrative was intentionally trying to avoid: sympathy. De Niro's character is unlike any of the other leads in Scorsese's crime films because it was all a job to him; he did only as he was told and did it without question. Unlike Liotta's character in Goodfellas, who loved every minute of it, and we as the audiences were presented with his image of his crime days. Even in Casino, or even Mean Streets, there was always a contrast of perspectives on those involved in the underworld just to give the audience something to connect to. In case of The Irishman, it's completely dry.
While I enjoyed the movie, beyond the performances, I found it very hard to actually care about anything that happened in the movie because Sheeran (De Niro) is completely apathetic to everything around him. He does as he's told and does it dispassionately. We ultimately see him bothered by this towards the end of the film but, even then, it's almost like he's trying to feel remorseful when in fact, he doesn't really care. That's something recurring in the movie and while it works for the character, I think it ultimately puts a wall between us, as a viewer, and the movie.