I would say...no, it's very much the other way around.
"Bastard" is a generic insult. What could potentially hurt the individual in question is obviously the "tall" part, not the "bastard" part.
Unless I've misunderstood something here (and that is entirely possible) - no, you are being "heightist"...or whatever you want to call it: you're suggesting that the person's height is something negative, clearly.
No, it's just a description of the basterd. Let's switch tall with "baggy jeans", "You baggy jeans wearing basterd". The insult is basterd not the descriptor.