I disagree.
I doubt any/many people would take offence to being described as the tall guy, the ginger guy, the guy with the beard, the guy in the wheelchair etc.
I'm sure a lot of people would take offence to being described as the fat guy, the smelly guy, the ugly guy etc, as they are all seen as societally 'bad' things to be.
I have never once felt ashamed to be black, or like my skin colour should be seen as a negative - so I have absolutely no issue in being described as the black guy in this context. I actually find it sort of comical, but I understand why it happens, when I hear people - almost always white people, if I'm being honest - tie themselves in knots to describe the only other person of a different race "Uhhh...you know...the guy...he uhhh...he was pretty tall...I think he was uhhh wearing a red t-shirt"
I've used similar terms when they're the best descriptor. "You know that white guy you were with..." Things like that.
I'm not offended by people lumping 'black' into a supposedly negative descriptor, and don't think the people that do necessarily have bad intentions....but it certainly gives me more pause for thought than simply being referred to with the most obvious descriptor.