If a black person comes to Spain, you have to understand not to use language that compares them to a monkey. Whether it's generally a normal expression for you or not. Because the overriding expectation is that you understand why doing so is a problem, not that the black person understands why he shouldn't think you doing so in this particular instance is racist.
If you instead decide to use that language, try to justify it as not being racist, don't apologise for your poor choice of words and then complain when you're criticised for something you could obviously be criticised for, all of that is on you. Not because you didn't properly explain what the expression means, but because you stupidly opted to use an expression that compared a black person to a monkey, something it should be beyond common sense for a grown adult not to do. And then tried to justify it instead of understanding and accepting why you were in the wrong and had said something stupid.
It's difficult to emphasise quite strongly enough just how easy it is for someone who isn't racist to avoid comparing black people to monkeys. If someone can't manage it, they're at the very least an idiot. And you can't walk around being an idiot and not expect criticism.