Well this is the kind of argument that's allowed men to get away with sexually assaulting and harassing women for time immemorial, yes.
Many of these cases wouldn't pass through a court of law because of the lack of proof beyond reasonable doubt, fine. A woman cannot easily prove, for example, that a man followed her into an empty public toilet, shoved her up against a wall and stuck his hand into her underwear against her will. Not unless she's clairvoyant and stuck a camera in the room beforehand. But does that happen? Of course it does. How often do you reckon the guy faces repercussions of any sort?
Due process will be gone through, that's about being dealt with under the law. In public opinion however, these things tend to come down to the balance of probabilities. If the public believe the women, then they believe them. Not much else you can say.
I mean ffs, even GOPers are cutting their ties with one of their own candidates over allegations (which he fulsomely denies, of course). Men are not the victims here.