Contact is not a foul in football though. That is something else. If you have the correct rules in place and enforce them with VAR you can improve this situation. To do nothing and accept it as part of the game is completely wrong for me.
Right, contact isn't a foul in football, but that won't be the divers' problem it'll be the ref's one. They'll say there was contact and they fall due to it, the ref is the one who awarded it as a pen, so it'll end up being ref's problem while the divers will escape safely. You can't prove their intention of diving of course.