Sorry to be pedantic but this isn't true and highlights why this is so complicated. A 4 year old male is not beating a 25 year old female at any sport, a 4 ft Male basketball player is not beating a 6ft 4 female player. There is a huge spectrum of overlap between male/female physical attributes and capabilities that are as varied as you and me (ie small, frail, hairless biologically male - tall, large frame, hairy biologically female and anything in between) therefore the most fundamental "base" separation in classification cant be Male/female.
To extend an olive branch, the "common sense of it" splitting male and female does largely group correctly over the general population, however it;
A: doesn't fit every case, therefore failing in its utility as a category especially in a professional athlete level were the edges of the curve are almost essential(human beings define categories for their utility, i'e they have a usefulness, they don't exist outside of social constructs in the vast majority of cases. we say that pieces of wood assembled in a shape we can sit on are "chairs" but chairs as a category of thing don't exist in the universe outside of our interpretation.
B: Doesn't guarantee a fair sporting event. If your aim was to guarantee a fair sporting event you would group virtually every sport into extremely precise categories. 18-24 basketball, weight range limit, height range limit, wing span limit etc.
Why do we enjoy sport, is it to gawp at freaks of nature destroying their competition? is it to try to determine who is the best based on work ethic? talent? intelligence at their sport? these are incredibly hard to nail down.
I don't know what the answer is, it clearly isn't allowing a biological male who experienced a full puberty, now compete as a female at the Olympics. But hiding behind Male/females categories also fails the litmus test.