First I don’t think they knew the size of the Ww army.
Falling back into a castle is always a last defensive resort
Meeting the enemy in battle has always been in the field whereby depending on the numbers and the way the battle is going they can maybe flank the enemy etc
There was no one coming to rescue them so they’d have nowhere to fall back to if they just started in the castle
Would they even have all fitted in the castle?
What if the NK had just knocked the walls down like the Great Wall? They’d have flooded in. At least this way they met them on their terms
So yeah it made sense to me because I’m obviously a military commander