Exactly, it's his last few years of being in his prime. Maybe in another position that gets rotated more it might be fine, but in defence (especially the central defence) you don't really rotate your starting pair. Ole also puts a big priority on ball-playing ability so he won't even be 3rd choice if whoever else remains is fit (Tuanzebe, Bailly, Jones?). I don't necessarily agree with Ole, but that's his decision.
Obviously we can make him stay if we want, but once a player gets to that age it's somewhat nasty to force them to stay as a backup that rarely plays if they want to go elsewhere for games. Especially if it's somebody who has been at the club for ages. Nicky Butt, Phil Neville and John O'Shea are examples. I'm sure we would have liked them to stay to provide squad depth, but they all wanted to move on as they got into their later 20's and we did the right thing by them.
I mean, if Ole has changed his mind and tells Smalling that he's going to be in a straight 50/50 battle with Lindelof for that spot then maybe things would be different, but that's not going to happen.