Yes
Surely a team like Tottenham would have been a better role for him before taking on Chelsea?
Few points you're missing here:
1. The job wasn't available at the time, while the Chelsea job was.
2. Given his performance with Chelsea, there is no guarantee he would have done well at Tottenham. You can't be this caustic about Potter and think he'd do well at Spurs (a dysfunctional club in its own right that has chewed and spat out Nuno/Conte/Mourinho)
3. Expectations are huge at Spurs to win a trophy, any trophy, and stay in the CL, despite the financial constraints. Chelsea have higher expectations but that comes with more resources.
With that in mind, i don't think anyone can say the Spurs position would have been ideal for Potter, compared to Chelsea. Even with the benefit of hindsight.
He jumped from a - don't get relegated team - to a - we expect to win every game team. Those a massive differences in expectations and he wasn't ready for it. He should have gone for a Tottenham next who are more of a - we don't expect to win every week and top 4 is a nice to have team.
Managerial success is not linear. There are examples of managers who made huge leaps and succeeded. There are examples of managers who made lateral or small leaps and failed. There are no guarantees or heuristics.
Which is why I always laugh at this idea of "starting at the lower levels and working your way up". There are managers in the Championship who will never rise above that. Potter could have rejected the Chelsea job, only for him to be fired after things went south at Brighton and that's the end of his chapter.
When a top opportunity comes up, you take it. If you succeed, you've skipped the fence into elite manager territory. If you fail, you have more money in your pocket, you can go back to the clubs you were managing before, and if you do well, eventually another top 6 club will forget the prior mishap and hire you.
But again, he took the job because he was offered it instead of thinking what the best move for his career actually was. Now he'll struggle to get appointed at top half team because he's publically proven he was out of his depth in a big way. There was no rush for this job, it would have come eventually, he was trying to speed run his career for no reason.
Potter won't struggle to get an appointment below the top 6 tier. And once he does, he'll just have to show a decent level of competency before his name is in the ring again. The managerial go-around is not as punitive as you think. Mourinho has been sacked 4 times from top 4 clubs in England.