I think he simply got found out and didn't have the creativity to expand his knowledge.
Success with his system required us to totally dominate the opposition, camping in their half. The route to goal was just way to complicated and demanded lots of numbers to succeed. For a goal, our team had to create wide overloads that involved many movements and connective passes. To achieve this, we'd have to have lots of numbers in the opposition half, and have players who were significantly physically superior to the opposition players to limit counter attacks. This inability to dominate resulted in lots of failed attacks, and without us being physically dominant enough to quickly win defensive duels in the opposition half, easy counter attacks that would either have us concede a chance or give the opposition a chance to pin us back in our half. His system didn't work and he didn't have the skill to reframe it to suit the pace and power of the league.
Any manager should be able to have a tactic with formations and patterns of play. That doesn't mean the tactics work. A good manager should understand what they want on the pitch, and should be able to combine the right shapes, formation and patterns of play to build a successful gameplan/philosophy/tactic. Like food, different conditions...such as the pace and power of the league or the competitiveness of the league or even the players available to you, can change the effectiveness of a given tactic, and might require a manager to change or add elements to make that tactic successful. This is why Pep was able to employ inverted full backs to deal with the threat posed by the directness of the Premier League. This is why Klopp employed a 433 with a hardworking midfield to combat the pace and power of the Premier League or the reason why Ancelotti applied the diamond formation last season, as he didn't have a top quality no.9. Good managers are adaptable and know how to combine formations, patterns of play, shapes and instructions to have their teams create chances whilst remaining solid.
This is why I've always had a problem with the notion that we need xyz player for a manager to be successful. Yes, there may be key positions, but what's constantly happened is that United have accepted a manager needing a whole new team to play any variety of decent football. This is how it was so clear that Ten Hag wasn't a good manager. The first excuse he had was that Maguire was too slow to play in his system. Then Ronaldo wasn't effective enough, which prevented the whole team from playing good football. Then it was that De Gea not being a sweeper keeper hampered us tactically. Then it was that Casemiro had gotten older and no longer had the legs to hold midfield. Then it was that, not having Luke Shaw at left back completely unbalanced our team. Finally, it was that Lisandro Martinez being absent, and not have a left sided centre back stopped us from progressing the ball. Teams with far less talent than us can progress the football, can press, can pass through midfield...yet despite failing when everyone was fit, we still always fell back to these excuses to explain our poor football. Ten Hag may have known how to play attractive dominant football in the Dutch League, but he clearly didn't know how to combine the shapes, patterns, movements and formations to build a solid yet attacking tactic in the Premier League. He didn't have that level of skillset, and his in game management decisions were a weekly display of this inability. He's an example of someone who can follow great recipes, but can't actually cook.