His passing isn't poor, but it's erratic. One minute he will play some really slick forward balls or do some nice diagonal switches of play, the next minute he will completely misplace a simple pass. For example McTominay is also erratic but never shows good vision and execution with his passing. I'm not the biggest fan of Fred's passing skills, not by any means, but he's still better than McTominay and Matic by some margin. Quality passing from deep is something we suffer from basically since Carrick retired.
The rest of your post I actually agree with. I expected more for our outlay. I do however think the energy and tenacity he brings are very underrated and hard to replace. As you well pointed out, the minute he stepped off we dropped a gear in middle of the pitch and lost all impetus. Hence I think he's the most important of our CMs despite his failings.