He's as much a contributing factor as anyone. His best games, whilst amazing, was during a spell where the entire team was also putting their relative shifts in. Matic was a beast for a small run of games, Herrera was great to watch, Rashford was on fire, Shaw looked menacing going forward and so on. Pogba scored some pens and had a few nice goals which is expected from advanced roles. But he wasn't the only one standout in his role during Dec/Jan.
Then he was bad when others were bad. There were very few, or next to none occasions where the team was bad and he was dragging us to victory. Our 4-0 loss to Everton, the loss to Huddersfield, the big European ties against Barcelona - he would go missing in every one just like the rest of the squad.
I'm not saying he's the worst, far from that. But he's shown nothing to suggest he should be segregated from criticism. For 6 months out of the 9 hes been broadly quite poor and his trajectory has followed the rest of the squad. If they are Europa League players then so is he.