There are 19 referees that are part of the Select Group allowed to officiate Premier League games as the primary official. Obviously not all of them are bad referees. However, I'd say there's definitely some sort of correlation between how bad a referee is perceived and how high their profile is. I'd say the only exception to this in recent(ish) years has been Collina. Looking at the list, the only referees I could say with any confidence are poor referees are Michael Oliver and Mark Clattenburg, two of the more high-profile referees.
Referees get a high-profile in a few ways: being noticeably good at their job, officiating high-profile matches, and making controversial or match-changing decisions. In fairness to refs, it's pretty hard to gain recognition for doing the right thing unless it's a tight call (penalties, offsides, etc.), and even then they usually just get a bit of nod from the commentators. Conversely, it's very easy to gain a reputation for doing the wrong thing, particularly when it changes the game, and even more so when that game is high-profile. If a referee does little wrong then he's not going to have his name mentioned very often and won't get much air-time.
Clattenburg and Oliver are both given high-profile matches to officiate with relative frequency, and both are likely to finish their careers as generally well-respected referees in the eyes of the media. Basically, they'll follow a similar trajectory to Howard Webb. Just as I don't rate Oliver and Clattenburg, I didn't particularly rate Webb either. The primary reasons for this are that they all enjoy the spotlight too much, and are all very reactionary. The simple fact is, referee's shouldn't enjoy the spotlight and they shouldn't base decisions on crowd or media reaction.
Enjoying the spotlight can help refs assert control over games because they'll be confident in their ability to control the players. It's also extremely detrimental because time and time again we see the same referees jumping at the opportunity to send someone off, award a penalty, or the new favourite, dish out a random booking for diving despite often ignoring a number of other dives. Another, less obvious habit of these refs is to steal the spotlight by doing doing absolutely nothing, whether it's walking twenty yards away from an incident to call a player over, just to give him a talking to, or giving it the old Benitez x-arms to deny a penalty claim. The latter usually depends on how loudly the crowd reacts.
There's clearly a lot of bias that goes into someone's opinion of a ref, whether it's due to a decision he made that had negative consequences for the team they support, or whether the ref has recently made the wrong call on a high-profile decision. That said, I think it's unfair to accuse people have having standards that are too high or that they think all referees are bad. I'll moan a lot about the likes of Oliver and Clattenburg thinking they're the main event in a Manchester derby, but I'm unlikely to ever mention the likes of Anthony Taylor, Craig Pawson and Robert Madley because they just get on with their jobs.