First of all, nobody said three challenges each. If you're worried about time, give them two each and have it so that it's one per half so you only get one in the second half. That's you'd usually want to waste time anyway.
If it's a baseless appeal, ie something that's clear cut, it'll be over in five seconds (two minutes if Moss has to get his arse to the monitor, but he'll be gone soon) and no significant time will have been wasted. If it's an outrageous call, the time is lost to protests anyway. How long did it take for Gylfi to take that free kick after Fred's "handball"? It would've taken the same time for the ref to sprint to the monitor, see that it hit his knee and restart play with a drop ball, plus we would've had the right decision.
If teams challenge throw-ins and they get it right every time and get to keep their challenges, well that's not a criticism of the system, is it? It would shine a spotlight on refereeing incompetence if they call them wrongly that many times, right?