We never do agree on this, but they left the last time because the intifada had come to a close with the factions, hezbollah in the north, actually making rare progress and general tiredness of policing Gaza. There was no peace process in 2005-06 as the US was in Iraq and Palestinian statehood was the last thing on their mind which gave Israel impunity over the next ten or so years.
Try policing 2 million impoverished people who have grown up in a prison environment surrounded by freedom fighters/(slash) terrorists who despise you because you are the ones who keep them there. That's a near impossible task.
Future of the region tells me it won't matter if it was Israel's 9/11 when it all becomes much worse in the weeks/months to come.