I think this in a roundabout way links to my general opinion. A boycott doesn't work unless it's done in mass. Players will go because their team is going, teams will go because other teams are going, media will go because other broadcasters are going, and fans/public will want to watch. A few fans giving up their TV license wouldn't be effective (would probably rile up more folks who would be angry at being unable to watch). One player, one team or one media organisation not going doesn't really work. Nor would 10, 20 or 40% even. The only effective boycott would have been to pressure FIFA to not hold it there in the first place. And for that to have been successful it wouldn't be a case of a few countries, a handful of players and pundits, and a media organisation or two, it would have needed united opposition from multiple FAs, continental associations and perhaps governments as well.
In terms of individuals going I kind of get Neville's viewpoint. If he didn't go, somebody else would. If he does go and openly and honestly criticise the country then fair play to him, but I would be surprised. Personally I think he shouldn't go and I wouldn't in his situation but ultimately it won't make a difference except to his own reputation. I think his only chance to save face is to reveal his fee and donate it to charity.