That's typical haves rhetoric. The Polish are generally hard workers, but for most the reason they are working in Western Europe is that they can sell their labour cheap because the cost of living in their own country is much lower. This leads to the situation that a British worker with a British priced house and a British cost of living has to work for Polish wages to compete, and has therefore much less money to spend.
It's not much more complicated than that and it's good for the haves and bad for the have nots. Because the EU is not democratic, it is and will be used by the haves at the cost of the have nots. The EU is the answer of the haves to democracy, with all it's bad things like good wages, affordable housing, decent working hours etc. The freedom of employment is dragging the situation of workers down towards the level of the poorest EU country, which gets richer, Polish are sending 1 billion from the British economy home to Poland each year, but that's why the EU keeps expanding with poor countries. It's not like they try to convince the Norvegians or Swiss to join with a democratic EU for example.