Right but UK science is genuinely amongst the best in the world. That isn't some verbal bluster, nor will it necessarily attract a German/French/Chinese Nobel laureate but its an undisputed fact.
Of course, that doesn't also mean that the Tories aren't talking nonsense or that they're wrong to link it to some kind of stupid 'points based immigration' they can only have because of Brexit. They could of course have done the same in the EU too.
Scientists don't spend their time going around interacting with the population at large, discussing politics. A scientist who decides to come to Imperial or Cambridge is employed directly by the university and will be funded for their research by the university, so I'm not sure what relevance opportunism will have in their actual day to day life. Same for a scientist at MIT or Sorbonne. I'm going to assume that there isn't much logic or reasoning in Trump's election or Xi's cult of personality or France's lurch to the right with Macron trying to outflank Le Pen on Islam or Japan's essentially one party rule for the last 60 years. India and Russia tend to be pretty high up on these lists too.
I think there's much than enough things to blame on Brexit, as well as more than enough reasons to shy away from moving to the UK, without trying to bring the logic or reasoning of the general populace into play I think. As much as its tempting post Brexit to stick the boot in in every way.