Majority of habitat loss is caused not by housing but by resource use (mining, agriculture, industry).
Again, you can kill 4 billion people and not make a major dent in world-wide consumption of all those things, which are as skewed to the top as carbon emissions. As your sentence in bold says, it is the resource consumption that is the problem.
Ok. We are pretty much saying the same thing aren't we.
A percentage of that agriculture etc must be required to service the needs of the larger population.
But whatever the human population, the most important thing is for it to live in a significantly more sustainable manner. And for us all to acknowledge that our planet has evolved over 4.5 billion years not just for us to use at the expense of every other living species. But to acknowledge that every other living species has the same entertainment to live on this planet as do we.