To be fair what he is saying is a bit more nuanced than that. Its worth going into this in a bit more detail because he has actually given quite a clear proposal of what he wants to do.
Davis thinks this is a game of chicken and ultimately they will swerve first. His position is that we organise trade deals while negotiating with the EU, so that the deals come into effect the day we actually exit. We have deals lined up with China, India, Australia, Canada, the US and others, so that as soon as we leave the EU, and are free to enter deals with other countries, we walk from one free trade area, into another, far larger one, comprising many bilateral deals. From those deals we negotiated we draw the confidence to tell the EU that if we dont get the deal we want (immigration controls) we WILL walk away from the single market. That yes, we will be disappointed to leave it, but that we are also willing to do so because we have other deals we can fall back on. This does not take into account the quality of the deals we have in place with the US etc, or what exactly they mean for consumers in this country (or exporters), but the point is, from an economic perspective, they would be there. That is the argument.
Davis believes that once they realise we will not be bullied into accepting their terms (free and unrestricted movement) and that we dont actually "need" the carrot they are dangling in front of us (the single market) because we already have a nice juicy steak (a pile of free trade deals) they will fold. The logic that they need to subject us to a punitive settlement will fall apart because them kicking us out wont hurt us anyway. We will trade, grow and prosper with or without their single market, so they might as well give us - and themselves - a good deal. There will be no point in biting off their nose to spite their face, because the face doesnt care about the nose anyway.
To his credit, while he does seem to believe the EU will ultimately back down, he acknowledges it may not. His point is that even if it doesnt we'll be OK because by the time we were actually out of the single market we would have arranged its replacement, in the form of bilateral deals with countries that together have a much bigger population than the EU, and with faster growing economies.
This is not without its problems. One I mentioned already, which is that in order to fast track a whole load of free trade deals inside two years, we are going to have to give away quite a lot in terms of consumer protections. These are the kinds of things that hold up deals. If you want to check the power of the corporations you need to carefully think through what concerns you have and how to address them, in ways that minimise the economic costs to the deal. That takes time. Agreeing to let goods be imported and exported across the board in the same way is easy and can be done fast, but that means a race to the bottom in terms of standards.
There is also the issue of whether a large number of deals can be negotiated simultaneously, despite the fact that we dont have civil servants with the requisite experience. Some of the countries that have offered to lend us negotiators to help us, like Australia, are the same countries we want to do deals with. So Im not sure how that will work. The EU is also saying not only can we not enter deals while we are a member, we cant even negotiate deals to be entered later. So that is also a problem - though I dont think that will be the reason the whole thing fails.