The government would pay out the UBI and yes, it would need to get its money from somewhere. There wouldn't necessarily be a lot of new money in it though; it depends on how it's done.
The idea of UBI is to ensure that everyone can have at least a certain level of income. Most governments have stats indicating what an annual living wage would be in different areas of the country for different household types, so that would be a reasonable figure.
Your question, then, is about distribution. One way would be to give everyone a lump sum, which simply goes on top of your income (whatever it is, if any), and is fully taxable. That's easy to administer and means there is never a delay in receiving the UBI (e.g., if you're suddenly unemployed, you don't have to first apply and wait a while before the UBI starts coming in). However, it's a little weird, since no income is taxed at 100%, so even the rich would retain some of it. It could instead be earmarked as UBI by the state and clawed back after the year depending on your total income, but that could be hard on people that have spent it already (you can't expect everyone to plan their finances that way). Therefore, the more likely way of paying out, would be to top up existing income (if any) until the threshold.
Neither variant would particularly affect inflation though. In case of the latter approach, the government would not actually be spending all that much extra. First, UBI would come in place of existing benefits programs, so that part is simply a different name for the same money. Second, while UBI payments would most likely far exceed exisiting benefits payments, the idea is that a large part of the extra cost would be recouped through less administration (many countries have a complex network of benefits programs that could entirely be removed), lower health care costs (since poverty correlates strongly with health issues), and increased economic growth (UBI would provide the financial security that would allow people to get additional training, start a company, etc.; it's been shown that entrepreneurship is higher in countries with stronger social security nets). Whatever additional cost of UBI has not yet been covered by this point won't be sufficiently significant to affect inflation.
In the former payment scenario (everyone gets a lump sum), there would be a huge additional cost to the state in year 1, but in year 2, this would immediately drop back to similar levels as in the second scenario, as then the state would start recouping excess payments through income tax.
Along the way in all this, I'vee also touched on what I think are the supposed benefits of UBI, so I hope this helps. I'm sure some people will also jump in to add further details - and correct me.