The cost of taxing the rich has become the central debate in the "fiscal cliff" talks.

But the costs of not getting a deal might be even worse for the country – including the wealthy, according to a new study.

The population of millionaires in the United States would fall by 315,000, or 6 percent, next year if the country goes over the fiscal cliff and the economy goes into recession as a result, according to a new study from WealthInsight. The total fortunes of millionaires would fall by $240 billion if the cliff’s combination of tax cuts and automatic reductions in government-spending were to take effect.

If the United States can solve the cliff deal and chalk up 2 percent growth next year, the number of millionaires would grow by 230,000, and their combined fortunes would soar to by over $1 trillion, WealthInsight’s study said.

That sounds like good news, but WealthInsight, the London-based wealth-research firm, said that if the threat of the cliff didn’t exist at all, the millionaire population would grow by 443,000 and their fortunes would grow by $1.6 trillion.

Still, even if the cliff is avoided, the solution could also drag down the number of millionaires.

"There also remains the possibility of a fiscal slope: a growth-sapping bargain which prompts a damaging degree of fiscal tightening," Wealth Insight said.

That uncertainty, along with looming tax hikes, will lead the wealthy to put their money next year into different types of investments. Alternative investments (hedge funds, private equity) will be the top-performing asset class for millionaires between now and 2016, according to WealthInsight.