Some of the increase is in the budgets of various Federal agencies, but, to be honest, the rate of growth of my agency doesn't seem to have changed much between the Bush and Obama years, with budget growth averaging between 6 and 9% annually for all but two of those years (a 13% increase in 2010 and no increase in 2011). This is substantially faster than the rate of inflation over that time, and it reflects poorly on the willingness of Congress to restrain the growth of discretionary spending, but it's a bipartisan shame. (As an aside, I think my agency does useful work - maybe not important, in the grand scheme of things, but useful and probably well worth the expense ($312 million in fiscal 2012, much of it from industry fees). The problem is that increasing the size of the agency every year - and the agency's request for fiscal 2013 was for an additional 10 employees - is that the law of diminishing marginal returns kicks in and we end up working increasingly trivial matters. Thus, although I'd be happy to defend the agency as an entity, it's much harder to defend the incremental increases in budget.)
I think what came before should be fairly uncontroversial. Now comes the tougher sell. First, the problem is about to become worse, both because of the aging population (requiring greater Social Security and Medicare spending) and, more imminently, because historically-low interest rates won't last forever, and when interest rates increase the cost of financing the $16 trillion (and growing) national debit will increase proportionately. We can't tax our way back to fiscal health. President Obama's proposal would raise about $50 billion annually in new income taxes from wealthy people, and additional $31 billion in estate and gift tax increases, according to the Washington Post. The President has other proposals that would bring the total revenue increase to around $160 billion annually. That's a nice start, but it's not cutting the deficit in a meaningful way.
Only two things will make a substantial difference in the deficit: increase taxes substantially, across the board, or decrease spending substantially, across the board. (Or some combination of the two.) The first solution is what I'll refer to as European. It envisions are large and permanent welfare state that is financed through a very broad tax base, such as the VAT. It's true that top income tax rates are higher in most of Europe than in the U.S., although corporate tax rates are generally lower, but the real action comes not from taxing the small number of wealthy people but through taxing a large number of middle-class people, and that's what a VAT does for you.
Alternatively, we can cut spending substantially. The U.S. spends vastly more on defense than any other country in the world - I saw a claim that the U.S. spends as much as the next 20 or so countries combined. National security hawks dislike the idea of cutting defense spending to pay for social spending, but that's a possibility. Cutting discretionary spending back to the levels of, say, 2008 and then limiting growth to the rate of inflation would be a big help. It's not as though we were hurting for federal spending in '08. That would require about a 22% cut in my agency's budget, which would be painful for everyone if done at once, but phasing in that kind of a cut over a few years would help.
My purpose in presenting a tedious series of numbers is to try to persuade the reader that the U.S. has a big fiscal problem on its hands. One can be okay with raising taxes broadly, at the cost of some economic growth and lower disposable income for the middle class, or one can come up with spending cuts that are palatable, or some combination of the two, but it's simply naive to think that raising taxes on the "rich" is a complete solution.