Throughout most of history, money has been a measure of value.  Everything sold had value to the buyer, and thus Gross National Product was a decent measure of how much we're getting out the world.  Thus anybody spending money was good because there must have been an associated buyer and thus some value.  Before August 15, 1971, this held true for governments as much as anyone, because that money was ultimately redeemable for something physical ($35 was supposed to get you one troy ounce of gold).  Thus, supply is theoretically finite, as there is a link to the physical world, which has important consequences.

Now, however, money supply is theoretically infinite.  Does this mean money has no value?  Of course not!  If you disagree, go out and see if you have any trouble giving away your dollars.  So where does the value come from?  Money's value comes from people's willingness to accept it for things they want.  So no problem, we all accept dollars, right? 

Well, not really, the catch is in how many dollars we accept for any given thing. 
After all, how does a government manage to pay for anything?  Sovereigns have always been able to mint, and later print, money because of their ability to collect taxes by punishing those who defy the sovereign's will, a.k.a. the law.  All throughout history, money has been either made of, or was redeemable for, something people actually wanted.  It still is: the freedom to live as you please.

When we let Nixon sever that last link between money and the physical world, we went from a physically accountable, asset-based economy, to a credit-based economy whose only accountability is in how much the holders of the currency will stand for.  That is to say, we went from a civilized world where we settle most of our differences based on the transfer of real world items, to a world where value is based more directly on one party's ability to impose its will on the counter-party.

There may be ways mitigate some of the problems inherent in fiat currency.  Keeping a balanced budget is one part of the puzzle.