While stannius' answer definitely alarms me, on the top of the same document (pub 505) the IRS seems much more relaxed.
The federal income tax is a pay-as-you-go tax. You must pay the tax as you earn or receive income during the year. There are two ways to pay as you go.
Withholding. If you are an employee, your employer probably withholds income tax from your pay. In addition, tax may be withheld from certain other income, such as pensions, bonuses, commissions, and gambling winnings. The amount withheld is paid to the IRS in your name.
Estimated tax. If you don’t pay your tax through withholding, or don’t pay enough tax that way, you might have to pay estimated tax. People who are in business for themselves generally will have to pay their tax this way. You may have to pay estimated tax if you receive income such as dividends, interest, capital gains, rents, and royalties. Estimated tax is used to pay not only income tax, but other taxes such as self-employment tax and alternative minimum tax.
So it seems not very cut and dry. FWIW I've done this method (claiming approx. income/4150 allowances on my W4, intelligently investing the float, then paying estimated payments ideally with a new credit card) for my state (Oregon) and Federal for the last two quarters and will definitely report back if I'm audited!
I have had a phone conversation with an Oregon tax auditor already, but this conversation reminds me that I didn't have a statement from a Federal auditor yet. Will work on that.
The IRS was recently required to not give public "tax law" support anymore and instead defer it to private individuals/companies. So, hopefully I'll get some leniency from the IRS if they call. After talking with two accountants that deal with worse things (non-payment, etc), they both thought that I wouldn't have any problems with this. We shall see!