Take a look at IRS Publication 15. This is your employer's "bible" for withholding the correct amount of taxes from your paycheck. Most payroll systems use what this publication defines as the "Percentage Method", because it requires less data to be entered into the system in order to correctly compute the amount of withholding.
The computation method is as follows:
Compute the employee's gross pay for the pay period (wages, tips, salary, commissions, etc).
Subtract any "pre-tax deductions" (med ins premiums, retirement, etc). This is the employee's "pre-tax net pay".
Divide $3800 (the amount one W-4 allowance is worth annually) by the number of pay periods in the tax year (e.g 26 for bi-weekly). Multiply this by the number of allowances claimed on the W-4 and subtract it from the pre-tax net pay to determine the "pay subject to withholding".
Taxes are computed "piecewise"; dollar amounts up to A are taxed at X%, and then dollar amounts between A and B are taxed at Y%, so total tax for B dollars is A*X + (B-A)*Y. Here is the table of rates for income earned in 2012 on a daily basis by a person filing as Single:
More Than But Less Than W/H Base W/H Pct 0.00 8.30 0.00 0% 8.30 41.70 0.00 10% 41.70 144.20 3.34 15% 144.20 337.70 18.72 25% 337.70 695.40 67.10 28% 695.40 1501.90 167.26 33% 1501.90 433.41 35%
To use this table, multiply all the dollar amounts by the number of business days in the pay period (so don't count more than 5 days per week even if you work 6 or 7). Find the range in which your pay subject to withholding falls, subtract the "more than" amount from the range, multiply the remainder by the "W/H Pct" for that line, and add that amount to the "W/H Base" amount (which is the cumulative amount of all lower tax brackets). This is the amount that will be withheld from your paycheck if you file Single or Married Filing Separately in the 2012 TY. If you file Married Filing Jointly, the amounts defining the tax brackets are slightly different (there's a pretty substantial "marriage advantage" right now; withholding for a married person in average wage-earning range is half or less than a person filing Single.).
In your particular example of $2500 biweekly (10 business days/pp), with no allowances and no pre-tax deductions:
$2500 / 10 = $250 // you're in the 25% bracket
$144.20 * 10 = $1442.00 //amount taxed at lower rates incl. in W/H base
$18.72 * 10 = $187.20 //W/H base for $1442.00
$2500 - $1442.00 = $1058.00 //Amount subject to 25% tax
$1058.00 * .25 = $264.50 //Addtl tax for 25% bracket
$264.50 + 187.20 = $451.70 //Total Fed W/H
So, with zero allowances, your employer should be taking $451.70 out of your paycheck for federal withholding. Now, that doesn't include PA state taxes of 3.07% (on $2500 that's $76.75), plus other state and federal taxes like SS (4.2% on your gross income up to 106k), Medicare/Medicaid (1.45% on your entire gross income), and SUTA (.8% on the first $8000). But, you also don't get a refund on those when you fill out the 1040 (except if you claim deductions against state income tax, and in an exceptional case which requires you to have two jobs in one year, thus doubling up on SS and SUTA taxes beyond their wage bases).
If you claim 3 allowances on your federal taxes, all other things being equal, your taxable wages are reduced by $438.45, leaving you with taxable income of $2061.55. Still in the 25% bracket, but the wages subject to that level are only $619.55, for taxes in the 25% bracket of $154.89, plus the withholding base of $187.20 equals total federal w/h of $342.09 per paycheck, a savings of about $110pp. Those allowances do not count towards other federal taxes, and I do not know if PA state taxes figure these in.
It seems odd that you would owe that much in taxes with your withholding effectively maxed out, unless you have some other form of income that you're reporting such as investment gains, child support/alimony, etc. With nobody claiming you as a dependent and no dependents of your own, filing Single, and zero allowances on your W-4 resulting in the tax withholding above, a quick run of the 1040EZ form shows that the feds should owe YOU $1738.20. The absolute worst-case scenario of you being claimed as a dependent by someone else should still get you a refund of $800 if you had your employer withhold the max. The numbers should only have gotten better if you're married or have kids or other dependents, or have significant itemized deductions such as a home mortgage (on which the interest and any property taxes are deductible). If you itemize, remember that state income tax, if any, is also deductible.
I would consult a tax professional and have him double-check all your numbers. Unless there's something significant you haven't told us, you should not have owed the gov't at the end of the year.