Doing what you suggest may actually be helpful.
Today, you have wealth of 145k and debt of 140k, for net wealth of 5k.
Your interest incurred is $671/month and your interest earned is $211, for a total loss of $460/month, just below the 491 $/month you are saving, so your total saving is $31/month currently.
However, even though in total, you have more money each month than the month before, you are getting more debt and thus more interest to pay each month. Your interest earned is increasing much slower.
That $31/month you are currently able to save? By the time you hit 51, that has become $0/month and is still dropping. By 60? Your debt has overtaken your retirement savings - that $5 net worth you have now is gone.
If you were to withdraw money from your retirement to pay off your debt (with the $32k penalties) you would have wealth of 70k and debt of 97k, for a net wealth of -27k (i.e. net debt).
Obviously, the above is not good. However, you reduce your monthly interest paid to $465, while also reducing your interest earned to $102. This is a total loss of $365/month, so you are saving $126/month.
Note that in this case, your $491 monthly repayment is higher than the interest you have to pay on the account, this means that each month, your interest payment becomes lower, thus you pay off more and more each month.
Your balance would be getting better each month (and at a faster rate each month. Your net wealth would be back in positives and above your wealth on your current trajectory before you hit 62.
By 65, you will have $9000 of net wealth if you use your retirement savings now, as opposed to $9000 net debt if you don't.
And just adding a few things on to the end
1) This is just the maths of it, and does not take into account your behaviour. If having that debt accruing is helping to motivate you to give up on luxuries, then this analysis does not apply. I am assuming that the $491/month is literally all you can save, and that no matter what changes, you will always deposit that $491. If you do not think you can continue to deposit that $491 if you stop seeing such high interest accruing, then do not do this.
2) I am assuming your interest earned on your IRA is 1.75%. If this is not the case, then please let me know, and I can adjust my numbers accordingly. From http://www.usatoday.com/story/news/politics/2014/01/28/obama-state-of-the-union-myra-savings-plan/4992743/
3) I'm assuming all numbers you mentioned are accurate, and will stay constant (interest rates may not)
4) This is not professional, financial advice. I am just a person on the internet.
5) This goes without saying (and will probably go down as well as "let them eat cake" did), but saving more money each month will be a more powerful, risk free way to get out of this problem. Work a 2nd job, cut costs however you can.
6) Sorry if you were looking for something more motivational or sugar coated.
7) Best of luck, feel free to ask any questions.
Graph below in red is your current trajectory, and blue is if you withdraw from your retirement to pay off your debt.