You will owe tax on all but the deposit that was not taxed.
e.g. You deposited $5000/yr for 3 years, and deducted $5K for each of 2 years (the third $5K deposit was a nondeductible IRA contribution which you reported to the IRS by filing Form 8606 with your tax return for that year). Now you convert the total balance of $18K ($15K of contributions plus $3K of interest/gains). In this case, $5K is not taxable income while the $13K is taxable income. This calculation is done on Form 8606 for the year of the rollover.
Edit in response to OP's comment. The Roth conversion is text based on the value of a day it was converted. One thing to be aware of is that you can re-characterize up until the time you file your taxes for 2017 in April 2018 (or with extension up until October). This offers you the opportunity to undo the conversion if for whatever reason the value is lower at the time you do your taxes or if the converted amount will put you into a higher tax bracket. You don't need to give the IRS a reason, it's up to your discretion.
As Dave note in a comment, the conversion isn't all or none. The recharacterization, along with this fact, help you to fine tune exactly how much in converted in hindsight.