If my 2017 state tax deductions are reduced by alternative minimum tax, does the state tax refund become taxable income in 2018?
The general rule is that you need to include the state tax refund only to the extent that you received a benefit from the deduction. When you do your 2018 taxes, you have to recompute your 2017 taxes (both regular tax and AMT) with the state tax deduction reduced by the refund amount. If the recomputed tax is greater than the tax that was paid, you must include part or all of the state tax refund in your 2018 income. See the sections titled "Recoveries" and "Itemized Deduction Recoveries", and the subsection "Subject to alternative minimum tax", in publication 525.
This is such a headache that I long ago decided to always underpay my state taxes simply to avoid it. (Of course there are other reasons to avoid overwithholding as well.)