First, for clarity: I assume you are NOT a US citizen or LPR, and thus are a non-resident alien, as defined for tax purposes (abbreviated NRA). Most countries tax individuals on the basis of residence, but US is the only major country which applies its 'resident' or 'domestic' tax to US citizens, and lawful permanent residents (aka green-card holders), even if/when they reside abroad, PLUS aliens who are resident in the US according to a specific definition (the 'substantial presence' test) -- thus there are some aliens who actually reside in the US but are taxed as non-residents, but AFAICT no aliens not actually resident in the US who are taxed as residents.
NRA taxation in general. The US income tax on NRAs makes a major distinction between income that is 'effectively connected' to a US 'trade or business', and income that is 'not effectively connected'; see Pub 519 chapter 4 How Income of Aliens is Taxed / Nonresident Aliens (like many pubs this is revised every year, which may break the href, so I give the text heading; also like most pubs this is also available in PDF for the current year at https://www.irs.gov/publications and for prior years at https://www.irs.gov/forms-pubs/prior-year ).
The first category is taxed using a modification of the US-'normal' (resident) tax scheme: you subtract some of the same deductions -- but not all itemized deductions or adjustments, and as you note not the standard deduction, except for students and apprentices from India -- and then apply graduated (progressive) rates and brackets. Formerly you also subtracted the personal exemption(s), but the 2017 tax reform law eliminated those, as well as reducing or eliminating some itemized deductions; for residents these are mostly offset by increasing the standard deduction and child and dependent credits, but non-residents don't get those (definitively not the first and probably not the second). Officially many of the 2017 changes to individual tax are temporary, and personal exemptions are scheduled to come back in 2026, but there are still 5 years before then for Congress to
screw it up worse make further changes.
The second category is taxed much more simply, at a flat rate that defaults to 30% but may be reduced for a particular residence country by an applicable treaty.
Generally US payors of second-category income to an alien are required to withhold at the 30% or treaty rate (which you identify by giving them form W-8BEN), and as a result this withholding will normally be exactly equal to the tax. However payors of first-category income have varying withholding requirements, and in any case the actual tax can vary depending on deductions and/or other income the payor doesn't know about, so the withholding is usually NOT exactly correct and you must file a tax return to reconcile it, and if the withholding was more than needed to get a refund. (Which if you don't have/keep a US bank account will be in the form of a paper check you may have trouble cashing; we have several Qs on that already.)
Taxation of IRA distributions and conversions. Normally (i.e. for 'resident' tax), IRA distributions are included in ordinary income and taxed as such -- except if you made nondeductible contributions, called 'basis', a pro-rata portion of each distribution is considered return of basis and is not taxed. In addition an 'early distribution' taken before age 59.5 is subject to an additional flat tax of 10% unless certain exceptions apply. (This is often described as a 'penalty', although the tax rules don't call it that.) A conversion to Roth is similarly taxed as ordinary income, but is NOT subject to the early distribution tax.
Pub 519 also states that 'pension' income (which for international tax purposes seems to generally include IRA, although the India treaty has a specific requirement for 'periodic' payment which this isn't) for 'personal services' (a tax treaty term which closely matches the 'compensation' needed to contribute to an IRA) after 1986 is 'effectively connected'. I don't see how this squares with the model Tax Treaty, nor the few specific ones I've gone though, which 'source' pension only to the residence State, but the IRS instructions don't indicate any legal uncertainty, so I must be missing something. Unless you are pretty old, I expect your IRA won't have been been for work done in 1986 or earlier.
The instructions for form 1040NR lines 16a&16b (also in PDF similar to pub 519 as above) are consistent with this, and even more specific: exception 2 item 5 tells you to include here the taxable portion of any conversion of a traditional  IRA to Roth IRA. (edit) As you noted, this is computed on form 8606 part II. These lines are in the part of the form that computes the 'effectively connected' income, and tax on it.
Withholding for same (for NRA). This gets a little muddier. As above, the withholding rules for payments to an NRA are generally simple for second-category income but more complex for first-category, which this is. Pub 519 chapter 8 Withholding on Pensions says:
If you receive a pension as a result of personal services performed in the United States, the pension income is subject to the 30% (or lower treaty) rate of withholding. You may, however, have tax withheld at graduated rates on the portion of the pension that arises from the performance of services in the United States after December 31, 1986. You must fill out Form W-8BEN and give it to the withholding agent or payer before the income is paid or credited to you.
This makes it sound like it's your option.
OTOH Pub 515 Withholding on Specific Income / Pensions, Annuities and Alimony (code 15) which is the instructions for the payor says:
For purposes of Chapter 3 withholding, in the absence of a treaty exemption, you must withhold at the statutory rate of 30% on the entire distribution that is from sources within the United States. You may, however, apply withholding at graduated rates to the part of a distribution that arises from the performance of services in the United States after December 31, 1986.
This makes it sound like the payor's option.
(edit) Now, what is/are the 'graduated rates'?
Chart C following pub 515 Withholding on Specific Income / Income Not Effectively Connected / Original issue discount (code 30), although referenced only in the not-ECI text, contains several ECI types, including
Pensions—part paid for personal services ... Graduated rates in Circular A or Circular E
Wages paid to a nonresident alien employee ... Graduated rates in Circular A or Circular E
But these references are outdated, and potentially inconsistent. Circular A is pub 51 (A)gricultural Employer's Tax Guide and Circular E is pub 15
(E)mployer's Tax Guide, and these are primarily about employment-related tax for 'resident' (including nonresident citizen) taxpayers. These formerly included instructions on income tax withholding on wages (not pensions), including a (simple) adjustment for wages to NRAs -- but for 2020 those were moved to a new pub, 15-T. Both of these do not cover withholding on pensions, qualified plans and/or IRAs to 'residents' -- and for 'residents' those withholding rules are quite different than for wages, unlike NRA where an effectively-connection pension is supposedly withheld the same as wages.
Pub 515 has a long section on Pay for Personal Services Performed
that covers several broad and special cases but subsection Wages Paid to Employees - Graduated Withholding states the general rule, followed by some exceptions and modifications, and at "Determining amount to withhold" an updated reference to pub 15-T.
In contrast the section on Pensions, Annuities and Alimony has only the two sentences cited above, and no explicit reference.
I interpret this to mean that NRA pensions (after 1986 thus ECI) should be withheld following the same rules as wages in pub 15-T with the NRA adjustment, even though 'resident' pension withholding is NOT the same as for wages. If I'm right the payor calculates according to one of several options in pub 15-T; this may or may not be quite the same as what you the taxpayer would or will calculate. (Which is the essentially same issue faced by 'resident' taxpayers: withholding on wages is often not quite right, and on other income often not even close, requiring estimated payments or special withholding adjustments.) (end edit)
I would try contacting the IRA custodian and asking how they handle withholding for, and for that matter payments to or for, an NRA. Worst case you may have to accept the 'standard' 30% withholding -- which means you must front the missing cash in order make the conversion amount complete during the conversion year, and wait until early next year to get this withholding refunded. Remember you don't have to convert all in one year, you can do any amount(s) -- and to minimize tax it would be best to stay within the 10% and 12% brackets each year, which is $40,125 for 2020 (will increase with inflation, if any, in subsequent years). (edit) It's also conceivable they might apply the rules for resident IRA distributions, in which case the distribution almost certainly qualifies as rollover-eligible and if it goes through you -- i.e. you receive the distribution and then put it back, as a rollover/conversion, in the Roth -- get flat 20% withholding (regardless of actual tax status), but for direct trustee-to-trustee transfer there is NO withholding.
Good luck :-)