And does the withholding mentioned above only apply to federal income
tax withheld from W2 income, or does it include FICA withholding as
For regular FICA withholding, no. The underpayment penalty is only about income tax and income tax witholding, not FICA tax. Regular FICA withholding should be the right amount for your wages.
But if you had excess Social Security tax withheld due to having multiple jobs and exceeding the maximum amount of Social Security tax per year, that excess is counted the same as estimated tax payments on Form 2210 line 11, with 1/4 of the excess considered paid in each quarter, so it will have the same effect as withholding for the purposes of avoiding a penalty.
Also, Additional Medicare Tax is considered income tax, so it is included in the calculation. Additional Medicare Tax withholding (which is included on Form 1040 line 25c) is included in the withholding on Form 2210 line 6, and Additional Medicare Tax liability (Form 1040 Schedule 2 line 11) is included in Form 2210 line 2 as part of current year taxes.