A single person making 6 figures is in the 24% tax bracket, a taxable income over $82,500.
This means a 3.125% interest rate costs you a net 2.375%*. This is less than long term inflation. I understand being 'risk averse'. Only asking you to consider the long term. The difference between being invested in stocks vs cash (i.e. CDs or government treasury bills/bonds) is huge over time.
My recommendation is to put just the 20% down. I won't present any argument towards a 30 mortgage term, I realize that wouldn't end well. But I will suggest that you move in, see what other costs you didn't plan for and only after the place is fully lived in for, say, 6 months, look at what your bank balance is. An emergency fund of 6 full months' expenses is a good idea, so regardless of how you move forward, investing, an available cash reserve is a good thing. You can always make extra prepayments. To Justin's point, I agree that "sleep factor" outweighs any data I can offer.
(I would ask you, are you employed by a company offering a 401(k) and matching contributions?)
- The standard deduction for a single in 2019 is $12,200. If your state tax (I don't know, is your state tax 5%, like mine, higher?) is about $5,000 and your donations, $7000, you don't itemize now, but any amount over this would turn into a higher deduction. The $4375 in first year interest and the ~$3000 property tax, added to the former, give you a total $19,000+ that comes off you total income before taxes are calculated. James' comment is 100% good as well. My made up numbers may not apply. The punchline is that if, as a homeowner, these numbers push you to itemize your deductions, your cost of borrowing is not the gross 3.125%, but a post tax 2.375%. (3.125*.76). If this needs further clarification, I'm happy to edit again.