I analyzed S&P data looking at the 100 15 year rolling returns from 1903-2018. No period produced a negative return, and only 5 of the 100 were below 4%. This was to look at the common advice to "take the 30 year mortgage and invest the difference". The difference being the savings from the payment required for a 15 year mortgage.
You're situation is a variation of this, as you have the money up front or much of it. In my case, I was claiming that at the 15 year mark, the saved balance would far exceed the mortgage in a way that blends into your approach, then being able to make no payments out of pocket, just using those funds. I took this approach myself, and despite an average interest of 6% for the 15 year preceding my retirement (2012), at that point we had a mortgage balance of $265K vs $349K extra in the 401(k). Maybe not a huge difference, especially given the $349K is subject to tax. But, by the end of 2014, the numbers were $453K vs $233K. A quick estimate shows I needed to withdraw $166K for payment through the end of 2020, and the balance remaining saw an S&P double, so this year, we are near $110K mortgage balance but $600K in the retirement acct. I wrote about this in 2015, Retired, With Mortgage and should update it to present numbers.
In your case, if you can manage to make payments out of current income and let that money ride, your chance of success is high (because Justin's warning of early plan market drop is super wise). Consider, at the 15 year mark in my own experience, there were 2 crashes, still the numbers worked. The data is available for your own analysis, it's worth learning how to manipulate a spreadsheet.
A note - there are those who suggest that this is risky. Risk is a number, a chance of failure. They are 100% right in one regard. If your finances are arranged so you can't sleep at night, turn the dial down. If it takes 100% investment in CDs or treasury bills to sleep, no argument from me. For sake of disclosure, I'm sleeping great with a 70/30 mix. 30% cash/bonds is over 6 year's spending money for us. [it was 100/0 as we entered retirement, FYI.]