At the area where I live (Finland), banks typically charge a lot more for additional mortgage credit taken after purchasing the house. So, if you are planning to purchase a house, and pay it with a mortgage, you get a very good rate, but if you pay back the mortgage and then realize you need additional credit, you get a much worse rate.
So, if this is applicable to your area as well, I would simply buy stocks after you have paid enough of the mortgage that it is only 50% of the house price or so. This is especially good advice if you are young.
Also, if your mortgage is a fixed rate and not an adjustable rate mortgage, you probably have a very low permanent interest rate on it as interest rates are low currently (adjustable rate mortgages will also have a low rate but it will surely go up). Some people say there's a bubble currently in the stock market, but actually the bubble is in the bond market. Stocks are expensive because the other alternatives (bonds) are expensive as well. Paying back your mortgage is equivalent to investing money in bonds. I don't invest in bonds at the current ridiculously low interest rates; I merely invest in stocks and have a small cash reserve that will become even smaller as I discover new investment opportunities.
I could pay back a significant percentage (about 50%) of the loans I have by selling my stocks and using my cash reserves. I don't do that; I invest in stocks instead, and am planning to increase my exposure to the stock market at a healthy pace.
Also, consider the fact that mortgage is cheap credit. If you need additional credit for consumption due to e.g. becoming suddenly unemployed, you will get it only at very expensive rates, if at all.
If you're very near the retirement age (I'm not), this advice may not be applicable to you.
Edit: and oh, if your mortgage is fixed rate, and interest rates have come down, the bank will require you to pay the opportunity cost of the unpaid interests. So, you may need to pay more than you owe the bank.
Edit2: let's assume the bank offered you a 4% fixed rate for a 10-year loan, which you agreed to. Now let's also assume interest rates of new agreements have come down to 2%. It would be a loss to the bank to pay back the amount of the loan (because the bank cannot get 4% by offering somebody else a new loan, only 2%), unless you paid also 10 years * (4% - 2%) * amount = 20% * amount of lost interest income. At least where I live, in fixed rate loans, one needs to pay back the bank this opportunity cost of unpaid interests.