When evaluating mortgage offers in the US for a primary residence, obviously the rate and fees is important to consider as it impacts the bottom line. Having a lender who is responsive when trying to close is also useful. What are the potential downsides of using a random cheap discount online lender? After the closing does the lender matter? What factors should be considered when evaluating a lender.
After the closing the lender does not matter. Many brokerages who don't hold the note will sell the note after the closing anyway. The only real issues you should concern yourself with are the lenders responsiveness to help you close quickly and the terms of your loan. Also be proactive and confirm terms and closing documents BEFORE the actual closing. Tell your lender and attorney you want to see documents BEFORE the closing to avoid surprises and have time to confirm that the terms are what you agreed to.
Some brokerage lender have a tendency to change some small terms before closing. Other than your terms and responsiveness. Your lenders don't differ all that much.
One item you can check beside the rate and fees, is how likely they are to sell the mortgage and the mortgage servicing.
Once the loan closes, for potentially 180 or 260 months you will be sending the payments to a servicing company. It may be the same company that handled the mortgage, or it could be sold to somebody else. In addition the mortgage itself could be sold off by the lender to an individual, bank, corporation, or other investor.
Why do you care about who services the loan? For the life of the loan the servicing company will have to process your payment, pay your real estate tax bill and PMI from escrow, and calculate the proper escrow amount. If you want to either pay off the entire loan early or make additional payments they have to handle those events correctly.
If you elect to use "a random cheap discount online lender" expect that all future interaction will also be via email and phone. If there is a problem that may be the hardest way to solve the problem.
If you really like the lender check to see if they sell the loan servicing. If they do sell the servicing then expect that you will have zero control over who they sell it to.
You can minimize stress by picking a lender that will continue to service the loan. You can also minimize stress by making a large enough down payment to allow you to skip the escrow: you will handle making sure that the tax payment is made to the local government, and that the hazard insurance is paid. I have only seen this done when you make a big enough down payment to avoid PMI. I have insisted this as a part of applying for the loan, if a lender wouldn't put the writing that I could avoid the monthly escrow account I wouldn't even consider them.