Congratulations on making it at minimum you are close to a 2 millionaire if I understand your numbers correctly. Here is what I would do if I woke up in your shoes:
1) Take some time. Budget some money and time. Go live abroad, take hang gliding lessons or become scuba certified. Something like that. The only thing I really dislike about your situation is that your wife may be precluded on going with you due to her business concerns. During this time dream, plan and decide what you want your life to look like. You seem to understand that you won't be happy doing nothing for a really long time. Its not a big deal if you blow 50K or so doing this. Take the wife to Paris, go visit the Galapagos Islands.
2) You are going to have to become wise about investing. I'd put close to one million in stock based mutual funds. That may sound scary, and you might seek others out to help you with this transition. I feel like that your time spent in your business may have precluded you delving into this area of knowledge. For now, you may just want to stick it all in interest bearing accounts, and slowly invest the money. Don't invest in things you don't understand, and you have to be on the look out for the next Bernie Madoff.
3) Its hard to speak to your desire to downsize your home. You could probably buy a nice ranch in Nevada from the sale of your home if that is what you desire, but you may kind of hate something like that.
4) Could you start more of a boutique business? Not one that occupies all of your time, but one that takes 20-40 hours per week. Something that interests you, not something that is overly a chore. Perhaps you can consult in the field that your former business was in. You most certainly have a lot of intelligent things to say.
5) Be generous. Find worthwhile charities to give time and money to.
Congratulations again. Take some time to dream, and then make those things happen.
Edit: You may need to make new friends. Actually wealthy people are a very small segment of the population and are out numbered by people who act wealthy. Its going to be hard, but you need to find people that have a certain level of wealth but are also don't make you uncomfortable with their level of spending (either high or low).