A desktop application that has the same features (although as already stated, nothing will be identical but if you are looking for functionality then certainly there will be) and pretty simple to use was Microsoft Money, however, Microsoft stopped supporting it with newer versions and while the existing versions will work, I still use mine, there will be no future updates. I like the interface, its simple to use and has all the features you want. They abandoned it in favor of Intuit's Quicken but personally I am not a fan of the Quicken interface.
They still had a more extensive and probably too much for the average user application called Office Accounting, but they abandoned future updates and supports on that in favor of Intuit's Quickbooks. Again, I am not a fan of the interface but they are very feature rich including invoicing and payroll, again overkill for the average user. They still have the Small Business Accounting in the form of Microsoft Dynamics, but that is utterly overkill for personal use.
I generally don't trust online or cloud based accounting solutions like Mint or even Quicken online because I don't trust my information security to some third party without knowing how they are securing it and what will happen to me if/when they are leaked due to breach. So I like to keep everything local to myself and that's a good move for you, you should do that. It seems at the moment the market standard without much competition is Quicken for personal use and Quickbooks for small business. I would recommend you start with Quicken and if your needs increase in the future, you can easily transfer into Quickbooks to scale up as they are fully compatible with each other.
Check it out here and compare their products to see what works best for your needs.
