Firstly, as pointed out in comments, an ISA is technically a wrapper around either a "cash" savings account, or a stocks and shares investment product. So, the decision between cash and investment is separate from the decision between an ISA and non-ISA product. Similarly, regular savings accounts are just a category of savings accounts with particular limits on what you can put in them and when; I'm not aware of any "regular saver cash ISAs", but in principle I believe they could exist.
The cost-benefit trade-off of choosing an ISA product as compared to a non-ISA product of the same type comes down to two factors:
- What, if any, tax savings will the ISA wrapper give you.
- What rate of return is offered by the ISA product you're evaluating, and how does that compare with the non-ISA product you're evaluating.
The first point in turn is affected by a few things:
- What income tax band are you in? The money earned inside an ISA is exempt from the tax that you would pay on it, which in turn depends on your total income.
- Are you likely to earn more interest over all than your "Personal Savings Allowance"? This is a new tax-free allowance, based on your income tax band, across all savings accounts (but not investments). It's based on the amount of interest, not the amount of savings, so low interest rates make it go further - right now, if you pay only Basic Rate income tax, and manage to put all of it in an account paying 1% interest, you'd need £100,000 to use up your allowance.
- Are you likely to benefit in future from putting money into an ISA? There's a bit of fortune-telling involved here, but since you get a new ISA allowance each year on top of what is already in ISAs, it can be worth moving money into an ISA just before the end of a tax year, to build up your allowance. This used to be more important when the limit was as low as £3,000 per year, rather than the current £20,000 per year.
That tells you how much the ISA will benefit you. To find out how much it will lose you, when compared to other products, there's no real alternative to surveying the market, which changes all the time. You just have to check sites like Money Saving Expert to find the best products in each category, and work out which one makes sense for you.