This taxation guide may be helpful in sorting out some of your questions. I'm not entirely versatile with UK tax, so my answer will stay broad. I think the answer may be to consult a professional advisor.
You may become non-resident but remain UK domiciled. Everybody has exactly one domicile and it is essentially their permanent home (the place where they intend to one day return and live. The test is based on your intent - do you intend to return to the UK or do you intend to make another land your permanent home? Simply traveling about the world will not establish a new domicile for you. So you may owe some taxes on your worldwide income or capital gains while a UK-domiciled non-resident, as suggested here.
If it helps, the UK tax residence rules are listed by HMRC online. If your business is a corporation, there's a different analysis. You may also want to refer to the UK-BVI tax treaty. HMRC offers a tax residence calculator to help sort your residence, if you plan to return often.