First of all, setting some basics:
- Risks are generally measured on a their severity and their likelihood of occurrence.
- Risk assessment must be both quantitative and qualitative where possible. Qualitative risk assessment is a minimum.
- Where no exact figure for a risk assessment can be produced (e.g. VaR = 10M$), conventionally a 1-5 rating scale where 1 is lowest and 5 is highest is used. This helps build a risk matrix (see Wikipedia - Risk Matrix).
- Risk appetite is your tolerance level for taking risks (or aversion for risk, inversely). While this does not change the risk management process in itself, this is what you should ask yourself when taking a final decision: am I willing to take that risk ?
What is a sound way to measure the risk of each investment in order to compare them with each other ?
There is no single way that can be used across all asset classes / risks. Generally speaking, you want to perform both a quantitative and qualitative assessment of risks that you identify.
Quantitative risk assessment may involve historical data and/or parametric or non-parametric models. Using historical data is often simple but may be hard in cases where the amount of data you have on a given event is low (e.g. risk of bust by investing in a cryptocurrency). Parametric and non-parametric risk quantification models exist (e.g. Value at Risk (VaR), Expected Shortfall (ES), etc) and abound but a lot of them are more complicated than necessary for an individual's requirements.
Qualitative risk assessment is "simply" assessing the likelihood and severity of risks by using intuition, expert judgment (where that applies), etc. One may consult with outside parties (e.g. lawyers, accountants, bankers, etc) where their advisory may help highlighting some risks or understanding them better.
To ease comparing investment opportunities, you may want to perform a risk assessment on categories of risks (e.g. investing in the stock market vs bond market). To compare between those categories, one should look at the whole picture (quantitative and qualitative) with their risk appetite in mind. Of course, after taking those macro decisions, you would need to further assess risks on more micro decisions (e.g. Microsoft or Google ?). You would then most likely end up with better comparatives as you would be comparing items similar in nature.
Should I always consider the worst case scenario ? Because when I do that, I always can lose everything.
Generally speaking, you want to consider everything so that you can perform a risk assessment and decide on your risk mitigating strategy (see Q4). By assessing the likelihood and severity of risks you may find that even in cases where you are comparatively as worse-off (e.g. in case of complete bust), the likelihood may differ. For example, keeping gold in a personal stash at home vs your employer going bankrupt if you are working for a large firm. Do note that you want to compare risks (both likelihood and severity) after any risk mitigation strategy you may want to put in place (e.g. maybe putting your gold in a safety box in a secure bank would make the likelihood of losing your gold essentially null).
Is there a way to estimate the probability of such events, better than intuition ?
Estimating probability or likelihood is largely dependent on data on hand and your capacity to model events. For most practical purposes of an individual, modelling would be way off in terms of reward-benefits. You may therefore want to simply research on past events and assign them a 1-5 (1 being very low, 5 being very high) risk rating based on your assessment of the likelihood. For example, you may assign a 1 on your employer going bankrupt and a 2 or 3 on being burglarized. This is only slightly better than intuition but has the merit of being based on data (e.g. frequency of burglary in your neighborhood).
Should I only consider more probable outcomes and have a plan for them if they occur?
This depends largely on your risk appetite. The more risk averse you are, the more thorough you will want to be in identifying, tracking and mitigating risks. For the risks that you have identified as relevant, or of concern, you may opt to establish a risk mitigating strategy, which is conventionally one of accepting, sharing (by taking insurance, for example), avoiding and reducing. It may not be possible to share or reduce some risks, especially for individuals, and so often the response will be either to accept or avoid the given risks by opting in or out on an opportunity.