Let me add a few thoughts that have not been mentioned so far in the other answers.
Note that for the decision of buying vs. renting a home i.e. for personal use, not for renting out there's a rule of thumb that if the price for buying is more than 20 year's (cold) rents it is considered rather expensive. I don't know how localized this rule of thumb is, but I know it for Germany which is apparently the OP's country, too.
There are obviously differences between buying a house/flat for yourself and in order to rent it out. As others have said, maintenance is a major factor for house owners - and here a lot depends on how much of that you do yourself (i.e. do you have the possibility to trade working hours for costs - which is closely related to financial risk exposure, e.g. increasing income by cutting costs as you do maintenance work yourself if you loose your day-time job?).
This plays a crucial role for landlords I know (they're all small-scale landlords, and most of them do put in substantial work themselves): I know quite a number of people who rent out flats in the house where they actually live. Some of the houses were built with flats and the owner lives in one of the flats, another rather typical setup is that people built their house in the way that a smaller flat can easily be separated and let once the kids moved out (note also that the legal situation for the landlord is easier in that special case). I also know someone who owns a house several 100 km away from where they live and they say they intentionally ask a rent somewhat below the market price for that (nice) kind of flat so that they have lots of applicants at the same time and tenants don't move out as finding a new tenant is lots of work and costly because of the distance.
My personal conclusion from those points is that as an investment (i.e. not for immediate or future personal use) I'd say that the exact circumstances are very important: if you are (stably) based in a region where the buying-to-rental-price ratio is favorable, you have the necessary time and are able to do maintenance work yourself and there is a chance to buy a suitable house closeby then why not. If this is not the case, some other form of investing in real estate may be better.
On the other hand, investing in further real estate closeby where you live in your own house means increased lump risk - you miss diversification into regions where the value of real estate may develop very differently.
Relation between being rich and being landlord
There is one important psychological point that may play a role with the observed relation between being rich and being landlord. First of all, remember that the median wealth (without pensions) for Germany is about 51 k€, and someone owning a morgage-free 150 k€ flat and nothing else is somewhere in the 7th decile of wealth. To put it the other way round: the question whether to invest 150 k€ into becoming a landlord is of practical relevance only for rich (in terms of wealth) people. Also, asking this question is typically only relevant for people who already own the home they live in as buying for personal use will typically have a better return than buying in order to rent.
But already people who buy for personal use are on average wealthier (or at least on the track to become more wealthy in case of fresh home owners) than people who rent. This is attributed to personal characteristics and the fact that the downpayment of the mortgage enforces saving behaviour (which is typically kept up once the house is paid, and is anyways found to be more pronounced than for non-house-owners). In contrast, many people who decide never to buy a home fall short of their initial savings/investment plans (e.g. putting the 150 k€ into an ETF for the next 21 years) and in the end spend considerably more money - and this group of people rarely invests into directly becoming a landlord.
Assuming that you can read German, here's a relevant newspaper article and a related press release.