For example, Latvia offers a residency by investment program, and I was wondering whether it matters if you stay in the country for more than 183 days in order to pay the taxes? Or are you already considered a "resident" and have to pay the taxes (worldwide income if resident, income from Latvian sources only if non resident) whether you stay in the country or not? In other words, does getting a temporary residence permit automatically makes you a tax resident?
There are requirements to become a tax resident in a country. Having a visa or resident permit is not enough to make you a tax resident. Some resident permit will require you to be physically present in the country for most of the year, therefore passing the 183 days mark. Others don't, but you are always required to pay taxes where you earn it. Double taxation treaties help you not get taxed twice.
In many cases, before you can become a tax resident in another country, you will need to break your current tax residence and this could have some financial implications. For Americans, there is the notorious EXIT tax.
There is a table here that gives you all the tax residency requirement per country https://globalresidenceindex.com/hnwi-index/taxation-index/
As for Golden visas, they often cater to people who travel a lot and have low physical presence requirements. A handful of them will allow you to become a tax resident with only 45 days or 90 days a year of residence as long as you don't spend 180 days or more in another single country.