Logistics of borrowing in a different country aside, financially if the reason the interest rates are higher is because inflation is expected to be higher, you might be trading smaller interest payments for the depreciation of your mortgage payment in the foreign country.
So let's look at this scenario:
- the balance on the mortgage is 10,000,000 XYZ (where XYZ in the currency in the foreign country)
- the XYZ inflation rate will be 10% (roughly in line with the interest rate)
- the inflation in Germany will be 2%.
- today, the exchange rate is 100 (to make the math easier) with no transaction costs
So you take the following actions:
- You borrow 100,000 EUR at 3.4% and pay off the XYZ mortgage
- Over the year your rental income is 11,150,000 XYZ (which would have exactly paid off the mortgages)
- In one year, the exchange rate is 107.84 (100 * (1.10 / 1.02))
- Your 11,150,000 XYZ now buys 103,393 EUR
- You spend 103,400 to pay off the loan for a net loss of 7 EUR
So just because the interest rate is much lower, you need to consider the expected inflation in both countries. Generally, if the interest rates reflect the expectation of inflation, the exchange rate between the two currencies will offset the effects of interest savings.
If the inflation in the foreign country is NOT as high as the interest rates indicate (and thus the exchange rate does not go up as much), then yes you might save a little by borrowing at a lower rate.