I know nothing about any Bolivian liability. As you say, you're not Australian tax resident, so there shouldn't be a liability there.
The NZ government website has a reasonably clear explanation:
New Zealand source income
New Zealand will also tax income obtained by a non-resident if it has a New Zealand source. This is known as source based taxation.
Generally, income will have a New Zealand source if it has a connection with New Zealand. Examples of income with a New Zealand source include:
- income derived from employment performed in New Zealand even if the employer is a non-resident
- pensions paid by the New Zealand government
- dividends paid by New Zealand companies.
I believe your situation is covered by the first bullet. I believe it's stated that way because it also applies if the employer is tax resident.
There's the possibility of an exemption:
Short term visit exemption
This exemption applies if:
a. your stay in New Zealand is for 92 days or less, counting the days of your arrival and departure as a whole day each
b. the total number of days that you're present in New Zealand in the tax year (1 April to 31 March) is 92 days or less
c. you're working for an employer that isn't resident in New Zealand, and
d. the income you earn while in New Zealand is taxed in your home country (the country where you're tax resident).
I believe you can't get this because your employer is NZ tax resident, but you might want to check with them.