As far as I can tell you cannot claim your UK NICs back in New Zealand.
The relevant tax treaty is on the HMRC page here and Article 2 is very clear about the UK taxes that are covered: income tax, capital gains tax, corporation tax and petroleum revenue tax.
There is a note at the end of Article 2 that says
This Convention shall also apply to any identical or substantially
similar taxes which are imposed by either Contracting State after the
date of signature of this Convention in addition to, or in place of,
the existing taxes.
and I briefly wondered whether national insurance might be seen as identical or substantially similar to income tax, but national insurance was imposed in 1911, well before the date of the NZ tax convention (1983).
Separately, while you are here, you might be able to "contract out" of SSP (State Second Pension) which means you effectively make lower NI payments. However this will depend on the nature of your employer's pension scheme and you should consult them and/or a financial advisor about this. More information about contracting out is available on Wikipedia, although note the point lower down the page that the second form of contracting out has been discontinued since 2012, so only the first form still applies.