The answer is in your question: derivatives are contracts so are enforced in the same way as any other contract.
If the counterparty refuses to pay immediately they will, in the first instance be billed by any intermediary (Prime Broker etc.) that facilitated the contract. If they still refuse to pay the contract may stipulate that a broker can "net off" any outstanding payments against it or pay out using deposited cash or posted margins.
The contract will usually include the broker as an interested party and so they can, but don't need to, report a default (such that this is) to credit agencies (in some jurisdictions they are required to by law). Any parties to the trade and the courts may use a debt collection agency to collect payments or seize assets to cover payment.
If there is no broker or the counterparty still has not paid the bill then the parties involved (the party to the trade and any intermediaries) can sue for breach of contract. If they win (which would be expected) the counterparty will be made to pay by the legal system including, but not limited to, seizure of assets, enforced bankruptcy, and prison terms for any contempts of court rulings.
All of this holds for governments who refuse to pay derivatives losses (as Argentina did in the early 20th century) but in that case it may escalate as far as war. It has never done so for derivatives contracts as far as I know but other breaches of contract between countries have resulted in armed conflict.
As well as the "hard" results of failing to pay there are soft implications including a guaranteed fall in credit ratings that will result in parties refusing to do business with the counterparty and a separate loss of reputation that will reduce business even further. Potential employees and funders will be unwilling to become involved with such a party and suppliers will be unwilling to supply on credit.
The end result in almost every way would be bankruptcy and prison sentences for the party or their senior employees. Most jurisdictions allow for board members at companies in material breach of contract to be banned from running any company for a set period as well.
edit: netting off cash flows
netting off is a process whereby all of a party's cash flows, positive and negative, are used to pay each other off so that only the net change is reflected in account balances, for example:
company 1 cash flows
- +1.2 M : co2
- +1.1 M : co3
- -5oo K : co4
- -3 M : co5
- +1.2M : co3
netting off the total outgoings are 3M + 500k = 3.5M and total incomings are 1.2M + 1.1M + 1.2M = 3.5M so the incoming cash flows can be used to pay the outgoing cash flows leaving a net payment into company1's account of 0.