If bank B has a transfer limit of $N, whereas bank A has a transfer limit that is higher, what happens if bank A attempts to withdraw from an account in bank B more than > $N in one transaction?
Does it get denied somehow, or does it go through?
If bank B has a transfer limit set, you bet that there is a nice reason for that. Either risk of fraud, liability, client preferences, profiling, credit scoring, etc, etc.
For a bank, the cost of denying something [1] is way lower than the potential damages and liabilities of allowing something to go through.
Regarding your concerns for the ACH, here is the summarized transaction walkthrough source:
An Originator– whether that’s an individual, a corporation or another entity– initiates either a Direct Deposit or Direct Payment transaction using the ACH Network. ACH transactions can be either debit or credit payments and commonly include Direct Deposit of payroll, government and Social Security benefits, mortgage and bill payments, online banking payments, person-to-person (P2P) and business-to-business (B2B) payments, to name a few.
Instead of using paper checks, ACH entries are entered and transmitted electronically, making transactions quicker, safer and easier.
The Originating Depository Financial institution (ODFI) enters the ACH entry at the request of the Originator.
The ODFI aggregates payments from customers and transmits them in batches at regular, predetermined intervals to an ACH Operator.
ACH Operators (two central clearing facilities: The Federal Reserve or The Clearing House) receive batches of ACH entries from the ODFI.
The ACH transactions are sorted and made available by the ACH Operator to the Receiving Depository Financial Institution (RDFI).
The Receiver’s account is debited or credited by the RDFI, according to the type of ACH entry. Individuals, businesses and other entities can all be Receivers.
Each ACH credit transaction settles in one to two business days, and each debit transaction settles in just one business day, as per the Rules.
Take heed of this like:
The Originator initiates a direct deposit/payment transaction.
In your scenario, the originator would be B. But since the transaction amount is higher than the limit, B would not even initiate the ACH transaction. The request would be denied.
So the transaction would look like this:
[1] Usually this cost comes down to just the processing costs of the denied transaction (and it is rather fail-fast like). For the other parties involved it may have additional costs (missed deadlines, penalties for not fulfilling an obligation, fines, etc), but for the bank that is irrelevant.
Or at least I saw it do so with Bank of America.