Let's say that I live in Connecticut and take out a loan with a bank in another state (for this example, let's say New York). Is there a general principle on which state's usury laws would apply, or is it lender by lender?
Because they are a bank, there is a relevant US Supreme Court decision, Marquette Nat. Bank of Minneapolis v. First of Omaha Service Corp. (439 U.S. 299 (1978)), which ruled that federal laws (in particular, the National Banking Act) allowed the bank's state to govern what rules apply and to override all state and local usury laws.
This Supreme Court decision is why most banks have their credit card operations in South Dakota.