There are analogies in corporate finance which compare equity to a long call option, and debt to a short put.
If you a long an at the money call (long stock) and long an at the money put (debit), you create a synthetic position in the underlying assets called a "straddle". When a bank loans someone money, it is essentially short a put (no upside beside recurring interest; risk of principal loss).
In theory, a straddle is less risky than buying the stock outright because you have some downside protection in case things goes against you.
Therefore it should be less risky to buy the stock of your lender than to buy stock outright. You are exposed to the bank's upside, but it is exposed to your downside.
For example, no matter how the bank performs, your debt will be no better or worse off -- the interest payments are not likely to change. If you do well, then the bank is not able to participate in your upside aside from lowering its default risk (which should in theory already be reflect in your rates). If you do poorly while the banks does well, then it finances your losses with their equity capital appreciation.
This is, of course, an oversimplification. There are other risk factors not captured by this specious analogy — e.g., counterparty risks in the case your debt is “callable”.