An equity swap is not really "borrowing" money - you are just paying a fixed percentage (not really "interest", just accounting for the time value of money) in exchange for the total return (increase in price plus dividends) of the equity. If the equity has a higher return than the fixed percentage, you make money. If it does worse, you lose money. It's a way of taking a long position without actually buying the stock.
If you want a short position on the equity, the process is just reversed - someone pays you a fixed percentage in exchange for the total return of the equity. If the equity returns less than the fixed percentage, you make money. If it does better, you lose money. So it's effectively a short position.
The "fair price" has nothing to do with the expected performance of the stock, but how much it cost you to borrow money. If you entered into a equity swap for $100 worth of stock at a fair rate of 5%, but could borrow $100 at 2% to buy the stock, you would make a risk-free 3% profit. So the fair price is more about interest rates than the performance of the equity.
For the short position, the dealer could borrow $100 to buy the stock, so their fair rate is the same as their borrowing costs. They don't have to borrow to but the stock, but that's the theory behind the "fair swap rate"
Dealers do not necessarily have to borrow stock to hedge their position - in fact the swap usually is a hedge. They may just take offsetting positions with other investors or have a naturally short position that they want use the swaps to hedge.