TL;DR: Cost to the company is likely to be the main factor, especially from 2005 onward when rules around ESPPs changed so that offering discounts off the purchase price and providing lookback provision became a direct cost to the company.
According to What is a "lookback"? on the myStockOptions.com website, lookback provision is a clear win for the employee, as it:
bases the purchase price not on the stock price at the time of purchase but, rather, on the price either at the beginning of the offering period or at the end of the purchase period, whichever is lower.
From an employee's point-of-view, this is a no-brainer: if the stock price has risen over the period, they can buy at the opening price for what amounts to an extra discount; if the price has dropped by the end of the period, they're no worse than if lookback wasn't provided. But what does (did) it cost the employer to offer lookback?
According to ESPP: What’s In It for the Company? on The Finance Buff website, prior to 2005, companies were not required to "book an expense" when providing stock options or an ESPP. Therefore, offering an ESPP (with or without lookback provision) was beneficial, since:
Employees got a profit at no cost to the company. If the company paid the employees the equivalent amount in cash, they would’ve had to book an expense and negatively impact their earnings.
So, during this time, since lookback provision was essentially always a benefit to the employee, and didn't cost the employer anything, it made sense to offer it.
However, from 2005 the situation changed:
The accounting rules changed in December 2004 when the Financial Accounting Standard Board (FASB) issued FAS 123(R). Now companies are required to book an expense for their ESPP unless the discount is no more than 5% and the program doesn’t have a look-back provision. As a result, many companies discontinued or scaled back their ESPP plans. Many of them reduced the discount from 15% to 5% and removed the favorable look-back provision in order to comply with FAS 123(R) and keep the ESPP off their income statement.
However, the same page also notes that "for competitive reasons", some companies would have continued with the schemes "as is". From the employers' point-of-view, any benefit of offering lookback provision are essentially intangible: as the above article says:
ESPP plans also create employee loyalty. When employees own stocks in the company (if they didn’t sell right away), they are more likely to work harder. In management buzz words, the employees’ interests are “aligned” with the company.
Prior to 2005, they got this "benefit" at no cost; after 2005 they would have to justify the intangible benefit against the tangible cost involved.
Stopping lookback provision, and/or cutting the discount offered under ESPP schemes at the beginning of 2005 would have made financial sense for many companies, as from that date they would incur a new, direct cost.
Any company that maintained these benefits past 2005 would need to periodically weigh the intangible benefits (to the employer: gained through the tangible benefits to the employees) of providing ESPPs and lookback provision against the (now tangible) cost of doing so. A company that has recently removed its lookback provision, and/or reduced the discount offered under its ESPP, may have decided that those potential benefits are no longer worth the cost.