For listed options in NYSE,CBOE, is it possible for an option holder to exercise an option even if it is not in the money? What about early exercise for OTM American option, is it possible? Under what circumstances would someone do it?
It is possible to exercise an out of the money option contract.
Reasons to do this:
You want a large stake of voting shares at any price without moving the market and could not get enough options contracts at a near the money strike price, so you decided to go out of the money. Then exercised all the contracts and suddenly you have a large influential position in the stock and nobody saw it coming. This may be favorable if the paper loss is less than the loss of time value that would have been incurred if you chose contracts near the money at further expiration dates, in search of liquidity.
Some convoluted tax reason.
For listed options in NYSE,CBOE, is it possible for an option holder to exercise an option even if it is not in the money?
Abandonment of in-the-money options or the exercise of out-of-the-money options are referred as contrarian instructions. They are sometimes forbidden, e.g. see CME - Weekly & End-of-Month (EOM) Options on Standard & E-mini S&P 500 Futures (mirror):
In addition to offering European-style alternatives (which by definition can only be exercised on expiration day), both the weekly and EOM options prohibit contrarian instructions (the abandonment of in-the-money options, or the exercise of out-of-the-money options). Thus, at expiration, all in-the-money options are automatically exercised, whereas all options not in-the-money are automatically abandoned.
It may be favourable to exercise a slightly out-of-the money call option just before the ex-dividend date, especially if the dividend exceeds the (negative) intrinsic value. Practically speaking, the dividend subsidizes your asset purchase. I have had stocks scooped out from under me in exactly this short call situation.