I'd agree that this can seem a little unfair, but it's an unavoidable consequence of the necessary practicality of paying out dividends periodically (rather than continuously), and differential taxation of income and capital gains.
To see more clearly what's going on here, consider buying stock in a company with extremely simple economics: it generates a certain, constant earnings stream equivalent to $10 per share per annum, and redistributes all of that profit as periodic dividends (let's say once annually). Assume there's no intrinsic growth, and that the firm's instrinsic value (which we'll say is $90 per share) is completely neutral to any other market factors.
Under these economics, this stock price will show a "sawtooth" evolution, accruing from $90 to $100 over the course of a year, and resetting back down to $90 after each dividend payment.
Now, if I am invested in this stock for some period of time, the fair outcome would be that I receive an appropriately time-weighted share of the $10 annual earnings per share, less my tax. If I am invested for an exact calendar year, this works as I'd expect: the stock price on any given day in the year will be the same as it was exactly one year earlier, so I'll realise zero capital gain, but I'll have collected a $10 taxed dividend along the way.
On the other hand, what if I am invested for exactly half a year, spanning a dividend payment? I receive a dividend payment of $10 less tax, but I make a capital loss of -$5. Overall, pre-tax, I'm up $5 per share as expected. However, the respective tax treatment of the dividend payment (which is classed as income) and the capital gains is likely to be different. In particular, to benefit from the "negative" taxation of the capital loss I need to have some positive capital gain elsewhere to offset it - if I can't do that, I'm much worse off compared to half the full-year return. Further, even if I can offset against a gain elsewhere the effective taxation rates are likely to be different - but note that this could work for or against me (if my capital gains rate is greater than my income tax rate I'd actually benefit).
And if I'm invested for half a year, but not spanning a dividend, I make $5 of pure capital gains, and realise a different effective taxation rate again.
In an ideal world I'd agree that the effective taxation rate wouldn't depend on the exact timing of my transactions like this, but in reality it's unavoidable in the interests of practicality. And so long as the rules are clear, I wouldn't say it's unfair per se, it just adds a bit of complexity.