There's an FAQ for virtual currency on the IRS website. There's a question, Q38, indicating that transfers between your own wallets or accounts are not taxable:
Q38. Will I have to recognize income, gain, or loss if I own multiple digital wallets, accounts, or addresses capable of holding virtual currency and transfer my virtual currency from one to another?
A38. No. If you transfer virtual currency from a wallet, address, or account belonging to you, to another wallet, address, or account that also belongs to you, then the transfer is a non-taxable event, even if you receive an information return from an exchange or platform as a result of the transfer.
However, in the same FAQ, there's other information stating that you do have a capital gain or loss if you pay someone for a service with cryptocurrency:
Q14. Will I recognize a gain or loss if I pay someone with virtual currency for providing me with a service?
A15. Your gain or loss is the difference between the fair market value of the services you received and your adjusted basis in the virtual currency exchanged. For more information on gain or loss from sales or exchanges, see Publication 544, Sales and Other Dispositions of Assets.
It seems like these are in conflict. Wouldn't a mining fee be a payment for a service? Assuming A38 trumps A14, would you then just offset the size of the lot by the fee and simply reduce the basis by a proportional amount?
For example, if you bought 100 XYZ at the price of 100 USD (1 XYZ = 1 USD), and say you incurred a fee of 10 XYZ in a transfer, you own 90 XYZ now. Is your basis then adjusted to 90 USD? Or is there some associated gain or loss based on the market price of XYZ at the time that the fee was incurred?