The answer unfortunately is not black and white.
The IRS website defines the burden of proof as follows:
Generally, taxpayers meet their burden of proof by having the information and receipts (where needed) for the expenses. You should keep adequate records to prove your expenses or have sufficient evidence that will support your own statement. You generally must have documentary evidence, such as receipts, canceled checks, or bills, to support your expenses. Additional evidence is required for travel, entertainment, gifts, and auto expenses.
While a credit card statement will show the supplier of the item in question, it will not show details of the item itself.
Credit card statements will show almost the same thing as cancelled checks, with the exception that a check normally has a memo field where you can describe what the service is (which importantly is accepted by the receiver of the check).
As a result you could make arguments either way. On one side, one could say that the credit card statement is adequate documentary evidence, while on the other, one could say that the annotations could be made-up, and there is no way to check them to see if they are true.
Unfortunately if you and the IRS disagree, you would have to take the matter to tax court and get a decision - probably too expensive to do to be worthwhile. You might just have to be prepared to say: "okay, I accept that these are inadequately documented, here is a check for the tax that I should have paid on these items".
I would suggest that if keeping all of the receipts are too much of a burden, that you record receipts for things that are potentially an issue (large amounts, items that are unusual, etc.). While say, a single transaction from Staples around the price of copy paper, annotated as "copy paper" probably doesn't need a receipt to back it up.
Some people implement a policy such as: receipts for all transactions over $20; just so that during an audit, the question of having inadequate documentation is never raised, and if it is (the auditor today thinks the threshold should be $15) the impact is not so great.