Canceling a credit card might hurt your credit score as @SnowMan said. Credit utilization ratio or balance-to-limit ratio have a good impact on your credit score and is often considered by many creditors while making a decision on new credit applications.
Also, frequent credit line openings and account closures can raise suspect on your credit report. If you want to cancel a credit card, make sure you have Zero credit in your credit report, which is next to impossible to average American.
At the time you close your account, if you have other credit cards with balance or a car loan, it will push your credit utilization rate higher and might hurt your credit score.
Alternative is that you can always downgrade the Credit Card you are trying to cancel to a type of credit card with no Annual Fees. If the bank doesn't offers a credit card of same type without a fee, you can ask the bank to convert it to another type of credit card without any fees.
For example, let's say you got a Amex Blue Cash Preferred credit card with an annual fees of $99 / year. But at the end of year one if you don't want to pay the annual fees, you can always call the bank to downgrade it to a normal Amex Blue Cash credit card without any annual fees.
Or you can convert that into an another type of Amex credit card which do not have any fees.