Cancelled cards don't fall off the system for a long time, up to ten years. Card terms change, with notice of course, but it can happen at any time.
I had a card with a crazy perk, 5% back in Apple Gift cards. This was pre-iPod days, but it was great to get a new computer every two years for free. But it was short lived. Three years into it, the cards were changed, a no-perk card from the bank. That is now my oldest account, and it goes unused.
Instead of holding cards like this, I wish I had flipped it to a different card years ago. Ideally, your mix of cards should provide value to you, and if they all do, then when one perk goes away, it's time to refresh that card.
This is a snapshot from my report at CreditKarma. (Disclosure, I like these guys, I've met their PR folk. I have no business relationship with them) Elsewhere on the page it's noted that average card age is a 'medium impact' item. I am 50, but I use the strategy above to keep the cards working for me. My current score is 784, so this B on the report isn't hurting too much. The tens of thousands I've saved in mortgage interest by being a serial refinancer was worth the hit on account age, as was the credit card with a 10% rebate for 90 days, the 'newest account' you see in the snapshot.
In the end, the score manipulation is a bit of a game. And some of it is counter-intuitive. Your score can take a minor hit for actions that would seem responsible, but your goal should be to have the right mix of cards, and the lowest interest (long term) loans.