My experience is in economics, so it may differ from an accounting or personal finance perspective somewhat; that being said, I find it perfectly acceptable to use a term like CAGR when the rate is positive or negative. Economists talk about negative growth rates all the time, and it's universally assumed that growth rates can be positive or negative.1 Ideally, the actual magnitude and sign of the value should be specified by the value itself. The term, whether it's "growth rate", some modified version of it like CAGR, or any label in a table or on a graph, should describe the calculation or source used obtain the value. I shouldn't need the name to indicate the sign of the number if the number is present; the name is only there to help me understand the value.
Unfortunately, I don't know of any specific term that represents the geometric averaging nature of CAGR and also eliminates the minor potential for semantic confusion. However, I think the minor problem of semantics needs to be balanced against the tradeoff of using a different term that isn't as common, if one were to exist. CAGR is a standard, well-known term that a) allows someone who is familiar with the term to instantly understand the procedure you're using, and b) allow someone who isn't familiar with it to quickly search and find an explanation, since searching for CAGR will numerous simple explanations of how it's calculated.
1. This is different from the concept of economic growth, which is usually assumed to be positive in informal discussions. In economic modeling, many of the first steps in creating a model are symbolic anyway, so "growth rate,", "change in output", and "economic growth" are used interchangeably to describe changes in GDP because the values either aren't known, irrelevant until later in the project, or pulled from data that describes it using one or several of the previously stated terms.
