I don't think there is a universal rule of thumb about an acceptable amount of tracking error because some indices are harder to track than others. Vanguard defines tracking error as:
the annualized standard deviation of excess return data points
Certain funds are easy to track (i.e. the S&P 500 index) and should have virtually no tracking error. Some tracking error is more acceptable for an esoteric index that's difficult to track. According to the New York Times the average tracking error of all ETFs in the US was 59 basis points (they're using a different definition of 'tracking error' than Vanguard):
The average tracking error of all E.T.F.’s listed in the United States
last year was 59 basis points, or slightly more than half of a
percentage point, according to a study released last month by Morgan
Stanley Smith Barney. For those E.T.F.’s with at least one year of
trading history, Morgan Stanley compared the difference in total 2012
return between each fund and its underlying index.
Index funds are meant to track indices and limit tracking error. An 'acceptable' amount of tracking error depends on how easy it is to track an index. The whole point of indexing is to earn zero alpha (i.e. low tracking error) and it's worthwhile to avoid funds that unnecessarily deviate from their underlying indices.