And most importantly what does the term "outliers" mean over here?
Get into a spreadsheet a list (from, say, the VTI ETF) of all companies and their P/E ratios. Sort the list by the P/E ratio and then plot the numbers by frequency (how many stocks have a P/E of 10.1, 10.2, 10.3, etc, etc).
You'll see most bunched in a broad "center", while there are some (with very low or very high P/E ratios) which lie very far out from the center.
As you can guess... those are the outliers.
Eliminating outliers is important when computing an average. The outliers lie really far from the center, and the number of elements in the list is relatively small.
Because of this, the "median" is usually a better statistical measure, since outliers which affect the mean (fancy word for "average") minimally affect the median". (This is also why you hear reference to things like "median family income" instead of "average family income": all those multi-multi-millionaires move the "average income" much more than it moves the "median income".)