The procedure to compare the costs of a mutual fund and an ETF is to find two similar investment strategies and look at their prospectus. All the costs are listed online, and many sites have tools to compare not just funds in the same family but across multiple families. This is the same procedure that would be used to compare any two investment vehicles.
Other questions on this site discuss the general difference between and ETF and an index fund. Even if the average fund of type A is cheaper than the average fund of type B, you have to look at the actual funds that interest you to determine which is cheaper. The cost will vary from fund family to fund family.
The costs for two different funds with the same investing style is a function of the general fund costs of that family, and the specific costs of that fund.
Keep in mind that costs are only a part of the investment equation; along with investing style, strategy, tax treatment, risk...