Close... Warning, I may be off a bit here; I'm sure someone will correct me if so.
Traditional 401k or IRA: money goes in pre-tax (so, yes, you avoid paying tax on it now), grows untaxed, taxes are due when you retire and start taking money back out of the account -- but your income, including these withdrawals, is likely to be lower than your peak earning years so your tax rate will be lower. You don't avoid all the tax, but you delay it and hopefully reduce it, and by doing so there's more money in your account earning returns.
Roth 401k or IRA: money goes in after taxes (you do pay income tax now). However, all returns on the money are untaxed (I believe), and you pay no tax when you're eligible to withdraw the funds.
Either or both kinds of 401k may be eligible for some percentage of matching funds from your employer (there are some incentives for them to offer this benefit). I believe that even if you're doing a Roth 401k, the matching funds legally have to go in as traditional plan. And yes, as that implies, it is possible to split your contribution between the two styles.
Note: the matching funds are "free money." If your plan offers a match, it is highly recommended that you contribute enough to your 401k to capture the maximum match.