I assume you exclude taxes not directly related to income (e.g. fire service tax, garbage collection tax, etc.).
(See last section if you want to know the "average" taxation situation for a certain income.)
The expenses in your second table are deductions for the purpose of calculating taxable income. They are not actually "deducted" from your bank account.
Essentially, the government recognizes that you'll have to pay certain professional expenses out-of-pocket to enable you to work your job and those income are not subject to the income tax. 4000 CHF is the maximum deduction amount if you use the simple calculation method (3% of gross income, min. 2000 CHF, max. 4000 CHF). Sometimes it might be beneficial for you to opt for a detailed method to claim actual expenses incurred (but of course, the taxation authority may require you to provide justifications and receipts).
Health insurance premiums (which are mandatory but technically not a tax) are also deductible. Of course, these deductions have limits (depending on the canton).
Your personal situation determines the actual taxable income and taxable net wealth. Commute costs (e.g. monthly pass for public transportation, a per-kilometer allowance for cars), payments for childcare, donations to charities, tuition for professional education, extra pension contributions (similar to US IRA or Canadian RRSP) etc. all impact your rate. In some cantons, even your rent to a certain extent is deductible under certain conditions.
In this aspect, it is similar to US and Canada where an accountant may be worthwhile due to the multitude of deductions available to optimize the tax situation.
That is also why no calculator can give you an exact and accurate number with only a gross income.
If you know well about your exact situation (and the relevant definitions in Swiss law), you can click the "Detailed calculation" button and see all the deductions possible on the official Swiss government calculator page (the first you linked). You can check it out anyway to get an idea of what can impact your taxable income.
Note that if you are a qualifying foreigner (non-Swiss citizen with a temporary residence permit and not married to a Swiss citizen or permanent resident), your taxation can be sometimes significantly simpler under the withholding tax regime (Impôt à la source/Quellensteuer). In that case, you just look up the table to find a percentage, multiply it to your gross income and that's all the income tax (for all levels: federal, cantonal, communal) you'll have to pay (in addition to social insurances). However (from 2021), using this simplified regime also means you cannot claim any other deductions (e.g. pension contributions, education expenses).
The table rates are calculated based on a "average" taxpayer in the canton in a similar personal situation (here the personal situation only includes your civil status, whether you live with your spouse or partner, whether your spouse or partner earns an income, how many children you have, your religion, and sometimes also if your job is part-time or full-time). So in a sense, this is what you could look at for a general idea about the taxation rates.
For Zurich, you can find the information here: https://www.zh.ch/de/steuern-finanzen/steuern/quellensteuer/quellensteuer-tarife.html (German only, unfortunately). You can also use the calculator at https://en.comparis.ch/steuern/quellensteuerrechner/ in English (not official, but mostly accurate since the way the tax is applied is standardized nationally).
Having a >120k gross income makes it mandatory to file an ordinary tax return though. You will still pay the withholding tax, but you will get a reimbursement if they withheld too much and pay extra if the withholding is not enough. If you earn under 120k and fulfill certain other conditions (e.g. no foreign income), you do not need to file a tax return.
For single persons, the table rate is probably more or less accurate. But for couples, due to assumptions made in the table rate calculation (the spousal income is "assumed" and "capped"), high-income couples will often experience the ordinary taxation very differently.