Assume you increase “x” of your net worth each year given a job you don’t particularly like/dislike much. When and at which salary/nw would you say you’d better be doing something else (family/hobbies/etc) or a part time and more enjoyable job instead?
It all depends on if "you" (specifically, the people in your house) are living below your means.
IOW, if the debt is piling up, then obviously not. But if you can still (relatively) comfortably (to clarify: that includes paying for your hobbies and other leisure activities) survive on the income generated by a lower-paying job, then by all means find a job that makes you happy.
Note that this all hinges on you controlling your finances (know where all your money goes, and follow a budget), not letting your finances control you.
When and at which salary/Net worth would you say you’d better be doing something else (family/hobbies/etc) or a part time and more enjoyable job instead?
To be able to do this you need to know that your new salary will support your lifestyle. By new salary that means money from a new full-time job, new part-time job, or hobby. It can mean money from your retirement sources of income if you are are old enough.
But when figuring your current lifestyle costs don't forget the money you need to save/invest to fund your retirement years. Breaking even when you are working age but not saving for retirement will force you to work longer, or to require you to work forever.
When to retire involves crunching the numbers to see if your assets and retirement income will cover your living costs no matter how long you live and no matter what the economy does.
The difficult part of this is estimating your future expenses and adhering to a disciplined spending plan once you retire.
There are lots of online retirement calculators as well as professional retirement planners.