First, assuming you have not been in the US in F or J status before this, you have correctly determined yourself to be a nonresident alien for tax purposes. But I will explain why that is. You are a resident alien for a year if you pass the Substantial Presence Test for the year, but days as an "exempt individual" are not counted in the Substantial Presence Test. Days you are in J1 status (other than as a J1 student) is considered an "exempt individual" as an "teacher or trainee", unless you have been an exempt individual for any part of 2 of the previous 6 calendar years. Assuming you haven't been in any status that is an exempt individual for any of the previous 6 calendar years, your days on J1 in 2023 and 2024 will not be counted in the Substantial Presence Test, and thus you will be a nonresident alien for 2023 and (if you do not come to the US in 2024 after your J1 ends) 2024.
Nonresident aliens cannot file as Married Filing Jointly, unless their spouse is a US citizen or resident alien and they choose to treat the nonresident alien spouse as a resident alien. As a result, you will have to file as Married Filing Separately (or, if you are a resident of Canada, Mexico, or South Korea, and you live apart from your spouse the whole year, you might be able to file as Single). Nonresident aliens also cannot claim dependents, unless you are a resident of Canada, Mexico, or South Korea.
As a nonresident alien, you would file form 1040NR for your federal tax return, and you need to file form 8843 since you are an exempt individual.
Residency for state taxes is not determined the same way as the resident alien/nonresident alien for federal taxes. You are a California tax resident if you are "present in California for other than a temporary or
transitory purpose", which is a subjective standard that I can't answer for you. Your J1 status does not directly matter. See Publication 1031 for more details.
For 2023, you might either be a California part-year resident (resident from January 27) or whole-year nonresident. Either way, you would file 540NR and 540NR Schedule CA. The only difference between California residents and nonresidents is that residents are subject to California tax on their worldwide income (including foreign income that is not included on 1040NR), while nonresidents are only subject to California tax on their California income (although the tax rate will still take into account their worldwide income, including foreign income that is not included on 1040NR). If you have no income outside California from the period you were in California, then there will be no difference in California tax between if you were a resident or nonresident for that period of time.
You will not have an underpayment penalty if you have withheld 90% of the taxes due for the year, or if the taxes due minus withholding is less than $1000. It is already past the 4th quarter estimated tax payment deadline, but it seems that your concern is not with an underpayment penalty, but rather that you want to pay taxes due before you file your tax return. I believe you can still make an estimated tax payment now, or if not, you can make it as an "extension payment", which is treated basically the same as an estimated tax payment, but it can be made up to April 15.
Also, if you have concerns about difficulty making payments in the future, you might think about your 2024 taxes too. You can make an estimated tax payment now, if you wish, if you think your withholding is not enough, and you don't want to pay taxes later.