You can submit a new W4 to your employer any time you need to make changes. You can reduce federal income tax withholding by adding deductions to line 4(b):
To know how much to enter in the Deductions section, use the IRS Tax Withholding Estimator.
In the Income and Withholding step of the estimator you can account for multiple jobs, employment gaps, etc:
After going through the estimator they will show results with a projected refund and a slider you can use to adjust how much refund you want to get. Sliding that bar down from what it initially shows will bring up a section on how to adjust your withholdings that will instruct you on how to adjust your W4 and also a link to a partially-filled W4 you can use based on your results.
Good point in the comments, don't forget to submit a new W4 at the start of the new year without this extra deduction so you don't end up with too little withheld next year.