In a word, yes. Else, your withholdings are looking at a completely different tax table. Your tax return in 2019 doesn't care who made how much, only the total (for federal tax purposes). A couple making $100K pays far less in tax than a single person making the same $100K. In your case, the "single rate" withholding is far higher than the true situation calls for.
On a final note, I'd suggest using the 2018 tax software, and running an estimated return for 2019. It won't be exact, but close enough to help you not have a surprise next April.