If you were not required to file a federal income tax return for those years (e.g. your income was below the threshold for needing to file), you can just send in the Form 8843 (using the correct version of the form from each applicable year) by itself.
From the Form 8843 instructions (on page 3 of the Form 8843 PDF), section "When and Where To File":
If you don’t have to file a 2019 tax return, mail Form 8843 to the
Department of the Treasury, Internal Revenue Service Center, Austin,
TX 73301-0215 by the due date (including extensions) for filing Form
1040-NR or Form 1040-NR-EZ.