(US) Quarterly estimated payments for income tax, and SE (self-employment = Social Security and Mediare) tax and/or household employment tax when applicable, are due 15 days after the end of March, May, August, and December, which is 4 times a year but not the equal quarters used for most other purposes. If the 15th is a weekend or Federal holiday, the deadline is the next business day.
This is laid out in the instructions published by the IRS.
There is no benefit for paying earlier or more than required. (Well, no benefit to you :-) If you have underpaid one or more earlier quarter(s), then making a subsequent payment early reduces the period on which you owe penalty; it is measured from the day due to the day actually paid. However you get the same benefit by making up the missed or shorted payment only, and making the new payment on schedule.
Yes, you can make more than one payment per quarter if you want; the IRS is happy to accept any and all payments you want to make any time(s).
Be sure if you pay by paper check to include a voucher with your SSN and tax type so it gets credited correctly; if you pay electronically you designate that info as part of the transaction. For a paper check by mail (or certain specified private services) you do get the benefit of the 'postmark' rule: if your payment is mailed on the due date it is credited as of that data even if it takes a few days after that to be received and processed.
There are other quarterly (or pseudo-quarterly) payments or taxes, such as payroll taxes on form 941, which work differently, but I consider out of scope for 'personal finance'.