First a warning: if you're a foreigner you must familiarize yourself with the FATCA and FBAR requirements. Search this forum and ask questions if you don't know what these are. The penalties are horrifying and can drive you bankrupt even if you don't even owe a dime in taxes, just because you didn't attach a piece of paper to your tax return.
To your question:
In the United States, brokers/banks do not withhold taxes from your gains (what is called "at source"), it is your responsibility.
Taxes are due when the income is earned, i.e.: when you sell the stocks. Paying on April 15th next year may trigger late payment penalties. The right way to do it is using estimated quarterly payments.
However, since capital gains is not your only income but you also have salary, you have another option. Using form W4, you can adjust the withholding your employer takes out of your paycheck so that it will cover not only the salary itself but also the extra income you have through capital gains. If you do that, you do not need to do the quarterly estimates. You can use the worksheets in W4 instructions to calculate the exact number.
What you need to understand is that the employer doesn't withhold the actual tax, but rather what you think your actual tax should be given your situation. Using form W4 you can ask the employer to withhold more or less, and you can change your selection during the year if adjustments are required. Any tax paid through withholding is considered paid on time (even if all of it is paid on December 31st).
If you withheld too much, it balances out on your yearly tax return (the one due by the April 15th), and you get a refund from the IRS. If you withheld too little - you have tax due on April 15th and you pay it. As long as due is less than $1000 or whatever your tax liability was the year before - you won't have penalties (the $1000 delta refers to withholding only, if you paid through estimates and have tax due - you may get hit by penalties even if you owe less than $1000).
Do I have to keep all record of transactions in order to calculate the
gains and appropriate tax payment?
You need to keep all records as long as the tax year is open (statute of limitation is 3-6 years for most income tax issues), however it is advised to digitize your records and keep it forever. You'll need it in case of an audit or dispute with the IRS or the State taxing agency.