Do I have to pay the stock investment income tax if I bought some
stocks in 2016, it made some profits but I didn't sell them at the end
You pay capital gains taxes only when you sell the stocks. When you sell the stock within a year you will pay the short term capital gains rate which is the same rate as your ordinary income.
If the stock pays dividends, however, you will have to pay taxes in the year that the dividend was paid out to you.
I bought some stocks in 2011, sold them in 2012 and made some gains.
Which year of do I pay the tax for the gains I made?
You would pay in 2012, likely at the short term gain rate.
I bought some stocks, sold them and made some gains, then use the
money plus the gains to buy some other stocks before the end of the
same year. Do I have to pay the tax for the gains I made in that year?
Yes. There is a specific exception called the "Wash Sale Rule", but that would only apply if you lost money on the original sale and bought a substantially similar or same stock within 30 days.
Do I get taxed more for the money I made from buying and selling
stocks, even if the gains is only in hundreds?
More than what? You pay taxes based on the profit you make from the investment. If you held it less than a year it is the same tax rate as your regular income. If you held it longer you pay a lower tax rate which is usually lower than your regular tax rate.