For an RRSP, you do not have to pay taxes on money or investments until you withdraw the money. If you do not reinvest the dividends but instead, take them out as cash, that would be withdrawing the money. For mutual funds, you would normally reinvest the dividends if holding the investment inside an RRSP. For stocks, I believe the dividends would end up sitting in the cash part of your RRSP account (and you'd probably use the money to buy more stocks, though would not be required to do so). Either way, you do not pay tax on this investment income unless you withdraw it from your RRSP.
For example, you invest $10,000 inside your RRSP. You get the tax benefit from doing so. You get dividends of $1,000 (hey, it was a good year), and use these to buy more stock. As the money never left your RRSP account, you are considered to have invested only your initial $10,000. If instead, you withdraw the $1,000 in dividends, you are taxed on $1000 income.
TFSA are slightly more complicated. You don't get a tax benefit from your initial contribution, but then do not pay tax when you withdraw from the TFSA. Your investment income is still tax-free, and you are (generally) much more limited in how much you can contribute.
For example, you invest $10,000 inside your TFSA. You get dividends of $1,000, and use these to buy more stock. Your total contributions to your TFSA remains at $10,000 as the money never left your account. You could instead withdraw the $1000 from your TFSA and would not pay tax on it. In the next calendar year (or later) after the withdrawal, you could "repay" the $1000 you took out without suffering an overcontribution penalty.
This makes TFSA an excellent place to park emergency funds, as you can withdraw and subsequently replace the investment while continuing to get the tax benefits on your investment income. RRSPs are better for retirement or for the home buyers plan. In general, you should not be withdrawing money from either your TFSA or RRSP, except in emergencies, when retiring, or when purchasing a home. I prefer indexed mutual funds or money market accounts for both my RRSP and TFSA rather than individual stocks, but that's up to you.