Let's say I have a small sole proprietorship with only a bit of income and expenses. I can do up a balance sheet and an income statement at year end. Is that all I need to submit with my taxes, and I just declare the income on some line in the tax return, or is there a whole separate form? Does the business need to be registered somehow?
You would report the overall income on your T1 general income tax return, and use form T2125 to report income and expenses for your business. Form T2125 is like a mini income-statement where you report your gross revenue and subtract off expenses. Being able to claim legitimate expenses as a deduction is an important tax benefit for businesses big and small.
In terms of your second question, you generally need to register for a business number at least once you cross the threshold for GST / HST. If you earn $30,000/year (or spread over four consecutive quarters) then charging GST / HST is mandatory; see GST/HST Mandatory registration. There are other conditions as well, but the threshold is the principal one.
You can also register voluntarily for GST / HST even if you're below that threshold; see GST/HST Voluntary registration. The advantage of registering voluntarily is that you can claim input tax credits (ITC) on any GST that your business pays, and remit only the difference. That saves your business money, especially if you have a lot of expenses early on.
Finally, in terms of Ontario specifically (saw that on your profile), you might want to check out Ontario Sole Proprietorship. There are specific cases in which you need to register a business: e.g. specific types of businesses, or if you plan on doing business under a name other than your own.
Finally, you may want to consider whether incorporating might be better for you. Here's an interesting article that compares Sole Proprietorship Versus Incorporation. Here's another article, Choosing a business structure, from the feds.