In general, you can provide cash to your company in two different ways: as an equity investment or as a loan.
In either case, you should write a personal check to the company, and then the company will deposit the check in a company-owned bank account. The company should also issue a written instrument documenting your investment (see below).
If you are making an equity investment ("buying shares"), the detailed mechanics depend to some extent on how the company is organized. In general, the company will credit your capital account for the amount of the investment. However, if you have multiple owners, you may need written corporate resolutions permitting you to make an investment; you will be issued shares according to the value that you have established for the company. Your bylaws or operating agreement should spell this out in detail.
If you are making a loan, make sure that the company signs a note that details the interest rate that the company will pay you, and the payment schedule. In this case, you don't receive any extra shares. Be sure to use a fair-market interest rate, and that the company makes interest payments to you as scheduled.
As a special case of a loan, if you are close to having cash flow that will cover the expenses, you can pay for some expenses with personal funds and then submit an expense report to the company. When the company makes a sale and has cash to reimburse you, it will cut a check to you to cover the expense report(s) that you have submitted.
With respect to the question about the company paying taxes on your investment: the simple answer is no, since your investment is not income to the company.
If your investment is as equity, there might be no immediate tax implications -- but there may be longer term implications, for example if you were to sell the company or liquidate.
If your investment is as a loan, the company will record the interest paid to you as an expense, which will reduce profits and thus should reduce the amount of tax paid by the company. The interest you receive from the company would have to be reported as income on your personal return.
Of course, if the company is a pass-through entity (S-Corp, sole proprietorship, etc), then the company doesn't pay Federal taxes (and maybe not state). The owner(s) report earnings on Schedule C and pay personal income tax based on that. What happens at the state level is a mess, every place is different.
Bottom line: A quick chat with your accountant will give you the best answer for your specific location and situation.