If I want to invest $4000 in the S&P 500 stocks (which is diversified and non-risk), how do I do this?
The S&P 500 is a stock market index, which is a list of 500 stocks from the largest companies in America.
You could open a brokerage account with a broker and buy shares in each of these companies, but the easiest, least expensive way to invest in all these stocks is to invest in an S&P 500 index mutual fund.
Inside an index mutual fund, your money will be pooled together with everyone else in the fund to purchase all the stocks in the index. These types of funds are very low expense compared to managed mutual funds. Most mutual fund companies have an S&P 500 index fund; two examples are Vanguard and Fidelity. The minimum investment in most of these mutual funds is low enough that you will be able to open an account with your $4000.
Something you need to keep in mind, however: investing in any stock mutual fund is not non-risk. It's not even low-risk, really. It is very possible to lose money by investing in the stock market. An S&P 500 index fund is diversified in the sense that you have money in lots of different stocks, but it is also not diversified, in a sense, because it is all in large cap American stocks. Before investing in the stock market, you should have a goal for the money you are investing. If you are investing for something several years away, an index fund can be a good place to invest, but if you will need this money within the next few years, the stock market might be too risky for you.
Buy the ETF with ticker "SPY". This will give you exposure to exactly the S&P 500 stocks, This is similar to the mutual fund suggestion by Ben Miller, except that the ETF has several advantages over mutual funds, especially as regards taxes. You can find information on the difference between ETF and mutual fund in other questions on this site or by searching the web.