The thing you have to understand is that the price of securities is determined not by who owns the securities, but by who trades them. Securities held in passive funds, by definition, don't trade very much, and so they don't affect the price much, even if they comprise a significant fraction of the security's float.
For example, General Electric (GE) has a float of 8.7 billion shares. The average daily volume of GE traded is right around 100 million shares. So, the average share trades about once every 87 days. Let's say the fraction of GE shares held in index funds gets up to 10% of the total float. Then, the typical share in the index fund would have to change hands roughly once every 9 days in order to make up comparable volume to the average share. In other words, the "passive" investors would have to trade more often than the "active" ones. That's completely at odds with what we mean by "passive" investing.
From this we can deduce that passive investors would have to own a huge fraction of a company's shares in order for them to dominate the trading activity that sets prices. For example, if the average passively-held share changes hands once a year, then about 70% of shares would have to be passively-held in order just to equal the volume of other trading.
Even if the passively-held shares get to be a huge majority of the total float, I'm skeptical that they will ever control the price. If passive funds ever start to force prices too far off of their fundamentals, that will create an opportunity for active traders to profit, which should attract, if not more active traders, then at least higher volume from the existing active traders. In fact, there's some evidence that this already happens (emphasis mine):
One of my little stock-market obsessions is that index funds free-ride on the work done by active investors. Someone needs to make decisions that allocate capital to businesses. A world in which everyone indexes, and in which no one thinks that active managers should be able to charge for their services, is a world that will spend too little time and effort on allocating capital to the right businesses. That's not the world we live in: A lot of people still actively work to allocate capital, though they are in some regulatory disfavor and sometimes have a tough time making money. Part of the way they make money, or try to, is by trading against the index funds which free-ride off their labor, but which trade in a relatively mechanical, non-fundamentals-driven way. The index funds have the advantage of free-riding, but the disadvantage of being predictable. Stocks should go up when they join an index. That's the price that the index funds pay to active traders for picking stocks. Stock picking is valuable; active investors pay for it in fees, while passive investors pay for it in, you know, front-running or whatever.
--Bloomberg View, "Can you really game index funds"
So, in other words, don't worry about the markets. They're going to be just fine because it doesn't really take that many people actively participating in order for them to function as engines of price discovery.
