A stock market index like a Sensex 30 or a S&P 500 is a collection of stocks based on a certain criteria. This is a hypothetical portfolio of stock that is said to represent either a section or the entire stock market. The number of stocks held, the weightage of each stock, the rebalancing rules etc are unique to each index and can vary significantly from index to index.

When we want to judge the performance of the entire stock market, we often use the performance of these indices as a proxy metric. So, when we colloquially say that the market is down, we refer to a specific index representing the market being down. It is completely possible for parts of the stock market or certain segments to be up but an index is the pragmatic metric to judge the performance of a market consisting of 5000+ individual stocks.

In this essay, we will discuss how market indices such as Nifty 50 and Sensex 30 work. We’ll start with how they are composed, then move into rebalancing of the index followed by a brief analysis of why indices are a good instrument for investment and we’ll end by touching upon what is Direct Indexing and why that might be the future of investments

How is Nifty 50 formed?

Traditionally, major market indices such as Nifty 50 or Nasdaq 100 are composed of the top ‘n’ companies in the stock market based on their Free Float Market Cap. Before we understand what the free float market cap is, let’s understand what the market capitalisation of a company means.

The definition of market cap is extremely simple, it is the current market price multiplied by the number of shares outstanding. Outstanding share is any and every share of the company that is owned by an entity other than the firm itself. So institutional holding, individual holding, shares in a stock option plan or shares held by the founder added up is the number of shares outstanding of the company. Outstanding shares do not include any treasury shares, that is the shares that are owned by the company itself. Thus, the market cap represents the dollar value of the company at any given moment.

The free float market cap in the same vein is a multiple of the current market price and the free float shares of the company. Free float shares simply refers to the shares of the company that are available to be traded in the stock market. Chunks of stock such as Employee Stock Options, shares held by the founders/ promoters etc are not available in the public market for trading. Hence the tradeable market cap of the firm, or the free float market cap of the firm is the dollar value of the tradeable component of the company at any given moment.

An example to understand the difference between the two is through HDFC and Reliance (as of Aug 2024).

We can see that reliance is almost 0.8x of HDFC, yet its weight in the sensex is 2 percentage points lesser than that of HDFC. This is because Mukesh Ambani and family (the promoters of Reliance) own a significant chunk of the shares in the company and those shares do not form a part of the free float shares and subsequently the free float market cap.

The reason for choosing free float market cap is very simple. The index wants to rank companies on the basis of the company’s tradeable value. The shares which are locked in are not available for investors to gain off of and hence in this particular instance, we must use free float market cap to weigh our index.

Once we have the free float market cap of every company in the stock market, we rank them in descending order of their free float market cap and choose the Top 30 companies in case of Sensex and Top 50 companies in case of Nifty 50. These companies form our hypothetical portfolio. We then sum up the total free float market cap of our basket of stocks and use that as our benchmark. The weight of each stock in the index is their free float market cap divided by the index’s free float market cap multiplied by 100. Here’s the exact data for Sensex 30 (as of June 2021):

One keen observation here is how HDFC went from being 10.63% of the index to 13% of the index from June 2021 to Aug 2024. This is a nice segue into our next topic - Rebalancing.

What does Index Rebalancing mean?

The definition of rebalancing is the process of adjusting the composition of a market index, ensuring that it’s reliable and relevant. Typically, rebalancing happens quarterly or semi-annually. There are 2 things that happen in an index rebalance:

1. The % allocation (weights) of the different stocks in the index change
2. The stocks composing the index change, i.e., some stocks are removed from the index and some are added

These 2 combined forms the index rebalancing process. The rebalancing process essentially ensures that the worst performing stocks in the portfolio are not marginalised by the best-performing stock’s good performance. Consider the below example, we start with a basket of 4 stocks in the index. Please excuse the fractional number of shares, for sake of simplicity, I did not reach the lowest common multiple of the portfolio allocation for the number of shares and left the number of shares bought in decimals.

Without rebalancing, we can clearly see how we are over allocated in Stock A (which performed better) and under allocated in Stock D which performed worse. Hence, through rebalancing we ensure that the worst performing stocks aren’t marginalised and the best performing stocks aren’t over bought.

This thought should raise certain questions with respect to the traditional investment philosophies. Consider the following philosophy: “Cut your losses short and let your winners win.” If at all, we are doing the complete opposite of that philosophy. We are buying more of our losers and selling our winners. How is this prudent investing?

Can we be following the principle of Value Investing where we buy low and sell high? That might be the case here, but in value investing, we must sell only when we reach the intrinsic value of a company. We aren’t even looking at the PE of the stock before “selling high” let alone doing a valuation exercise. Then how does this follow the principles of Value Investing?

This is definitely not Momentum Investing. It might be the antithesis of it, where we cut the stocks that have an upward momentum and stock up on companies with a downward momentum.

This might follow the core principle of Growth Investing, where we are betting on the growth potential of the stocks that did not perform well, not taking into account any other factor. But can we assume that a stock has growth potential just because it's big? The argument in favour of this hypothesis is the fact that if you are not growing at a competitive pace, you would be removed from the index and a new stock with the growth potential will be added. Thus, making this a growth investment philosophy at least in principle.

The philosophy aside, portfolio rebalancing is what ensures that an index performs better than almost all actively managed funds. In a portfolio rebalance, we are not overtly greedy with one stock or the other always having a portfolio that the market balances for us. With this discussion in mind, let’s understand why an Index Fund is considered the best investment and why even Warren Buffet endorses them.

Why invest only in Index Funds?

One of the most prevalent theories of how the stock market functions is the base theory of all Microeconomics - The Efficient Market theory. This essentially says that given enough buyers and enough sellers that have equal information, we reach the most efficient price for an item in this case - the stock price. When the market reaches efficiency, that is, there is no alpha to be generated due to over or under valuing of the stock, every individual in the market is bound to make the same returns in the market which is almost equivalent to the growth in revenues/ profits of the entire market.

In this scenario, the market knows best where to put the money and no individual can perform better than the market. Thus, when we rebalance the portfolio, we essentially re-calibrate our holdings with that of the collective intelligence of the market. In a case of efficient markets, the maximum returns to be made is at the price and balance that the market decided for us and hence the index is the best measurement for it.

In reality, the markets are not efficient. There are charges to be paid on every trade, the playing field is not level and information supplied is not equal. But, with the advent of technology, we are much closer to an efficient market than we ever were. When we add the irrationality of humans (read emotion driven investing) along with the move towards market efficiency, more often than not, the market wins. That being said, there are innumerable inefficiencies in the markets to make money off of, but the condition for that to work is rigorous research and well-controlled emotions.

Direct Indexing - The future of Investing?

If we want to own a piece of an index right now, we must invest via an index fund. We buy a fractional unit of the index in exchange for a meagre amount of investment. Given that fractional shares are not allowed in India, it is virtually impossible for a retail investor to buy 1 unit of the index. It would cost Rs 24.5 Lakh to buy just 1 unit of an index in June of 2021. I am confident this number is higher in Aug of 2024 and will keep increasing. Thus, the necessity of an index fund.

Just blindly following an index is a manual task and something that a well-written program can do phenomenally at minimal cost. Thus, the question which comes to mind is: Is the expense ratio that is given to fund houses worth it? We are losing out on some returns because there is no available solution for us to buy an index directly. It should be noted that like an index, an index fund is also rebalanced quarterly. This brings up a scope to return certain test to understand question such as the following:

1. How much are we losing out to expense ratios?
2. Could we generate an alpha if we rebalance our portfolio’s daily/ weekly/ monthly instead of quarterly?
3. Could custom indices (such as additional metrics in rebalancing) help generate higher returns to investors?

We must run back-tests to understand the implications of these questions or run a simulated trading environment that assumes the role of a direct investor and does paper trades. Here’s what such a system would look like:

In 2021, I had started developing this direct indexing bot but left the project mid-way. Have pushed the code on github, check it out here. I never ended up completing it and deploying it. Hit me up if this is something you’d like to work on as a side project with me.

In terms of business potential, depending on the answers that this simulation gives us, there is a potential for a small case like product but driven on the principles of efficient markets along with scope for certain customisations. All in all, I do believe eventually index funds would be direct and completely automated given the manual nature of the process.