The Challenge of Measuring Governance at the Subnational Level

Reuben Abraham, CEO, IDFC Institute
Vaidehi Tandem, Junior Fellow, IDFC Institute
Kadambari Shah, Senior Associate, IDFC Institute
Vibhav Mariwala, Senior Analyst at IDFC Institute

Indices can provide a simple way to measure and assess a country’s progress on important governance goals. But there are potential downsides to judging a country’s performance based on just one number, especially if that country is large and diverse. In India, for example, the performance of individual states across governance metrics will often deviate from the national average. Moreover, there are also likely to be significant subnational variations that could drive India’s performance in one direction or another. 

Given these concerns, governments from larger nations should consider effective ways to track governance at both the national and subnational level. 

Tracking India’s progress at the Subnational Level

Since coming to power in 2014, Prime Minister Narendra Modi has strengthened tools and initiatives to measure the performance of states across several key areas, including economic development, health and wellbeing, and business reform. The Government of India’s (GoI) aim is to incentivise states to improve the livelihoods of their citizens by providing clear and regular data on shortcomings, successes, and progress. 

The National Institution for Transforming India, also known as NITI Aayog, was established by the GoI to assist in the design of long-term policies and programmes for India, while also providing technical advice at the state and national levels. One important NITI Aayog initiative is the ‘Aspirational Districts’ Dashboard, which tracks and measures changes in the socio-economic status of 117 districts across 28 states. The live dashboard provides states with regular updates on progress and the information is then used by leadership to drive and implement reform.

Another example is the ‘Swachh Survekshan’ Cleanliness Survey, which was launched by Modi in 2016 to monitor states’ progress in meeting goals set out in the ‘Swachh Bharat’ (Clean India) Mission. It was hoped that the annual survey – commissioned by the Ministry of Urban Development – would encourage healthy competition as cities vie to be crowned India’s ‘cleanest’. In 2020, Indore in the state of Madhya Pradesh was named India’s cleanest city for a fourth year in a row. 

Other examples include the Ease of Living Index and Municipal Performance Index launched by the Ministry of Housing and Urban Affairs in 2018 and 2019 respectively, and the Healthy States Progressive Index launched by NITI Aayog, the Ministry of Health and Family Welfare, and the World Bank in 2018.

Bihar and Karnataka

The sheer size and diversity of India presents challenges when comparing states. Consider Bihar and Karnataka. In India’s north lies Bihar, a state with a population of 83 million according to the 2011 census. To the south, Karnataka is home to 53 million people. If Bihar and Karnataka were countries, they would be the 20^th and 28^th most populous in the world; but there is an enormous economic disparity between them, with Karnataka nearly five times richer than Bihar. Karnataka has a per capita GDP of approximately INR 231,000 in current terms, or 1.6 times India’s per capita GDP. At the other end, Bihar has a per capita GDP of INR 48,000, which is just under a third of India’s per capita GDP. 

Such differences become increasingly evident when measuring Bihar and Karnataka against CGGI indicators. For example, the CGGI measures several indicators within the Outcomes for Opportunity pillar, including a country’s literacy rate. While India’s literacy rate was 74% in 2018, rates in Karnataka and Bihar were 75.3% and 61.8% respectively. This is a difference of 13 percentage points, with Bihar performing well below the national average. 

There are also subnational variations in India’s business environment, as measured by the World Bank’s popular Ease of Doing Business rankings. India ranked 142 in 2014 and by 2019 had climbed to 63. However, India’s performance is based on perception-based interviews conducted by the World Bank with experts and public officials in the capital Delhi and financial hub Mumbai. The country’s business environment can hardly be captured by looking solely at these two major cities given the significant variations at the state level in terms of business environment. 

This variation has been identified by the NITI Aayog-IDFC Institute Enterprise Survey, which revealed stark differences in the number of days needed to set up a business across individual states. Similarly, the Department for Promotion of Industry and Internal Trade’s (DPIIT) annual ranking of business regulatory environment also showed disparity. The ranking assesses state performances based on their implementation of a checklist of reforms. In 2015, Karnataka ranked 13 (out of 28 states and 8 Union Territories) with a score of 88.4% while Bihar ranked 16 with a score of 75.8%. By 2019, Karnataka had implemented 100% of DPIIT’s reforms compared to the 82% implemented in Bihar.

CGGI also measures Protection of Property Rights within the Attractive and Stable Marketplace pillar. Poor land record management and a weak system of land titling has led to a rise in property disputes in India. In response, the GoI recognised a need to modernise the process of land transactions. In 2008, it launched the National Land Record Modernization Programme, later renamed the Digital India Land Record Modernization Programme, to digitise land records, transaction registrations, and maps, and conduct land surveys. The GoI monitors progress through a central portal where states can enter and update their own data. The portal has shown wide variation in levels of state reporting, as well as progress made. For example, Karnataka has computerised 99.6% of its land records while Bihar has computerised 76.6%. 

Disparities are not unique to Karnataka and Bihar, nor are they unique to India. But in drawing on CGGI indicators to compare the two, we can demonstrate how a single state could improve or hinder a country’s overall governance performance. 

Collecting and Interpreting Data

There are also challenges in collecting data at the subnational level in India. 

First, the quality of data and levels of reporting vary significantly across states, especially in measuring socio-economic levels, and health and crime statistics. Indian states do not rely on the same metrics and processes for data collection. Some states like Maharashtra produce annual economic surveys that include district-level economic data. However, nearly half of all districts in India don’t report district-level economic data at all. 

The problem here is largely a divergence in state capacity. Drawing on work by Daron Acemoglu, Camilo Garcia-Jimeno and James Robinson, state capacity broadly refers to the ability of the state to provide services and effective governance to its citizens. Milan Vaishnav sums up weak institutional capacity in India as 3Ps: too much paperwork, too many processes, and too few personnel. Capacity varies across states, which leads to inconsistencies in their ability to collect and report data that is critical to measure subnational performance.

As seen in the case of India, subnational variations may be driving certain data and by extension, rankings, in a particular direction. Given these considerations, countries with large populations would benefit hugely from instituting subnational ranking frameworks.

Second, even if it is possible to measure performance using consistent and high-quality data, the indicator itself may not accurately represent the de facto situation on ground. For example, some states in India have made great progress in reforming their business environment but effective implementation has stalled. This could be attributed to a range of reasons, including governmental failure to effectively communicate policy changes to end-users, or technological shortcomings where online portals set up by departments don’t always work.

The Importance of Context When Measuring Governance

India scores 0.559 with an overall rank of 48 out of 104 countries in the Chandler Good Governance Index. But there are important considerations when discussing the performance of a nation as large and diverse as this. Surveys can oversample metropolitan areas at the expense of rural areas and collecting and contextualizing fully representative subnational data can be a challenge. As seen in the case of India, subnational variations may be driving certain data and by extension, rankings, in a particular direction. It is important to keep these differences in mind when interpreting a nation’s performance, both with respect to the CGGI and governance indicators more broadly. Given these considerations, countries with large populations would benefit hugely from instituting subnational ranking frameworks. 

IDFC Institute is a “think/do tank” in Mumbai, India