Strengthening India's Statistical System | 17 Jul 2024

Recent debates about the quality of India's official statistics, particularly data from the National Sample Survey Office (NSSO), have highlighted important issues in the country's statistical system. The concerns raised about sample design and data quality, though not proven statistically valid, underscore the importance of modernizing India's statistical approach.

The core issue revealed by these debates is not necessarily that India's statistical data is fundamentally flawed, but rather that the country's statistical system has not kept pace with global advancements in data science and integration.

To maintain the credibility and relevance of its official statistics, India needs to invest in modernizing its statistical methodologies, improving the frequency and timeliness of data releases, and exploring innovative approaches to data collection and analysis. This will be crucial for providing policymakers with the accurate, up-to-date information needed to make informed decisions in a rapidly changing economic and social landscape.

What is the Current Statistical Framework in India?

• Central Government:
  • The Ministry of Statistics and Programme Implementation (MoSPI) serves as the central nodal agency for the country's official statistics system.
    • The National Statistics Office (NSO), under MoSPI, oversees the integrated development of the national statistical system.
      • NSO comprises the Central Statistics Office (CSO) and the National Sample Survey Office (NSSO).
    • Apart from NSO, various line ministries/departments maintain statistical establishments for data collection, dissemination, and coordination with NSO.
• State Government:
  • In states, the official statistical system is typically laterally decentralized across various departments of the State Government.
    • At the apex level, Directorates of Economics & Statistics (DES) coordinate statistical activities within the State/UT.
  • States are responsible for data collection, compilation, processing, and preparing results for most sectors, with state-wise data contributing to national-level statistics used by the Centre.
• National Statistical Commission (NSC):
  • Established in 2006 based on recommendations from the C. Rangarajan Commission, NSC serves as the apex advisory body on statistical matters.
• Placement in Seventh Schedule:
  • The subject of 'Statistics' is included in both the Union and Concurrent Lists of the Seventh Schedule of the Constitution of India, specifically listed under Entry 94 (Union List) and Entry 45 (Concurrent List).
• Legislative Framework:
  • Specific legislative acts governing statistics include the Census Act, 1948; the Registration of Births and Deaths Act, 1969; and the Collection of Statistics Act, 2008.

What are the Major Issues Related to India’s Statistical System?

• Census Delay and Its Implications: The repeated postponement of India's 2021 Census represents a critical disruption in the country's statistical system, with wide-ranging impacts on governance, policy-making, and resource allocation. Key examples include:
  • Policy Distortions: Outdated demographic data leads to misaligned policies.
    • For instance, education planning for school infrastructure and teacher recruitment is based on 2011 population figures, potentially underestimating current needs in rapidly growing urban areas.
  • Economic Miscalculations: The delay affects the revision of state-wise poverty ratios and Centre-state tax sharing.
    • States like Bihar or Uttar Pradesh, with higher population growth rates, may be underfunded based on decade-old data.
• GDP Estimation Methodology Concerns: India's GDP estimation methods have faced scrutiny for potential overestimation, impacting the credibility of economic growth figures. For instance, the 2015 revision of India's GDP series sparked significant controversy.
  • It raised the GDP growth rate for 2013-14 from 4.7% to 6.9%, leading to skepticism about its accuracy.
  • Former CEA Arvind Subramanian stated that the 2011-12 GDP series released in 2015 overestimates growth. This made India surpass China in 2015 to become the fastest growing major economy.
• Employment Data Reliability and Frequency: The discontinuation of the NSSO's comprehensive Employment-Unemployment Surveys created a significant data gap.
  • The Periodic Labour Force Survey (PLFS), introduced in 2017-18, faced criticism for methodological changes that made it difficult to compare with previous surveys.
  • This issue underscores the need for consistent, comparable, and frequent labor market data.
• Poverty Estimation Challenges: The government has not released official poverty estimates since 2011-12, partly due to methodological debates.
  • The Tendulkar Committee methodology, which set the poverty line at Rs 27 per day for rural areas and Rs 33 for urban areas in 2011-12, was criticized as too low.
    • The Rangarajan Committee suggested higher thresholds, but its recommendations weren't officially adopted.
  • This lack of consensus and updated data has led to widely varying unofficial estimates of poverty, hampering effective policy formulation.
• Discrepancies in Mortality Data during COVID-19: The pandemic highlighted significant gaps in India's death registration system.
  • According to WHO, there were likely 4.7 million deaths, directly or indirectly attributable to Covid-19 in India in 2020 and 2021.
    • India officially estimated only 4.8 lakh cumulative deaths linked to Covid-19 as of December 2021, which implies that the WHO estimate is nearly 10 times the government count.
  • This massive discrepancy points skeptics to issues in death registration and cause-of-death reporting, crucial for health policy and demographic projections.
• Informal Sector Measurement Challenges: India's large informal sector, estimated to employ over 80% of the workforce, poses significant measurement challenges.
  • The Economic Census (6th ) was last released in 2013-14. And the 7th Economic Census is yet to be released.
    • The 6th Economic Census reported 58.5 million establishments, but experts argue this likely undercounts home-based and highly mobile economic activities.
    • The lack of robust data on this sector affects policy formulation for a significant portion of the economy.
• Data Suppression and Delayed Releases: There have been instances of withholding unfavorable statistical reports.
  • A notable example is the NSSO's 2017-18 consumption expenditure survey, which reportedly showed a decline in rural consumption.
  • This survey was withheld from release, citing data quality issues. Such actions raise questions about the independence of statistical institutions and the transparency of the statistical system.
• Lack of Technological Integration and Big Data Utilization: Despite initiatives like Digital India, the integration of big data and advanced analytics in official statistics remains limited.
  • For example, while countries like Estonia use digital footprints for real-time economic indicators, India's statistical system still heavily relies on traditional survey methods.
  • The potential of GST data and digital transactions for enhancing economic and agricultural statistics remains largely untapped.
• Environmental Data Gaps: India lacks comprehensive, regularly updated environmental statistics.
  • For instance, the country's last comprehensive forest survey using ground-truthing was conducted in the 1980s, with subsequent surveys relying primarily on satellite data.
  • This affects the accuracy of forest cover estimates and carbon sequestration calculations, crucial for climate policy and international commitments.

What Measures can be Adopted to Strengthen the Statistical System in India?

• Comprehensive Legal and Institutional Reforms: Enact a new Statistical Act to replace the outdated Collection of Statistics Act, 2008 (amended in 2017).
  • Strengthen the autonomy of the statistical agencies like National Sample Survey Office through legislative reforms, ensuring they have the authority to release data without political interference.
  • Streamline National Statistical Service to streamline recruitment and career progression for statisticians across all government departments.
  • Establish an independent regulatory body to oversee data quality and methodological standards across all official statistical products of the government.
• Modernization of Data Collection and Processing Infrastructure: Implement a nationwide digital data collection system, replacing paper-based surveys with tablet or smartphone-based data entry.
  • Develop a centralized, cloud-based data storage and processing infrastructure with robust security measures.
  • Integrate various administrative databases (e.g., GST, income tax, land records) with the statistical system for more comprehensive and frequent data updates.
  • Establish real-time data pipelines from key economic indicators (e.g., high-frequency indicators like power consumption, e-way bills) for timely economic monitoring.
• Capacity Building and Skill Enhancement: Create a dedicated Statistical Training Institute for continuous upskilling of government statisticians at all levels.
  • Develop partnerships with leading global statistical organizations and universities for knowledge exchange and best practice adoption.
• Implement a mandatory statistical literacy program for all concerned government officials involved in policy-making.
• Enhanced Data Transparency and Accessibility: Develop a user-friendly National Data Portal providing access to all official statistics, including metadata and methodologies.
  • Implement a pre-announced calendar for all major statistical releases to ensure predictability and reduce speculation.
  • Establish a public consultation process for major methodological changes in key statistical products.
• Strengthening Subnational Statistical Capabilities: Create State Statistical Innovation Funds to encourage modernization of state-level statistical systems.
  • Implement a ranking system for state statistical capabilities to foster healthy competition and improvements.
  • Establish regional data processing centers to support smaller states and union territories.
• Blockchain and Distributed Ledger Technology: Implement blockchain for maintaining an immutable audit trail of all changes to official statistics.
  • Use smart contracts for automated data sharing agreements between different government departments.
  • Create a blockchain-based system for secure and transparent conduct of household surveys, ensuring data integrity from collection to publication.
• Big Data Analytics and Alternative Data Sources: Develop methodologies to incorporate big data sources (e.g., mobile phone non-personal data, social media, web scraping) into official statistics.
• Reforming Census and Sample Survey Systems: Implement a rolling census model, conducting surveys continuously over a 5-year period instead of a single decennial exercise.
  • Develop a master sample frame updated annually for all household surveys.
  • Introduce adaptive survey designs that adjust sample sizes based on real-time data quality indicators.