An avoidable controversy over sample surveys

Source– The post is based on the article “An avoidable controversy over sample surveys” published in “The Hindu” on 25th July 2023.

Syllabus: GS3- Government policies and interventions

Relevance: Issues related to government statistical exercise

News- An article recently raised doubts on the soundness of data collection procedures of some of the surveys such as the National Sample Survey, National Family Health Survey and Periodic Labour Force Survey (PLFS).

As per the writer, most of the surveys from 2011-12 till 2019-21 underestimates the proportion of the urban population or overestimates the rural population significantly.

What are counter arguments against the writer’s viewpoints about India’s statistical system?

Bias in population estimate– The sampling designs of the NSS or the PLFS do not specifically aim to estimate the total number of households or population. Instead, their primary purpose is to estimate the major socio-economic indicators.

The estimates of households or population size are considered auxiliary information. Data users appropriately adjust the survey-based estimates for rural and urban areas separately by using projected population figures derived from the Census.

The estimates of the number of households derived from the NSS align closely with the Census-based figures for households. The writer’s accusation about non-representative samples due to outdated sampling frames becomes less relevant for two main reasons.

Firstly, these surveys primarily rely on the population census lists of villages and urban blocks for sampling. It ensures comprehensive coverage.

Secondly, for sampling urban blocks, the NSS and PLFS utilise the latest Urban Frame Survey (UFS) blocks, covering all towns in the country. This helps address any urbanisation that occurred after the census through State government notifications.

Regarding the classification of geographical areas as rural or urban, all these surveys include census towns as part of the urban sampling frame.

Systematic bias in response rate- Some households don’t not share information in the survey. As household income increases, the response rate tends to decline. This challenge is encountered in similar surveys worldwide.

To address this issue, the survey methodology recommends replacing non-responsive households with other households that are as similar as possible.

However, it is important to acknowledge that the substituted households may have relatively lower income levels. It could introduce some downward bias in the overall estimates.

A majority of the welfare programmes of the government are targeted towards households in the lower income brackets. So, a very low non-response rate in these surveys is not likely to have a serious impact on the overall household level indicators.

What is the way forward to remove the concerns regarding the soundness of data collection procedures of some of the surveys?

Sample design and data quality are two distinct components of a survey. Sample design typically involves careful selection using scientific methods.

To remove the concerns about the representation of affluent households, there is a need to create a list of such households through alternative sources.

Incorporate a representative sample of them alongside the conventional survey of the rest of the population should be done.

Evaluating the coverage of the UFS frame is important to overcome the issue of underestimation of the urban population. Establishing a methodological study unit for further improvements in survey design is also a positive step.

To enhance data quality, strengthening the training of field personnel, conducting field inspections, concurrent data validation, and implementing effective publicity measures are other essential steps.