Lost in translation: Why pollsters go wrong so often

livemint.com:

Subscribe to enjoy similar stories. The recently concluded elections in Haryana and Jammu and Kashmir came with their usual collection of opinion and exit polls. The election results gave us one more example of how these polls can be at variance with what people actually choose.

There is a tendency among scholars to believe that this problem applies only to political and opinion polls. It is important to realize, however, that the reasons for some of these failures lie in the very nature of conversational data collection, where data is gathered through a structured process of conversation between a data collector and respondent. As most sample surveys and India’s population census rely on conversational methods, it is useful to examine the nature of errors that occur in this process.

First, these conversations involve two parties: the interviewer and the respondent. They come into the discussion with very different perspectives and backgrounds. One source of error arises from the fact that the two parties often do not speak the same ‘language.’ By language, we do not mean merely linguistic considerations, but also the nuances of understanding.

Take, for example, the question, ‘are you working?’ Economists and statisticians understand ‘work’ to be any effort that yields pay or profit. However, in general conversation, the understanding of what constitutes ‘work’ can be very different (for example, it also encompassing notions of ‘decency’ and ‘adequacy’). Thus, young individuals preparing for government jobs may describe themselves as being unemployed, even though they may be earning from part-time tuitions.