BE and the inherent weakness of opinion polls

The behavioural sciences and BE in particular have informed us about how and why we behave as we do; why we may think one thing vs. another and why we don't fit the profile of perfect rational economic man. BE has exposed and defined a plethora of cognitive biases which bring alive our tendencies to behave in certain, often irrational ways. 

With this in mind, opinion polls can be seen in a different light and explain why we might be advised to treat them with caution, or at the very least with a more critical awareness. Without doubt, companies have already considered many influential elements such as priming and order effects, but there are myriad other potential stumbling blocks and biases to bear in mind.

BE shows that, in general, people are biased towards the status quo and want to follow the crowd. So if we see an opinion poll showing what the majority thinks, believes or is doing, then in many circumstances, we will have a tendency to follow this norm, to instinctively follow the herd. What this means is that opinion polls have an element of self-fulfilling prophecy about them, making them unreliable. The government has utilised this social norm bias by designing communications which, for example, emphasise the (high) percentage of people in your area who have paid their tax or completed their census forms on time. For example, HMRC letters which informed people that the majority of people in their area had already paid their tax and reminded people about the importance of paying tax for their local services outperformed the control group letters (which contained no such information) by around 15%.[1]

There is also another intriguing self-fulfilling prophecy element which has been identified. Research has shown that if you ask someone if they are going to do something and they answer in the affirmative, there is more chance that they will feel compelled to do what they said they would do than if they had not been asked at all. This is what behavioural economists call the commitment bias. It has been demonstrated in a number of studies, in particular, where people are asked in telephone surveys whether they intend to vote.[2]

Another very simple cognitive bias which could impact opinion polls is the availability bias, which shows how people predict the frequency of an event based on how easily an example can be brought to mind. For example, people worry more about being involved in a plane crash when they are flying than they ever fret about being in a car accident while they are driving, because they have a more vivid memory of plane crashes than of car accidents - plane crashes are more easily 'available' to them in their memory. More generally, people are lazy - especially when it comes to answering questionnaires - and in our busy world, tend not to spend a lot of time thinking deeply about such things, and are more likely to be influenced by what first comes to mind. On-the-spot responses could be superficial or based on what respondents think is the more 'correct' answer. However, at a later stage with more time to reflect on the issue or when the situation is more private, respondents could produce a different, more genuine viewpoint.

Perhaps the most interesting area of potential influence on opinion polls is the behavioural context in which the questions are asked. A recent session at the 2012 WEF summit in Davos on 'Designing Better Lives'[3] highlighted how identity can change depending on context. We all have multiple identities - we can be parents, children, students, couch potatoes or marathon runners, and our behaviour and opinions can be shaped depending on the context in which questions are asked and answered and by the identity which is in the forefront of our minds at the time. Different contexts can have different mindsets and emotions attached and can prime people in different ways. The session highlighted an example of research which shows that women do less well in exams if they are asked to confirm their gender immediately before starting the exam. So a sharper understanding of the context in which the respondent is answering could be a critical framework for understanding and interpreting responses.

There's another angle on the impulsivity of response; research has shown that humans have a 'hot-cold' empathy gap. In other words, when we are in a 'cold' or unemotional stat,e we can be poor predictors of how we might behave when we are in a 'hot' or emotional state. Hence, consumers or survey participants find it hard to predict their own behaviour even if they want to - context is king!

One way of bringing alive the gap between the hot and cold zones would be to think about your attitudes towards meals you might eat in the coming week, and, depending on whether you answer questions relating to this at a satiated (cold) moment versus pre- or mid-meal in a restaurant (hot), your expectations might be very different.

Another example of hot-cold empathy gaps was demonstrated in a great study[4] by Dan Ariely and George Loewenstein, who found that opinions and predictions of sexual behaviour varied considerably depending on whether an individual was in 'the heat of the moment' or not. Test results from male college students on their sexual preferences varied hugely depending on whether they were in a 'cold, rational, unaroused' state or, let's say, a little more excited.... When asked about their sexual preferences in a cold state, the majority stated they would always use a condom, would never consider date rape, were not interested in a threesome, or having underage sex, spanking, tying their partners up, having sex with an animal, etc. But in a 'hot, excited' state, the answers men gave were considerably more adventurous... For example, when asked (in a non-aroused state) if they would like to tie up their partners, only 47% answered yes. When aroused, 75% said they would like to do this. And the concept of sex with animals was over twice as appealing in an aroused state as compared to a cold state. Overall, for the 19 questions about sexual preferences, students' judgments about their predicted sexual behaviour were 72% higher when aroused than they had predicted when they were in a cold state.

So, what with hot and cold states, identity fluctuations, our desire to follow the herd, our love of the status quo and our superficial memory, opinion polls need to be treated with a new level of insight or critical awareness.