When users report problems, is this really the problem they are experiencing? When users talk about requirements, is it really what they want? It may not be true. For example, a company once conducted a study. They asked users about their preference for mobile phone color through a questionnaire. When asked whether they would like to buy a red mobile phone, many users answered yes. However, after a questionnaire, the researchers told users, “You can choose between a red phone and a black phone as a gift to take away.” It turned out that the majority of users still chose the black phone as a gift, even those who said they would rather buy red.

For users, you can’t trust everything they say, and that’s the hard part of user research; Because everyone’s behavior is not always rational, users in the study sometimes cannot give accurate answers, even some simple questions; This may sound crazy, but it’s a common problem in in-depth interviews, tests, questionnaires, focus groups, etc. For example, in a study of a banking APP, when users were asked about the ease and security of logging in, different research methods got inconsistent results. When we ask users in the process of using the APP, they say they don’t care about security, but want a more convenient way to log in. But when another group of users were asked through a questionnaire, they were told the opposite: they were more focused on login security, even at the expense of convenience. Most users will probably encounter such inconsistent results, either in their research or in their application. By understanding the real reasons behind the discrepancy between what users say and what they actually do, understanding the influence factors that distort users’ cognition and behavior may help us better avoid them in our research and thus improve the authenticity of users’ answers.

Why is there a discrepancy between what users say and what they actually do?

Behavioral psychologists have long studied that humans aren’t perfectly rational animals, and that our behavior is actually influenced by a range of factors, from emotions to cognitive biases. In order to make our irrational choices seem more reasonable, we often find all sorts of excuses, such as eating enough to give us the strength to lose weight. There are many cognitive biases in life, but three are the most common and often affect users’ behavior.

1. Society celebrates sex and conformity

Social desirability refers to an individual’s attitude or behavior of being praised in order to impress others. That is, people present themselves as they think is “right” rather than reacting to the situation. In the questionnaire survey of banking APP, users may think that since it is a banking APP, they should pay more attention to login security rather than convenience, just like everyone knows that they should eat healthily.

In order to avoid social approval, we should avoid setting biased questions with “right answers” as much as possible. In the research, users can ask questions in a neutral tone, set some open questions as far as possible, and set equal options for selective questions. For example, in testing, we ask users some questions about product features. We can ask: What do you think of this feature? What do you think of the convenience of this operation? Not: Don’t you think this feature is awesome? Do you think this operation is very complicated?

Conformity is another factor that affects user behavior. The so-called conformity effect refers to that when individuals are influenced by the group, they will doubt and change their own views, judgments and behaviors, and change in the direction consistent with the majority of the group, so as to keep consistent with others.

In the classic Asch’s Conformity Research conducted by the social psychologist Asch in 1956, a real subject sat at a table with six to eight other people (who were actually assistants of the experimenter) and the experimenter presented them with three line segments of varying lengths. They were asked to judge which line was the same length as the standard line segment in the other painting. Each person took turns giving his or her verdict, and the subject was placed in the second-to-last position. In most trials, everyone gave the same correct answer. But in several pre-determined key trials, the experimenter told the assistant in advance to give the wrong answer. They found that, even when the correct answer was obvious, participants in the key trials agreed with the group on average 37 percent, and 75 percent did so at least once.

In order to avoid the herd effect, we should arrange one-on-one tests or interviews with users as much as possible in order to minimize their “imitation” of others. Researchers should also make sure not to mention other users’ choices or test results. In a focus group study, the moderator should try his best to encourage people to express different opinions and avoid the homogenization of opinions caused by conformity effect.

2. Wish thinking

Another problem we run into is the user’s desire thinking. People often say they think they are, but they are not. Research shows, for example, that 75 per cent of business people believe they have a better sense of justice than the average businessman; Eighty percent of drivers said they drove better than the average driver (including those who ended up in hospital after an accident); Ninety-four percent believe their sense of humor is above average. The psychological tendency to feel that everything is above average is wishful thinking. Another popular way of saying it is the ego bias or the belief that you are better than others. When we are influenced by desire thinking, self-perception, although inconsistent with the actual level, still affects some of our behaviors and choices. For example, in a survey of network security education services, at first most users said that they knew how to protect themselves on the Internet and did not need network security education services. However, in subsequent interviews, it was found that they only knew some basic network security knowledge and most of them were wrong.

This is a very difficult question to avoid, because the user thinks they are saying this, but the user is actually “sincere” in their answer, but they don’t realize that the wish thinking has quietly helped them improve their abilities. To avoid this problem, ask for specific things, not how the user would behave in some introductory scenario; Our research should focus on what users are doing or have done recently, based on real situations. For example, before starting the test, please emphasize to the user that this test is not evaluating the Ta’s ability, but the quality of the product; Users in the process of testing any problems can be said, so that users as far as possible to express the most real thoughts and feelings.

3. Different scenarios and user mentality

People react differently in different environments. We behave differently at home, in stores, or at work. We also react differently when we are tired, excited, distracted, focused; Although scenarios have a strong influence on our mental states, it is still difficult to predict how we will react to a particular situation until we are in it. Just as in the study of banking APP, users will encounter various problems in actual use, such as security setting will increase the complexity of login operation, verification code failure will bring frustration and so on, all of which will lead users to want a more convenient login method, while users who participate in the questionnaire survey, It is difficult to predict what problems you will encounter and how you will react in real-world situations.

Daniel Kahneman explains why we can’t predict our own behavior in his book thinking, Fast and Slow. He believed that people’s thinking activities can be divided into two categories, one is fast thinking, driven by emotion and instinct, and the other is slow thinking, driven more by reason. When we try to predict how we will behave in a given situation, we use slow thinking to weigh the sanity of our choice. Once we make a choice, we are instinctively driven to think fast. We can’t completely stop slow thinking from affecting research results, but we can at least reduce it; Researchers can set up a research environment that is as close to the real world as possible. We can observe the APP there might be some usage scenarios in advance, if the user is often used in some noise and confusion of our APP, we will test them in the coffee shop or a public place or research, even though we cannot create the same scene, that will also as far as possible, the reduction of the user’s usage scenarios, The results will be closer to the truth.

In addition, we may or may not be careful to influence users’ mentality in the research process, which will affect the authenticity of the research results. For example, in the study of network security education service mentioned above, after discussing the hidden dangers and consequences of network security with users, it is found that users have surprisingly high scores on network security. In another set of tests, users were asked to rate their Internet security concerns before and after the interview, and scores did improve significantly. In fact, the two scoring results are useful. The scoring before the interview indicates the initial state of mind of users, while the scoring after the interview indicates that sharing of network security information can provide users with more attention to network security.

How to avoid the influence of these factors and improve the authenticity of users’ answers?

  • Observe users and, if possible, observe how they actually behave in real situations where what they do is much more accurate than what they say;
  • The test should be carried out in real use scenarios as much as possible. It is difficult for users to predict their reactions in different scenarios, because the closer the test environment is to the real use environment, the more accurate the test results will be.
  • Ask for specifics, not how often the user has done something, but when was the last time;
  • Asking about past experience, whether we are optimistic or pessimistic, some of our views of the future are distorted by cognitive biases, and our recent memories are more accurate than others. Don’t ask the user what they assume, ask the user what they have already done;
  • All the questions are given the same choice so that the user doesn’t appear biased, so that they don’t guess the “right” answer, but give a real answer;
  • Set questions objectively, otherwise it’s easy to give the user hints. For example, if you ask someone who brushes their teeth twice a day or once a day, you will probably get a false answer.
  • Knowing more about cognitive biases and irrational behaviors and understanding the influencing factors that distort users’ cognition and behavior can be better avoided in research.

Original link:

http://www.uxbooth.com/articles/design-research/bridging-the-gap-between-actual-and-reported-behavior/

Author: WuHB