All answers are given in a context
By Tor Berntsen, advisor in change management and communication
I spoke to a CEO who thinks the employee survey they used to work is bad. She says: When I ask my colleagues a question in a survey, I do not really know why they respond as they do. Therefore, we need to have workshops afterwards to find out what they really meant when they answered the questions. I have participated in many such processes and the explanation behind the answers is always individual. The fact that Per Johansen scores 4 points of 5 possible motivation can be explained in several ways:
- He recently had a chat with his leader and received good feedback. If the survey had been completed the week before, he would answer 3 points.
- He is really self-motivated and self-motivated and scores 4 on all surveys wherever he works. He could choose 5, but chose to keep a little back on principle, since everything could be better.
- The department he works in is so small so if he chooses 2, representing what he really means, he knows that the manager will figure it out and make a lot of fuss. By choosing 4, he can continue as before.
- He has only worked there for a few months and does not know everyone so well. There is no reason to complain about anything. 4 seemed like a real number.
- He has just begun working on an exciting project. If the survey was taken at an earlier date, the answer would be 2.
The fact that the questions are quality assured by competent professionals does not prevent the answers from being given in a context. Therefore, the value of the answers is very limited and benchmarking is really an impossible concept. We compare apples and pears.
What information are you looking for? I ask.
She says: “I know that the last changes have led to someone losing good colleagues and that the tasks have changed. And I know it’s hard for anyone to accept this situation. But the market has changed. We had to adapt. And the changes we made were necessary. So this is our new reality. I can not do that. It was not possible to continue as we did. I do not need a survey that tells me that someone experienced the transition as difficult. For that I know. What we need now is to help everyone accept the new situation. And when the changes are accepted, we’ll know what motivates you and how we can support you so that you can be the best issue of yourself.
This conversation inspired us to develop a new methodology for surveys that we have subsequently chosen to call scenario surveys, as opposed to regular surveys that are generic. What is special about scenario surveys is that we start by describing the situation we are in using video, images and text. Then we ask questions about how you as a staff member relate to this description. When the scenario is established, we ask for performance, motivation and well-being. We avoid generic responses like “to a small extent, some degree, largely and so on” The answer options should instead represent attitudes that exist.
We have used this method several times and get significantly better quality of responses compared to generic surveys. Quality in this context means that when we interpret the answers and compare it to a control group, the gap between our interpretation and the explanation from the control group is significantly less than when we use generic surveys. In addition, the attitude effect comes. The fact that all employees get a common “official” description of the current situation makes it easier to accept the changes. Or as the CEO stated: “When we agree on which boat we are in, it’s easier to know the community!”