• Home
  • Continuous Listening

Deliberate open-ended questions for high response rates and depth

How was your day? What are we doing next weekend? What do you think of this idea? How can we best solve this problem? Open ended questions. People are tired of closed questions that surveys present us with: the well-known survey fatigue (interesting report dating back to 2013 by the CIPD). This is understandable. After all, we want to be able to give our opinion, to contribute to the issue at hand. And we don’t want to be forced to choose from a limited number of answers. Open-ended questions also let people know that you take them seriously, that you trust them. Gallup has been researching this for many years: employees love, Einstein quote futureno require, need to be take seriously for their opinions. You also get a higher response rate and a much better response; they take answering your question seriously. As a result, your research becomes more reliable. Don’t forget to ask more questions about the future and present, than about the past; the past doesn’t resonate strongly enough with people.

Closed question with open answer

Even if you ask a closed question (score, multiple choice, statement), it is almost always better to offer the open text field as well. You must then explicitly ask for an explanation of the score or of the answer to the multiple-choice question. By asking for an explanation, the score is given in a more considered way, which makes your results more reliable. Moreover, you learn about the reasons that lie behind scores and you can finally, reliably interpret your numerical results. This also enables you to analyse whether low scores have been given by mistake, while the open answer indicates the opposite; or vice versa. Your research results will improve, which in turn will lead to better follow-up and targeted, fast action! Managers need to be helped by your insights, not overwhelmed by data and additional home work. Just adding an open-text field to a closed-end question is not an option. If you don’t give the open answer meaning by a clear, guiding open-ended question, why do you expect respondents to give it meaning?

So, instead of asking: “How do you assess ABC?” and allowing a score or throwing in a text field,
ask this: “How do you assess ABC at this moment, and can you clearly substantiate that?”

Don’t limit them, or yourself

The current technology of survey providers is very limited in processing open answers. Therefore, they often discourage this, and you are left with the big question ‘what is behind the scores’. Or they limit themselves to counting words, topic modelling and trying to discover combinations of words. Consequently, the ease of processing the data for such providers is more important than the quality and depth of your research. Even worse, more important than the needs of your participants; there is hardly or no specific room for open answers. Open answers are driven by the open-ended question, not by a functional text box to put in ‘anything you want’. They can’t say what they do want to say. The section below is from the website of one of the market leaders for employee surveys, showing that after the survey, you have to find out for yourself what’s behind it.

the why behind numbers

 

Instant useful information

CircleLytics technology, on the other hand, has been developed to process open answers and convert them into information. Information that you can use immediately. Useful top 5s for instance. You can see right away what was said by which target group and why. This means you will get the most out of the open answers, making sure participants are heard.

Prof Neely (University of Cambridge): “People often reveal their true thoughts and feelings in the open-ended comment boxes. In general, the content of these comments offers a much more reliable predictor of a people’s behaviour.”

How do you deal with open answers?

What is the difference between open answers processing by a survey tool and by CircleLytics? We will explain this using a simple example. After a reorganisation of HR, in which they started working de-centrally as business partners, management and HR wanted to investigate the results. The question was simple: “How do you assess the effects for you of decentralising HR and can you explain this?” Approximately 3,000 employees gave a 1-10 score including their explanations. The average score was high. The answers were then grouped based on traditional word count and topic modelling. These are simple ways of processing texts which made it possible to group 14 themes and to count which words were mentioned most often. The latter resulted in a nice word cloud. That’s it?

For surveys, yes. For CircleLytics and participants, this is where it all starts. Our tool allows for a unique second round per question. In this second round, the unsorted answers are sent back to the participants in sets of varying compositions to maximize diversity of perspectives. Employees (or other participants such as customers, members, etc) can rate these and give them a score of -3 to +3, choose key words and explain their score. Do you know how much they enjoy doing that, how happy that makes them, to learn from each other’s answers and enrich these?

Sentiment & Semantic Analysis

They give a sentiment, a weighting, to the answers of others. You immediately have your real-time sentiment & semantic analysis and natural processing of language. No algorithm can beat the human mind in processing something so complex as natural language. To humans, language comes natural, yet to algorithms it’s one of the most complex things to capture in a meaningful, actionable and reliable way.

By selecting key words, the human minds (the participants) give those words even an extra weighting and extra semantic, contextual value to the data. That is an enormous enrichment; even before our AI/ML/NLP technology starts to perform analyses, hundreds or even thousands of participants are already doing the natural language processing. In practice, this proves extremely attractive: more than 70% of the participants rate more than 15 answers from others. Participants love this way of answering questions, learning from others and structuring the output, and give this two-round method a report mark of 4.1 on a 5-point scale. Very different from the fatigue that plagues surveys….

Now let’s see

Back to our example: 3,000 participants assessing an average of 30 answers, is almost 100,000 thought processes in just a few days! That 2nd round is called collective intelligence. This gave the client a whole new insight. Instead of 14 themes, it turned out that after the 2nd round, only 3 themes were favoured and supported. On second thought, the participants focused on those 3 themes, not on 14 at all. Moreover, a fourth, new theme emerged, which only a few people thought of in round 1, but was positively scored by a large group of participants in round 2!

The theme was about someone’s observation that HR business partners themselves were having a hard time…. A few had empathy for this in the 1st round. A regular one-round survey, a word count and topic modelling simply did not reveal the theme. Only through the 2nd round of CircleLytics could participants see the opinions of others, learn from them and gain a deeper understanding. On closer inspection, they thought differently. An unprecedented enrichment of the research. And a wonderful example of deep democracy in which a minority opinion is clearly revealed.

Think again

That is why the CircleLytics technology is not called a survey; it’s called a dialogue, based on deliberate open-ended question and human (and after that AI/NLP) processing of the open answers. Because dialogue requires that people are prepared to think differently about your question. They learn from the answers of others. People think better in the second instance. They usually think fast first, and then slow down (also read Kahneman’s research), which gives your research more depth. You also come up with subjects that no technology had come up with. Topics that are sometimes only mentioned by a few, and that cannot emerge without the collective intelligence of all brains together.

So when would it be useful to have a survey?

Forget the word survey for a moment. Based on our experience, we advise to determine the design per question in CircleLytics. This means adding a closed scale, presenting a multiple choice, or offering a text field. You can adjust the question when you add a text field. You can now explicitly ask participants to fill in the text field and what you expect them to do. Finally, you can determine whether the question will be included in the 2nd round and set the deadline for this. This is how you can combine the old survey with the new possibilities of the open text field and the 2nd round. If a question leaves no room for thinking for yourself but is just a score or ticking of a predetermined answer, you leave out the 2nd round, and possibly even the text field. This can all be done in a single tool. Tips for formulating strong questions, can be found in our WhitePaper with design principles.

Analyses and results

In conclusion: the response rate and quality are higher and, more importantly in our opinion, you will achieve accurate results and a reliable interpretation of the figures. You can also further analyse the results in CircleLytics by using the unique Weighted Word Count that takes sentiment (score/selection of words) into account. This leads to completely different insights than the old word count.

Everything has an expiry date. The regular surveys you are used to, something that causes fatigue among participants and shows fragile quality, need renewal.

We would like to invite you to get to know the power of people, based on (mostly) open questions, with the CircleLytics Dialogue Solution.

Request demo

 

Dialogue

HR has to go through it every year, quarter or month, and so do the employees: the employee survey. Collecting employee scores for 10 to sometimes 80 general questions. No wonder employees are not happy with these surveys. It’s unclear what happens next, and they can’t give their real opinion. As a result, management and HR are burdened with what to do next. The big question is: what is the reason behind the survey figures? And, what will we do?

 

If you know why, you can anticipate

In the meantime, 200+ organisations have worked with employees to find out the why, their true opinions, the story behind the figures. The story of the people themselves. Especially now that employees combine home and office, and are seriously considering changing jobs, it is even more important to know what really matters to them. General questions with a score do not give us that answer. The next step is to start talking to your employees.

 

HR Director: “Gallup research has shown for years that employees are more motivated if they are allowed to express their opinion. That turned out to be a real goldmine in our organisation.”

 

In dialogue, you can really listen

The questions you ask your partner, friends and family are not limited to multiple choice and scoring questions. Why, then, are they used for employees? From now on, you can also enter into dialogue with employees using open questions. When you use open questions, you tell your employees that you trust them, that you invite them to be open, and that you show you’re interested. Open questions will get you answers that matter, that make the difference, that give insight. Check out our white paper with all the design principles for formulating solid open questions.

 

He who asks, receives. And then? 

Our customers, mainly HR and management teams, use CircleLytics to ask open questions to 10 to 100,000 employees. We are 100% inclusive; everyone simply participates via a link. The answers may be given anonymously. Afterwards, they can see the anonymous answers of others and indicate whether they find them valuable or not. This ensures structure and priority.

 

What is the result of listening to employees?

Happy employees. You take them seriously and that makes an impression. What’s more, you learn a whole lot from what they say, how they solve problems, and how they make improvements visible. Just because you ask them. Management is very enthusiastic about this because they now understand what lies behind the figures and HR gives them the information they need to make better decisions.

Interested in the CircleLytics online dialogue? Visit our website

 

Maurik Dippel, CEO, Co-Founder

Rules. Which ones really hinder employees, do we understand why, and can we solve them? How do you as a works council, HR, manager or CEO ensure that employees actively think about this, come up with ideas and help set priorities? After all, they know exactly what it is that frustrates them every day.

I remember a lecture on Mintzberg and the usefulness of bureaucracy to manage chaos in growing organizations. Today, it is thought that we need to be flexible, embrace bottom-up decision-making, servant leadership and agility precisely because markets are so fickle. Onward! Right? Down with rules, then? Freedom and trust so we can be creative and solve problems together? As Seth Godin pointed out in his book and TED talk Tribes, people are most easily led in the direction they always wanted to go. In a McKinsey study on stability versus agility, the authors state that modern leadership is the art of removing procedures, structures, and rules that get in the way of change, and keep what is needed.

And which rules and procedures should remain? Which ones are in the way of people changing in a direction they have wanted to go for a long time? 

The remarkable thing is that we don’t really appreciate change. At least, not if we have no influence on it, have no part in it, or have no co-ownership of it. We shy away from change when we have no role in it, according to, for example, research from the American Psychology Association on Well-Being. Stress increases as employees see the values they implicitly live and work by threatened. And vice versa. Values are therefore smarter to deploy than rules! Netflix discovered this much earlier. They follow a very simple rule: Avoid Rules! People over Process! However, a close examination reveals that Netflix only allows rules if something can lead to irreversible damage. Netflix apparently allows a margin of error, so to speak, to learn. Fail fast often is what start-ups and gurus like Gary Vaynerchuk sometimes say. So do we. And rules hold back this learning process.

In his book Doorbreek uw bedrijfscultuur (Break through your company culture), Rudy Snippe talks about the self-referential ability of organizations. They maintain their own system, and rules, and confirmative bias ensures that people are mainly open to signals that confirm them right and do not threaten their already existing perception or views. Managers or employees who say: “this is the way it has to be, because those are the rules, don’t ask me why” or “it doesn’t work that way here” are not wanted by anyone, except your competitor. Well, not even them..

There is evidence that €5 billion of administrative burden is wasted in the healthcare sector in the Netherlands alone. It’s quite bold to state that this ‘can therefore be scrapped’, because rules also provide structure, support accountability and make risks transparent and manageable. And last but not least, the legislator also wants to see all kinds of things complied with, such as requirements for a license, tender, etc.

Do we leave it at that? No, definitely not. What if you had a smart online dialogue twice a year and asked your employees which rule they think could be abolished and why? What if you could let employees respond to each other’s ideas, and you would know which ones could potentially be abolished, and more importantly, why? What if you also asked which rule allows them to do their work in a stable and structured way?

One of our clients tackled the ‘rules’ topic by simply asking its employees (nearly 3,000 employees) to name rules that only bother them and the organization and do not benefit the client, prioritize others’ answers and comment why you favour their answers more or less. The outcome was clear and could be divided into:

1 rules that hinder employees from doing their jobs and unexplainably still exist;

2 rules that are inconvenient, but serve a purpose;

3 rules that need to be looked at more seriously but have the appearance of not being so.

Anything under 1 can be simplified or removed after further analysis. Note the chain effect: ask the question where the rule got its origin. Do we still understand if the rule is outdated and unnecessary? Conduct further online dialogue in a small(er) group if necessary.

Anything under 2 is explained in more detail. Apparently, the main thing you need to make clear to employees is that these rules exist and why compliance is necessary. Realize that they apparently didn’t know that. Therefore, go over why they didn’t know that? Why are there rules that are unknown, untrained, and/or untraceable? You can also train important rules smartly with groups of employees via dilemmas. Contact us if this appeals to you.

Finally, anything that falls under 3 should be analysed more closely. You can have deeper online dialogues with the group that identified these rules to clarify what to do with them. Again, what is the origin, why don’t we understand why we have these rules?

You get the idea. Clearing clutter and making important rules visible are both necessary. Something you simply can’t pass up. You don’t want to wait for this: you either want to get rid of rules or see them enforced. There is nothing in between, in our opinion. So neither work, nor the customer, nor your employee falls between two stools.

How do we proceed: We can set up a program with your organization or department for the above challenges. You will have the first results with groups of 10 employees up to 10,000s within 2 weeks.  You will be amazed to see the results: a quick return on investment. This will not only be reflected in money and agility, but also in higher employee satisfaction and commitment. The best companies in the world, have a foundation based on the commitment of employees. At the time, not the CEO of IKEA, but an employee came up with the idea of removing the legs of the LÖVET table, in order to develop “flat packaging” as a smart, customer-friendly logistics solution. Don’t underestimate employees, they have ideas, they want to improve their work and they know which rules can be thrown out and which ones are badly needed better than anyone.

Fancy a challenge? Contact us today to get started tomorrow.

Vragen

 What are right questions to ask employees? And how do you get more insight from the participants’ answers? How and why did they come to their choice? What is the real underlying story? Or does that linger under the surface?

The right questions will lead to a successful round of questions

Asking the right questions sounds simple, but it’s not. Countless surveys you have ever distributed as an organization show this. The response rate is low, the answers hardly give the desired results and the frustrations of employees only increase because they have the feeling that ‘nothing is being done’ with their response.

Therefore, we won’t use the word survey. We talk about challenging, listening to each other, working together, where the approach is to engage in a conversation with each other. In question rounds, the right questions are critical to making a question round successful. By successful we mean; whether the outcomes lead to solutions that contribute to the purpose of the questions.

Coming up with questions is difficult

We found that many organizations struggle with asking the right questions. To this end, we conducted research in recent years at dozens of organizations with many tens of thousands of employees, examined studies and designed 100s of questions. We asked them how they are doing, what could be improved or even what needs to be improved.

The 2nd unique round of CircleLytics is used to challenge and inspire each other to think deeper about the questions presented. Organizations such as Fivoor, Inspectorate for Health Care and Youth, Movisie, Unilever, Municipality of Utrecht, HR Community, RSM (global) and several others engaged in a dialogue with employees and their network in this way.

In recent months, dozens of organizations have surveyed 10,000 employees about how they are doing, and what could or should be improved, including what work can be done at home, or not, and why? How much of the work should be done at home, how much at the office? What do employees want, what do they demand? How much room does the labour market allow for not meeting the employees’ demands or wishes? Will employees continue to have the same wishes and demands when it comes to working at home or in the office? What if this changes? What is the influence of the government and social developments when it comes to choices regarding working at home or at the office?

Together with several client organizations, we designed questions, did research and read studies. As a result, we designed top-notch dialogue questions for you and your organizations. You can use these unchanged, enrich them or you can make a selection.

People, society and organizations are currently facing unprecedented circumstances. Insecurities regarding health, finances, jobs and a future for you and your environment go hand in hand. In addition, home and office work must be combined and that in itself is complex enough without corona . Societies and countries are also under pressure to keep their heads above water, to be accountable to parliament and citizens and to ensure that healthcare is or will be up to standard, to offer confidence to all. And then there is the labour market, accessibility of offices, cost of office space versus working from home, etc. Organizations are already making different choices, like Lloyd’s and Goldman Sachs do in this article.

In this sense, we are part of a great experiment in which many people and agencies have opinions, point to each other and are constantly confronted with new, mostly uncertain information. Uncertainty and forms of social isolation can lead to stress and mental problems that transcend normal absenteeism. This can result in a serious loss of confidence, commitment and productivity.

CircleLytics has developed top-notch questions that allow the leadership of the organization, HR, the works council and all employees together to determine what is going well, and what can or must be improved. Contrary to the various corona surveys, the CircleLytics questions reveal the scores: why and how employees think about ‘how to do better’. This provides deeper insights into how to make sound decisions compared to using scores from an old-fashioned survey. It also results in collaboration: employees learn from each other and listen to each other. Something that is desperately needed in these times of little social interaction.

People need trust and connection. You increase this by taking them seriously, showing them that their opinions matter and connecting them in the 2nd round.

We designed a set of questions for dialogues with managers and a set for dialogues with employees. These questions are incorporated into CircleLytics and can be used immediately. We recommend that you don’t limit the dialogue with your people to just one because conditions, markets and uncertainties are constantly changing. You can enrich the questions with matters that apply specifically to you.

An example question:

“To what extent is information from leadership clear and available to you?”

Enrich this question with, for example, the following:

“And please indicate what improvement you see for a higher score next time.”

Or:

“And please explain your score with a recent, clear and telling example.”

Another example question:

“What is the biggest challenge in leading your team when you work from home?”

Enrich this question with, for example, the following:

“Describe a situation that other executives can learn from the most.”

or you can give even more direction to the participant:

“In answering this question, don’t focus on video conferencing/IT but on the social side of your work.”

The first round of questions will generate scores and quality answers. These serve as input for the second round of questions in the Circlelytics dialogue. This is how we challenge participants to think more deeply about the questions presented earlier. They learn from each other and listen to each other. This brings about collaboration and creates support for final decisions. A dialogue emerges and the power of an organization’s collective intelligence now comes to the fore.

Ask it!

Ask managers and employees the right questions, show that their opinion matters and connect them in the second round of questions, so that you can respond as an organization to the real needs that are felt by employees. The right questions help management, Human Resources, the works council and all employees to gain more insight into what is already going well, what can be improved and what really needs to change. And make sure you repeat these rounds of questions regularly, because circumstances are constantly changing.

Looking for more information on these dialogues for your employees and/or managers and are you ready increase your insight immediately? Then email us and start your Proof-of-Concept today.

 

Request demo

 

Back to top
Close Offcanvas Sidebar