Are you considering conducting research in and around the St. Louis, MO area, across the United States or around the world?
Are you looking for expert advice in designing a web-based study that will help you obtain the highest quality data?
Are you concerned about issues such as having an appropriate sample size or response rate for your research or being able to detect differences between subgroups within a population?
Are you looking for a skilled and highly experienced team of analysts to help bring validity to your research project?
We can help!
The Research & Planning Group has administered hundreds of online surveys to hundreds of thousands of participants over the last four decades using the Qualtrics platform, and we can help you through the entire process!
Online surveys have become a fairly standard approach for quantitative research because they are:
- Fast to set up, launch and analyze, with many studies able to be fully fielded and into analysis in a matter of days or weeks after launch.
- Cheaper and easier than mailed or paper surveys to administer, collect and tabulate.
- Able to introduce visual stimuli such as advertising concepts, branding or logo concepts, videos or so much more!
- Scalable to regional, national or international participants without adding significantly to the cost since no travel is required.
- Convenient for participants and accessible through a wide variety of devices, including mobile phones.
We use the Qualtrics platform at RPG because we’ve evaluated the alternatives and we believe Qualtrics is the gold standard of online survey software. We have our own license to utilize for client surveys, and we are also quite comfortable working within our clients’ licenses if they have their own Qualtrics account.
Our team is Gold-Certified in the Qualtrics platform, which means we are expert users who have participated in many levels of training directly with Qualtrics. We’ve also got over a decade’s worth of experience utilizing the platform and know many of the power user tips and tricks that can make surveys far easier to analyze and transform into useful insights.
The ultimate purpose of online survey research is generally to do at least one of the following:
- Take measurements to inform performance metrics such as Overall Satisfaction, Likelihood to Recommend or Net Promoter Score.
- Measure sentiments such as touchpoint satisfaction, switch intention or affinity for a brand or organization.
- Test newideas or concepts through descriptive language or provided stimuli.
- Prioritize new services, features or benefits.
- Hear the voice of participants expressed in scaled data or frequency of response, enabling the “wisdom of the crowd” to help prioritize action items
- Compare subgroups within the population through breakout variables or profiles.
- Examine trends over time using longitudinal comparisons between days, weeks, months, quarters or years.
- Implement a customer recovery program using real-time ticketing for disaffected or unhappy customers and loop it into your CRM or customer service system to allow for rapid response and immediate service recovery
- Build a customer panel using multiple surveys to build up customer profiles and allow you to have a robust group of customers waiting to give you feedback at a moment’s notice
There are, of course, many other possibilities for online surveys, and we are always happy to provide input on how they can be best designed to meet your informational needs! One of the reasons we utilize Qualtrics is because we’ve found it can be programmed to incorporate just about any survey design or application we can think up… and we’re always excited about facing new challenges!
Here are a few things we’ve learned about online survey research:
- Online surveys need to be short, but the actual guidelines depends upon the sample. The general rule of thumb for online surveys is that they should be able to be completed within 5 minutes and have no more than 2-3 open-ended questions. Longer or more complex surveys tend to generate higher rates of early terminations or inattentive behaviors (straight lining, yea-saying, nay-saying, ignoring open-ends, etc.), and those problems tend to rise exponentially the longer the survey takes to complete.
But this varies depending upon how you’re recruiting respondents. If you’re using a research panel, for example, where participants have already agreed to be available for surveys and are receiving compensation for completing surveys, you can generally increase the length to up to 8-10 minutes with only minor effects, provided that the survey is fairly engaging and not overly complex in design. (Highly complex surveys generally require higher incentives than research panels tend to offer their participants and are often best completed through a sample specifically recruited for the study.)
If you’re using a sample sourced from an email list, the shorter you can keep your survey, the better. Our general rule of thumb is to include no more than 10 simple questions about your topic plus a handful of questions for screener participants and collecting demographic data. But again, if you are talking to heavy users or highly-engaged audiences, you can increase the length or complexity somewhat without too much concern regarding ill effects.
But with all that said, the most effective online surveys tend to be the shortest and most focused, and that’s one reason we recommend taking the time to articulate a specific purpose and detailed objectives before the work of writing questions begins. - Online surveys need to function well on mobile devices, because that’s where most of your responses come from. Depending upon your population, you can expect anywhere from 40-80% of your responses to come from mobile devices (primarily smartphones). If your survey is not optimized for this experience, you will likely either have a poor completion rate or you will have serious concerns about the data’s quality.
One reason we utilize Qualtrics for online surveys is because it automatically optimizes surveys for smartphones and allows our research team and clients to test the desktop and mobile designs simultaneously in a preview window. - Online surveys should be highly focused and stripped free of any questions producing useless data. Over the years, we’ve come to question the value of collecting conventional demographic data at the end of surveys – in a survey where we’re concerned about opinions, do we really need to slice and dice the data by income, race/ethnicity, education and gender?
It’s a question worth asking, and if your online survey has a standard 10-question demographic block that’s not going to provide any useful understanding of the information, we’d recommend you drop it. Your respondents won’t miss it.
We have dropped these questions from many of our surveys, and we don’t miss them either. The upshot is that this has allowed us more time to ask questions that are actually relevant to our topic of study, and - Online surveys need to build in time for pilot testing, baseline-setting or validation. Far too often, the need for quick results leads those conducting studies to distribute surveys that haven’t been properly tested or validated. Unfortunately, a survey instrument that isn’t well-designed and carefully tested can generate misleading data, and that may lead you to draw the wrong conclusions from data that appear to be trustworthy at first glance.
If you’re trying to support a decision with good research, it’s important to take a few days or weeks to ensure that the survey tool you’re using is doing what you want. Some of the things that seem easiest about survey design, like developing scales or writing questions, are filled with pitfalls that can greatly change the results of your research, and there is always the possibility that a survey will generate extreme results that aren’t repeatable in future applications of the same survey.
The best way to hedge yourself against them is to employ good practices like using pilot tests to ensure your survey instrument is working as planned, to set baselines from a first application that can be matched against known parameters, and to validate your survey through other forms of research.
And while we all want insights from the research we conduct, be wary if a survey keeps giving you results that run counter to what other sources have shown, because chances are good there’s something in the survey or your sampling plan that needs to be examined before the data can be accepted. As we often tell our clients at RPG, “if a survey’s results surprise you a little, you probably did something right. If the results surprise you a lot, you probably need to try again with a different survey.” - Online surveys should be paired with qualitative research to ensure you’re getting the best value out of the information you’re seeking. Have you ever pored over a bunch of statistics and felt like you’re only getting part of the story or that your data aren’t actionable?
It happens all the time, and that’s one reason we recommend pairing quantitative research with some sort of qualitative component, whether it’s an open-ended variant where some of the closed-ended questions are changed into forms requiring more thoughtful answers from a select subgroup of respondents, broader in-depth interviews to provide additional depth and detail on your topic, focus groups that can provide a lot of breadth on your topic or some form of ethnography or behavioral observation that can focus on getting beneath the surface of your participants’ stated attitudes and generate tremendous, original insights.
At RPG, we often design and conduct online in-depth interview studies from start to finish with heavy collaboration from our clients. While we are the experts on conducting research, we know our clients are experts on what we do. When we work together as a team, we can ensure that we get the best possible information to help our clients uncover what they need to know through research.
If you’d like to learn more about our approach towards online in-depth interview research, feel free to contact us. We’re always here to be a resource, and we’re happy to help you however we can!