June 11, 2015

Tips for Creating Web Surveys for Completion on a Mobile Device

By Kyley McGeeney

Updated July 1, 2015

PM_2015-06-11_web-surveys-on-mobile-01“If you’re doing a Web survey, you’re doing a mobile survey,” according to Michael Link, chief methodologist for Nielsen, recent American Association for Public Opinion Research president and a leading authority on mobile surveys.1 Indeed, in Pew Research Center’s American Trends Panel, a nationally representative, probability-based panel designed to be primarily Web-based, fully 27% of respondents completed their most recent survey on a smartphone (another 8% used a tablet to do so).

With so many respondents taking Web surveys on smartphones, creating surveys with smartphone respondents in mind is critical. This includes both writing the questions with mobile respondents in mind and ensuring that your software properly renders the questions regardless of the type of device respondents are using.2 If surveys aren’t designed for completion on a smartphone, there can be data quality issues such as inaccurately recorded responses, lazy answers and skipped questions.3 Furthermore, if they become frustrated, smartphone respondents are more likely than others to abandon a survey altogether.4

Luckily there are things researchers can do to make surveys more smartphone-compatible. The following are eight tips for creating better surveys for completion on a smartphone.

1. Software should be mobile optimized

Mobile optimization means the software automatically detects the device used, specifically the screen size, and adjusts the layout of the survey accordingly. The font and spacing are larger, as are any buttons that need to be pressed, so that respondents don’t need to pinch to zoom. Additionally, there is no horizontal scrolling, but there may be vertical scrolling.

2. Shorter is better

This refers to both the number of questions and the questions themselves. The longer the survey is, the more likely it is to lose respondents – true for any survey, but especially true for surveys taken on a mobile device. And because smartphone screens are small, shorter questions and response options make it easier for smartphone respondents to read and answer questions, which should improve data quality.

3. Avoid fancy features

It’s tempting to include features such as sliders and spin wheels in surveys. However, research has shown that these kinds of features are difficult for mobile respondents to use correctly because they require a high degree of dexterity.5 They also may require more time for respondents to use than simpler formats. It’s best to stick to radio buttons, checkboxes or text boxes, if possible.

4. No grids

Surveys presented in grid format are meant to use space efficiently by placing questions and response options into a matrix design. But grids also have drawbacks on all devices. For example, researchers have found that grids make it easy for respondents to simply choose the same response for each item in the grid, a phenomenon known as “straightlining.” These data quality issues are even more pronounced on smartphones, as grids often require both vertical and horizontal scrolling, meaning that not all questions and response options may be visible at once.6 These issues can also cause smartphone respondents to leave the survey.7

5. Ask multiple questions on the same screen

If grids aren’t ideal, then what is a better option? The answer: It’s okay to put multiple questions on the same screen.8 Previously it was en vogue in Web surveys to put one question on each screen. But slower mobile load times can unnecessarily burden smartphone respondents. One way to get around this is to group questions about the same topic on the same screen.9

6. Maximize use of the smartphone screen

There is only so much real estate on a smartphone screen, and it is precious. Avoid cluttering up the screen with logos or graphics (they take longer to load anyway). Additionally, position navigation buttons at the bottom of the screen so that respondents are forced to scroll past all questions and response options; that way nothing is missed.

7. Use a unique URL in the survey invitation

PM_2015-06-11_web-surveys-on-mobile-02Don’t require respondents to enter an access code, username and/or password to access a survey. Doing so creates one more barrier to completing the survey. This is especially true for smartphone respondents, as these requirements mean respondents will have to switch back and forth between the survey’s invitation email and the mobile Web browser. For an easier user experience, create unique URLs for each respondent so that when they click the link in their invitation, they are automatically taken directly to their survey and can begin immediately.

8. Invite respondents through a text message

As long as you have explicit consent to send text messages to respondents, consider sending survey invitations via text message in addition to email. Include the survey URL in the text message so respondents can click directly to the survey instead of having to wait for them to check their email.

For further reading on this topic please see the American Association for Public Opinion Research’s (AAPOR) report of the Task Force on Emerging Technologies in Public Opinion Research and the AAPOR webinar “Smarter Smartphone Surveys 201: Data Collection Methods and Survey Design Considerations.”

  1. Link, Michael, Jennie Lai and Kelly Bristol. 2013. “Accessibility or Simplicity? How Respondents Engage With a Multiportal (Mobile, Tablet, Online) Methodology for Data Collection.” Presented at the Annual Conference of the American Association for Public Opinion Research, Boston.
  2. Mitchell, Nicole. 2015. “The Changing Landscape of Technology and Its Effects on Online Survey Data Collection.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL; Barlas, Frances, Randall Thomas and Patricia Graham. 2015. “Purposefully Mobile: Experimentally Assessing Device Effects in an Online Survey.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  3. Antoun, Christopher. 2015. “Effects of Mobile versus PC Web on Survey Response Quality: a Crossover Experiment in a Probability Web Panel.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL; Barlas, Frances and Randall Thomas. 2015. “The Mobile Influence: How Mobile Participants Affect Survey Results.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL; Mitchell, Nicole. 2015. “The Changing Landscape of Technology and Its Effects on Online Survey Data Collection.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  4. McGeeney, Kyley and Jenny Marlar. 2013. “Mobile Browser Web Surveys: Testing Response Rates, Data Quality, and Best Practices.” Presented at the Annual Conference of the American Association for Public Opinion Research, Boston.
  5. Antoun, Christopher. 2015. “Effects of Mobile versus PC Web on Survey Response Quality: a Crossover Experiment in a Probability Web Panel.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  6. Sterrett, David, Michael Stern, Gwendolyn Rugg, Ethan Raker, Jiwon Baek and Ipek Bilgen. 2015. “The Effects of Grids on Web Surveys Completed on a Mobile Device.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  7. Wang, Mengyang, Allan McCutcheon and Laura Allen. 2015. “Grids and Online Panels: A Comparison of Device Type from a Survey Quality Perspective.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  8. Richards, Ashley, Rebecca Powell, Joe Murphy, Shengchao Yu and Mai Nguyen. 2015. “Gridlocked: The Impact of Adapting Surveys Grids for Smartphones.” Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.
  9. Mavletova, Aigul and Mick P. Couper. 2014. “Mobile Web Survey Design: Scrolling Versus Paging, SMS Versus E-mail Invitations.” Journal of Survey Statistics and Methodology, 2(4), 498-518; McGeeney, Kyley and Jenny Marlar. 2013. “Mobile Browser Web Surveys: Testing Response Rates, Data Quality, and Best Practices.” Presented at the Annual Conference of the American Association for Public Opinion Research, Boston.