1. Home
  2. Best Practices
  3. 7 Steps to Successful Mobile Surveys

7 Steps to Successful Mobile Surveys

Every customer has unique preferences, tastes, likes, & dislikes

Intuitively, customer individuality makes perfect sense. We’re all different. Yet at the end of the day, many brands know little about their customers on an individual, human level. Even some of the most customer-centric brands are guilty of overemphasizing aggregate metrics, like star ratings, net promoter scores, and trends in retention and per-customer revenue. They trade unique characteristics, habits, and preferences of the customer as an individual for a big-picture view of the “customer” as an abstract concept. This myopic viewpoint is not only flawed, but costly. When a brand combines individual customer data points to calculate such averages, it sacrifices the individuality of each customer. It sacrifices the knowledge of what drives brand affinity for the simplicity of data collection and management. Knowing these aggregate metrics is crucial to a brand’s success, but provides only a piece of the puzzle. Consider the following dilemma to see where these aggregates fall short: Your company knows its app’s download count increased five-fold over the past year, and its average revenue per user grew by 20%. By all conventional measurements, things are looking pretty great. Surely, then, your metrics will only continue to improve with your anticipated update. But what if things don’t work out as planned? What if your next update leaves customers unsatisfied? Worse yet, what if a new competitor enters the market with lower prices? How confident are you that your customers will remain loyal to your app? Will they ride out the storm or jump ship? These are questions you can only answer by looking at the individuality of each customer:

  • What do they care about in your product?
  • How can you earn and maintain their business?
  • How can you prevent them from ditching your brand for a compelling alternative?

In short, what makes a customer a customer?

When you answer these questions, you gain sustainable competitive advantage. Customer experience is expected to surpass both product and price as the dominant source of competitive advantage by 2020, and it all comes down to identifying and being able to deliver the experience your customers desire.

The customer insights needed for personalized marketing are elusive, but nonetheless there—in the behavior of your customers, in their interaction with brand communication, and in their words (or more often, in their silence). We often hear of the need to ‘give customers a voice;’ but in reality, your customers already have a voice. It might lie dormant, buried or hidden amid the clutter of data, but it can always be unearthed. You just have to have the tools in place to actively listen for it.

Over the remainder of this guide, we share seven brand-tested steps for using in-app surveys to excavate the “Voice of the Customer” and lay the groundwork for winning competitive advantage through a comprehensive understanding of the customer. Enjoy!

1: Pick Your Survey Tool

We have an obvious bias here, but the very first thing you need to consider is your choice of survey tool. This includes both the decision of whether to use an in-app survey (like Apptentive) or a web survey (like SurveyMonkey) and the decision of whether to make your survey platform inhouse or seek out a third-party solution. We’ve done our best to highlight the pros and cons of each of the following page.

Our Recommendation

Use an in-app survey rather than directing mobile customers to a web survey so as to not detract from the customer experience. If you are using a web survey, be upfront about asking customers to leave the app for an external link.

In-App Surveys vs. Mobile Web Surveys

In-App Surveys
steps-01

Pros:

  • Allows for a frictionless customer experience as customers never have to leave the app
  • Dramatically higher response rates when compared to web surveys
  • Easy to integrate into points of engagement within the app (‘mobile moments’)

Cons:

  • May require additional engineering time to integrate into your app

(Mobile) Web Surveys

steps-02

Pros:

  • Wider variety of established survey and analytic tools available to choose from

Cons:

  • Lower response rates
  • Requires your customers to leave the app to take the survey
  • Branding and the customer experience are often less cohesive between the app and the externally hosted survey

In-House Solution vs. Third-Party Solutions

In-House, Custom-Built Surveys

Pros:

  • Total flexibility

Cons:

  • Very resource-intensive in terms of developer time and money
  • Necessary testing of, and updates to, the survey tool may take time away from your product roadmap

Third-Party (SDK) Solutions

Pros:

  • Turnkey solutions available with reporting tools and dashboards
  • Access to existing documentation on best practices for customer success

Cons:

  • Requires some developer time, depending on ease of SDK integrationhosted survey

For more on choosing between an in-app survey and a mobile web survey, see the Being Apptentive blog: The Mobile Surveys vs. Web Surveys Debate

2: Determine Your Research Question

Before jumping into designing your survey within your chosen tool, take some time to identify a single concrete research question that you hope to answer with the results of your survey. This will provide a benchmark for your data analysis and can help to keep the survey short and concise if you constrain yourself to asking only those questions necessary for addressing your research question.

Research questions are generally classified as either attitudinal, behavioral, demographic, or technical. An example of a research question for each of these categories can be found below:

  • Attitudinal: How do new customers like my app?
  • Behavioral: How do customers interact with my app? What are their most common use cases?
  • Demographic: With which age bracket is my app most popular?
  • Technical: How can my app be improved, in the eyes of my customers? Once you have carefully selected your research question, several of the remaining steps will ensue naturally–including your target audience and the type of data you need to collect.

3: Identify Your Survey Audience

It’s now time to determine which customers are best suited to honestly and accurately address your research question. Once you’ve identified the customer segment, you can begin to seek out and target pain points within the app where the survey will provide the most value to both the customer and your team.

Common customer segments for mobile research studies include:

Our Recommendation

Use event-based targeting to best ensure a nonintrusive survey experience. Don’t immediately ask new customers to take a survey, and only ask customers to take your survey once rather than asking each time they load an event.

A Sample Generated With Event-based Targeting

Delivering your in-app survey to customers who engage in pre-specified ‘mobile moments,’ for example:

  • The third time a customer uses the Search function
  • The first time a customer shares content via the app
  • After a customer updates to a new version
  • After a customer uses a new/beta feature

Advantages:

  • Can be used to address narrow, feature-specific research questions
  • Can be used to refine and beta test new versions and rollouts
  • Respondents inherently have familiarity with the app event they’re providing feedback on

Disadvantages:

  • Resource-intensive, unless using thirdparty mobile engagement solutions since activity is a necessary condition of the sampling selection

A Random Sample

Randomly choose a percentage of your mobile customers to survey.

Advantages:

  • Diversity of responses makes the data highly representative of your overall audience
  • Surveys can be prompted even if mobile analytic capabilities do not collect information on customers and customer activity

Disadvantages:

  • Sample may be too broad to address narrow research questions that require a high level of familiarity with the app
  • Responses may be too generalized to uncover trends based on customers’ familiarity with the app, device used to access the app, etc.

A Sample of New Customers

Customers prompted during their first time using the app.

Advantages:

  • Allows you to collect information, unbiased by existing loyalty, that can be used to improve the customer experience and retention

Disadvantages:

  • Requesting customer information from first-time customers may create a negative initial experience with the app, and has been shown to be a leading factor in app exits

A Sample of Loyal Customers

Prompted the n-th time a customer opens the app or a few months after installing and regularly using the app.

Advantages:

  • Can be used to uncover which features loyal customers find the most valuable / what is bringing them back to your app
  • Can be used to gather suggestions from those already familiar with those apps–and customers can be further delighted if those suggestions are acted on

Disadvantages:

  • Responses may have an upward bias and have difficulty capturing equally important negative experiences with the app

An Opt-in Sample

Instead of identifying sampling criteria, let your customers proactively navigate your menu to find ways to voice their feedback.

Advantages:

  • Survey can be easily integrated as a link in your app’s navigation with no third-party tools or prompts
  • Least intrusive–avoids potentially annoying notifications and prompts

Disadvantages:

  • Can lead to selection bias as those who opt-in may not share fundamentally different views/attitudes than those who do not opt-in
  • Opt-ins have the lowest response rate and the survey link can be difficult for potential respondents to find, making it hard to create open dialog with customers

4: Design Around the Research Question

When it comes to writing the actual questions for your in-app survey, start by asking yourself what sort of data best addresses your research question. Some research questions (particularly attitudinal) lend themselves to quantitative questions and rating scales. Others (particularly technical) are best addressed with qualitative, open-ended questions.

Survey questions can be classified as open-ended, closed-ended, or mixed. We’ve addressed leading use cases of each classification and their ability to address your anticipated needs:

Open-ended (i.e. Providing a textbox for the respondent to type their answer)

  • Exploratory in nature
  • Less likely to result in bias from leading questions/response choices
  • Provides qualitative responses similar to a focus group
  • Can be time-consuming to answer, particularly when using a mobile device, leading to lower response rates

Closed-ended (i.e. Multiple choice questions and rating scales with pre-defined response choices)

  • Can provide both qualitative and quantitative responses
  • Questions typically take less time to answer and experience higher response rates
  • Questions and response construction requires more care remove bias

Mixed (i.e. A multiple choice question with a fill-in ‘Other’ option)

  • Allows room to write in answers that were not considered when the survey was designed—may uncover new customer needs/sentiments

For more on quantitative vs. qualitative questions, and their use cases, Check out this guide by our friends at SurveyMonkey.

5: Integrate Into the Mobile Experience

steps-03Now that you’ve determined your audience, the next step is to identify where and how to reach them within your app. If you’re using event-based targeting, seek out mobile moments in the customer journey. These are typically points of engagement where customers may feel happy, frustrated, or lost. Concentrating on these moments allows you to hone into a single part of the customer experience and can lead to more actionable results if the feedback directly relates to the point of engagement.

When prompting your in-app survey, be careful to not interrupt or annoy customers in the middle of a task. For example, if you want to ask about the in-app shopping experience, prompt your survey after checkout instead of after a customer adds an item to his or her cart. Remember, your survey should be designed to add value to both you and your customer.

Our Recommendation

Use event-based targeting to best ensure a nonintrusive survey experience. Don’t immediately ask new customers to take a survey, and only ask customers to take your survey once rather than asking each time they load an event.

If you’re targeting new customers, give them time to try out the app before prompting your survey. A ratings or feedback request upon first log-in is a sure way to create a negative first impression. And finally, if you’re using an opt-in sample without any sort of prompt, insert a link to your survey into a logical place in your app’s navigation menu–such as the Help or Contact section.

6: Pre-Test for Experience and Functionality

Before making your survey live, it’s important to pilot the survey internally or with a small sample of customers. This allows you to ensure that your survey is working as intended and identify any weaknesses or ambiguity.

While testing your survey, check that the following items are in order:

  • Questions proceed in the intended order, and responses are mapped to the appropriate next question or prompt if conditional branching and skip logic are in place
  • Questions are easily understood by those unfamiliar with the survey
  • Response options are clear and do not contain unintentional overlap
  • Optional and required fields are appropriately coded
  • Responses are accurately collected upon completion

After you’ve tested your survey and are satisfied with the results, we’d recommend resetting the response count so that the test data does not interfere with your actual results. You’re ready to push the survey out to your intended audience.

7: Analyze the Results

By this point, you’ve launched your survey and have collected a statistically significant number of responses. Now it’s time to dig into the fruits of your labor.

Data analysis should be a straightforward process for those quantitative questions directly mapped to your research question. We recommend going through each question individually and asking yourself whether each result confirms or rejects your hypothesized answer to your own research question: Are the result of customer rating scales consistent with what I’m seeing in app store ratings? Has my latest version update improved customer sentiment? Do customers feel comfortable navigating my interface and new features?

In the case of qualitative questions, you may need to do a little more work to get meaningful and generalizable data. For open response questions, we recommend categorizing the responses into themes–such as feature requests, usage or interface questions, customer testimonials, and customer complaints. You can then search through your results for the most common trends and begin to incorporate these insights into your product roadmap and QA efforts.

As a last step, decide whether you want to keep the survey running or close it. This is often a matter of saturation (if you already have the sample size you need and additional responses provide little value) and relevancy (is your research question still a priority, or is there a new question worth asking in a separate survey?).

Best Practices of Survey Design: The Do’s and Don’ts of Mobile Research

Before diving in creating your next mobile research instrument, we want to leave you with a few best practices that we’ve discovered with the help of our customers. These are proven tips for creating effective mobile surveys optimized for response rates, customer experience, and the collection of actionable insights.

When designing your mobile research instrument, do:

  • Design with mobile in mind.
  • Keep questions brief and concise.
  • Allow customers to opt out at any time.
  • Aim to address your research objective with as few questions as possible.
  • Limit the number of options for multiple choices.
  • Break the questions up so that only one or two appear at a time.
  • Provide an ‘Other’ field with a textbox for fill-in answers to your multiple-choice questions if you suspect that some respondents may have answers you had not previously considered.
  • Add an option for ‘Don’t Know’ or ‘Not Applicable’ for questions that some respondents may not be able to answer.
  • Pre-test your survey internally to identify any weaknesses and ambiguity.
  • Prompt the survey at an intelligent time during your customers’ in-app experience

And don’t:

  • Create overlap in multiple-choice responses. All responses should be mutually exclusive to avoid confusion.
  • Present rating scales with large matrices of options or questions ones that require scrolling on a mobile screen. All questions should be written with mobile viewability in mind.
  • Create vague responses that are open to the interpretation of the respondent (e.g. If asking about use frequency, give tangible options like ‘twice a week’ and ‘once a month’ rather than ‘often’ or ‘rarely’.
  • Frame questions in a way that leads the respondent or creates bias (e.g. “Why do you like this app?”).
  • Request personal information at the start of the survey as this may lead to lower response rates. If you need this information, make the questions optional and move them to the end of the survey.

Our Dedication to Customer Success

At the end of the day, those companies that stand out from the ever-growing sea of apps are those that listen to, engage with, and retain their customers—wherever they may be. In other words, those apps that communicate with their customers on an individual, human level.

At Apptentive, we’re pioneering a suite of tools to make creating and managing a channel for open communication easier, more affordable, and more scalable than ever before.

And we’re here to help every step of the way. As an enterprise customer, you’ll have access to a library of resources like this guide and unlimited access to a dedicated Customer Success representative trained in executing flawless in-app communication campaigns. Wherever you may be in your mobile research efforts, our Customer Success Team is here to bridge the gaps and elevate your results. We’ll work with you individually to:

  • Identify your most pressing research questions and the best strategy for addressing them;
  • Segment your mobile customers into meaningful subgroups and cater your messaging accordingly;
  • Discover customer pain points to enhance the customer journey and create an experience customers love;
  • Target the right time, place, frequency, and medium for customer engagement;
  • Intertwine customer insights and your product roadmap to develop a pre-validated app and minimize wasted resources; and,
  • Build real and reciprocal customer relationships by engaging customers in a manner consistent with their communication preferences.

To start communicating with your mobile customers and turn customer feedback into competitive advantage, schedule a demo today. We’ll take care of the heavy lifting.

Join these companies in Being Apptentive.

client-logos

 

Updated on September 22, 2016

Was this article helpful?

Related Articles