Skip to main content

​
Survey Diagnostics Funnel Overview

The Survey Diagnostics Funnel allows you to understand the estimated population of end users at particular steps in Sprig’s survey selection and contact engine. The funnel will enable study owners to self diagnose why they may not be getting as many responses as expected and troubleshoot survey design. Sprig uses an event-driven model. Each time a user sends an event to our backend, we check to see if there is a potential survey or set of surveys to present to the user. This process is documented in a surveys engagement funnel located in the Responses widget. Here’s an example of what a survey engagement funnel may look like: Inspect Response Rate Modal It’s important to note that not all of these steps will be present in each funnel. In particular, the following steps may not always be shown, as these will be dependent on the configuration of the study in question as well as other studies on the account:
  • Matched Filters
  • Included in Sampling
  • Selected for this Study

​
Survey selection works in the following steps:

  1. A single event from a single visitor is received by the Sprig backend
  2. We select the entire universe of in-progress surveys for that environment
  3. The survey selector code begins to filter this universe down by filtering out surveys that:
    1. Have the wrong SDK version
      Step: “Had required SDK”
    2. Do not match resurvey criteria
      Step: “Met resurvey criteria”
    3. Have a different event trigger than the one the user triggered
      Step: “Triggered event”
    4. That the user has already seen (based on resurvey criteria)
      Step: “Hadn’t seen this study yet”
    5. That the user doesn’t match filters and attributes for
      Step: “Matched Filters”
    6. Need to be sampled or throttled
      Step: “Included in sampling”
      Step: “Selected for this study”
  4. The result may be:
    1. A single survey that is then served to the user
    2. A randomized array of surveys, from which the first will be served to the user
The last four steps in the engagement funnel are:
  1. Received Study
  2. Viewed Study
  3. Started Responding
  4. Completed Study
These steps happen after the universe of surveys has been filtered down to the one particular survey this user should see, and that survey has been sent to their SDK instance.

​
Using the Diagnostics Funnel

  1. For any active or completed Web or Mobile Survey, navigate to the Survey Results page.
  2. Select Inspect to launch the Inspect Response Rate modal.
  1. The debugger will show you a list of checkpoints and the estimated population of end users at each checkpoint.

​
Note on Limitations:

  • Survey Diagnostics Funnel will be unavailable for surveys completed before October 2023.
  • Audience population estimations are limited to the last 30 days of an active survey.
  • If any edit is made to an active study, including copy updates or targeting criteria, the diagnostics funnel will reset and begin collecting data from time last updated.
Launch from the Response Rate tile

​
Checkpoint Steps

Checkpoint steps and corrective measures are detailed below:
CheckpointStep DetailsPossible Corrective Measure
Has required SDK*Users that are sent to Sprig and have the correct SDK installedUpdate SDK version
Met resurvey criteriaUsers that are qualified based on global or survey level recontact period settingsOverride global contact period, or adjust survey level recontact period
Triggered eventUsers that triggered the survey contact eventAdd additional events to trigger this survey
Hadn’t seen this study yetUsers that have not seen this survey before
(Shown only when “Show once per user” is on)		Adjust “Show once per user” setting
Matched filtersUsers that meet Audience, Event, or Group targeting filter criteria
(Shown only when at least one filter set)		Adjust filter settings to be more inclusive, or verify that filters are configured correctly
Included in send limitStudy send limit (#of times per day or sampling) settings are captured under Audience > Response settings (Shown only when this count is different than the previous stage)Adjust sampling or send limit criteria to be more expansive
Selected for this studyStep is relevant only when multiple studies have the same targeting criteria
(Shown only when # selected for this study differs from previous step)		Modify this study, or a conflicting study to have different triggering events
Received studyUsers that have received the studyN/A
Viewed studyUsers that have viewed the studyModify the “waiting period” delay to be shorter
Started respondingUsers that have started respondingN/A
Completed studyUsers that have completed the entire surveyUpdate study design (shorter, question types, etc) to increase completion rates
*For mobile studies, if SDK update is bundled with an app update, end users will need to update your native app in order to qualify for a study with a minimum SDK requirement.

​
FAQs

​
How is overall response rate calculated?

The response rate graph is calculated by dividing the number of responded surveys by the number of seen surveys within a day based in Universal Time Coordinated (UTC).

​
How do event properties get applied?

When targeting select users based on event properties, these event properties are applied during step 5, “Matched filters”. The “Triggered Event” step will evaluate the event, and then “Matched filters” will evaluate the event property filters applied.