Start with specific research questions and choose interaction modes that best capture the insights you need.
2-4 participants work best for deep discussions, while 6-8 are ideal for diverse perspectives and brainstorming.
Always run a practice session to ensure audio, video, and chat features work smoothly before your actual survey.
Send clear instructions about the interactive elements and technical requirements beforehand.
8 min
Learn the basics of setting up video, audio, and chat interactions in your surveys.
Master role-based matching and dynamic re-pairing for more sophisticated research designs.
Discover how to extract insights from recorded interactions and chat transcripts.
Tips for managing virtual focus groups and maintaining engagement in online settings.
Yes! Given Survconf relies on multiple underlying services (Amazon Web Services, Heroku, Agora, etc.), it is a good idea to pilot your survey and make sure interactions are properly being captured immediately before launching to ensure that none of the services are down. You can pilot your survey yourself by accessing your survey link from two different browsers (for text chat) or two different web cam-enabled devices (for video/audio) to be paired with yourself to pilot the interaction. To ensure everything is working correctly with your survey setup, you should also pilot downloading your test videos from AWS to make sure they were uploaded correctly.
Given many potential participants are hesitant (or are unable) to go on video, we found the following options will help to result in a greater number of pairs/groups.
* NOTE: Before planning your studies on Prolific or MTurk, please also take a look at this helpful document, put together by Dean Baltiansky, a doctoral student at Columbia GSB!
SurvConf is not currently optimized for smartphone use. Although SurvConf does work on some (but not all) smartphones, we advise against allowing participants to use smartphones as: a) some participants will not be able to properly access their smartphone’s video camera via SurvConf within a survey, and b) the participant experience of engaging in a multi-person video call will be fundamentally different on a smartphone (e.g., where not all other matched participant videos are visible and/or are much smaller). By default, SurvConf does not restrict smartphone users from participating in case you wish to include them, but if you would like to restrict smartphone users from entering the survey, this page provides details on how to do so https://www.qualtrics.com/support/survey-platform/survey-module/mobile-survey-optimization/.
If your study collects video/audio that exceeds Amazon’s free storage tier level (5 GB at the time this post was written), you will be charged their excess storage rate (~$0.02-$0.03 per GB) for storage each month. To avoid being continually charged this amount, log into your Amazon Web Services/S3 account and delete the videos/audio that you collected once you have downloaded your own copies of the videos.
Given many potential participants are hesitant (or are unable) to go on video, we found the following options will help to result in a greater number of pairs/groups.
* NOTE: Before planning your studies on Prolific or MTurk, please also take a look at this helpful document, put together by Dean Baltiansky, a doctoral student at Columbia GSB!
Like any video-conferencing software, participants may have technology issues for a variety of reasons, such as malfunctioning devices or privacy/webcam settings that do not allow the survey access. To limit the potential of these issues, SurvConf includes a technology check for webcams and microphones at the beginning of the survey (note: this is not needed for text-only surveys) that can be used to filter out participants who may have issues before they are even paired with a partner. Additionally, in the instructions document, we provide step-by-step information on how to ensure that SurvConf is not blocked by firewalls that may exist in certain universities or enterprises (e.g., by running SurvConf via a cloud proxy). However, some participant technology issues are still unavoidable. As a result, we suggest you over-recruit in the expectation that some participants will not be able to interact with their partners due to their technology issues.
Although video interactions will provide the richest data, there are tradeoffs (with regard to added complexity) that you should consider. Using text/chat interactions in SurvConf is the easiest to set up, the easiest to recruit participants for, and minimizes the likelihood of technology issues. Alternatively, the way we recommend thinking about using video interactions in a study (especially via platforms like MTurk/Prolific), is to imagine if you were to put 4 people who don’t use video conferencing tools regularly in a video conference for the first time; the odds of at least one out of those four people having some technology issue (e.g., webcam, microphone, or connectivity issues) is non-trivial. Consequently If you opt to use video or audio interactions (especially with >2 interaction partners), you will want to make sure to over-sample to account for dyads/groups that you may need to drop due to technology issues and/or failed check/filter questions following the video/audio/text interaction to gauge whether participants (or anyone else in their dyad/group) was not present or responsive in the interaction.
Yes, all of Qualtrics normal functionality works. To accomplish this, you can use a combination of altering the conf_length variable (as described in the instructions) and/or add a normal Qualtrics timing question to the video/audio/text chat interaction page and set the relevant criteria through the timing question (e.g., auto-advance or delay showing submit button).
SurvConf works well for comparing differing modes of communication (text vs audio vs video). Our one recommendation is that to avoid confounding participants being aware of each other’s physical appearances versus the effect of the mode within a particular interaction, at the beginning of the survey, you have an introductory video session where participants get to see each other irrespective of condition. Then, later on in the survey, for the interaction of interest, you manipulate the mode of interaction used. Given SurvConf’s ability to have multiple interactions with the same partner within a study, this design is easily accomplished.
Yes, SurvConf is built using part of the code from SMARTRIQS (https://smartriqs.com/), which also enables text-based interactions in surveys. SMARTRIQS is also set up to allow for economic games (e.g., the dictator and trust game) between participants.
Additionally, ChatPlat (https://chatplat.com/#) creates the code for you to include participants’ text-chats within the survey and stores the text-chat data in a user-friendly form. Their program offers functionalities for text-chat that SurvConf currently does not (e.g., providing emoticons).
ZoomGroupStats (http://zoomgroupstats.org), an R-package created by Andrew Knight, provides a variety of analyses for video, audio, and text interactions, including sentiment and conversation analyses.
Most answers are included in the SurvConf instructions.
If you have any other questions, you can email SurvConf@gmail.com
SurvConf
Transform your surveys with interactive video, audio, and chat features. Open source, free, and no coding required…just better insights.