We will start with the premise that data collection methods must be consistent with the research objectives. No method is the best approach for all research designs. The criterion for deciding on a method must relate directly to the research requirements.
Consideration of secondary factors should only influence the decision when the primary objectives are met. Occasionally, lower priority criteria, such as Time in the field, or a concern about professional survey recorders to decide which data collection method to use. These types of issues need to be addressed but should not dominate the decision on method of data collection for a study.
Applying an eclectic approach to data collection ensures the strengths and weaknesses of each approach are evaluated in the context of the research objectives. For example, unsupported awareness and preference surveys are difficult to complete using a web-based approach, especially if the study investigates multiple products or product types and needs exploration questions.
Conversely, while not impossible, it would be extremely undesirable to attempt a coherent study using telephone interviews. Conjoint studies are much better suited for a web-based approach.
The Internet has far-reaching capabilities that allow researchers to expand the coverage of their research. More geographical coverage (larger number of countries), typically at a lower cost, is an attractive combination. Lower costs and / or more coverage can help the research team add national coverage in new markets or where it may be difficult and expensive to conduct telephone interviews due to the lack of native speaking interviewers (although this may seem the opposite in some countries).
The speed of data collection is often the reason why a web-based approach is chosen. However, caution is that best practices may still require several weeks (3-4) to allow time to execute waves of emails to perform a random stratified test.
For some studies, the web is the most desirable approach. It is the single best way to conduct a co-study. It is also the preferred method of information gathering when long lists of topics or complex statements are part of the research instrument’s design (e.g., multiple-answer questions with either an “all that applies” or k of N-question structure).
K for N question structure is a ratio where there are N items listed and k is the maximum number of items the respondent is allowed to select (e.g., “Choose up to four”). K of the N question type requires respondents to listen to more information than they are likely to remember. Human heuristics studies show that with lists of more than 5-7 items, only the first and last heard items are remembered, which can interfere with the results (Note: rotation list helps but cannot solve this problem).
Another benefit of a web-based survey is that it allows respondents to participate when they have time and, if necessary, start-stop-and-restart. It also allows for interesting incentive programs that can combine downloadable content with other types of incentives. In fact, it may be highly desirable to use downloadable content as an incentive rather than a cash-based approach to avoid attracting professional survey recorders. Immediately available value-added content will attract business executives, IT professionals and other professional groups, but will be of little value to the survey takers looking for cash payouts with little or no interest in the subject.
Telephone interview method
Among the key benefits of telephone interviewing is the researcher’s ability to control the sample. Interviewers are able to find the person in an organization that is most qualified to respond. They can screen respondents and use probes to find the “right” respondent. Quotas for demographic layers and unique qualifications can be controlled directly using a computer-assisted telephone interview system (CATI) used by most field houses. Broad-based business listings can be used instead of selecting email lists, which helps with randomization.
Using unanswered answer questions is easier to implement with telephone interviews than it is with a web-based approach that relies on a respondent to take the time to write answers. Interviewers can also survey and encourage respondents to consider multiple responses using phrases such as, “Are there any other companies you can think of?”
Phone interviewing, which is a more active recruitment method than web-based research, helps avoid professional investigators, regardless of the incentives used. The quality of the list is clearly as important in telephone interviews as in web-based data collection. However, a bad phone book can easily be registered and replaced if it does not work well.
Therefore, the ability to create highly stratified samples with multiple selection criteria and multiple sampling ratios is a significant strength of the telephone approach. Data collection can be improved for some question types, and as mentioned, the type of question is an important criterion for choosing a data collection method. Not all questions are best suited for the phone, but when the study requires exploratory and open-ended questions, the phone may be the method of choice.
The question is not whether a study using a particular approach to data collection is superior or inferior to another approach, but rather whether the research objectives benefit better from one approach over another. Some research designs are clearly in line with specific data collection methods, while in other cases more general strengths and weaknesses of the two methods can be used to determine the approach to be used. In some cases, the differences between data collection approaches are less dramatic and issues such as field resource availability, cost factors, and time in the field are the deciding factors.