I have heard a number of concerns about Web-based research. One of the most reasonable — at least, the one that I find most compelling — is that participants may be more motivated when an experimenter is physically present. This makes some sense: people might take a test more seriously if it is handed to them by a living person than if they run across it on Google. The presence of someone watching you provides a certain amount of social pressure.
The implications of this question go beyond Web-based experimenting to any type of work conducted “virtually.”
There are some empirical reasons to suspect this might be a serious concern. In the famous Milgram experiments, participants’ behavior in a simulated torture scenario varied considerably depending on the appearance and actions of the experimenter.
That said, my Web-based experiments do not involve torture. Some are actually quite fun. So it is not at all clear whether the lack of a physical experimenter actually affects how the participant behaves.
Luckily, Heike Ollesch, Edgar Heineken, and Frank Schulte of the Universitat Duisberg-Essen have looking into just this question. They ran a couple simple experiments. In one, participants compared visual images. In another, they memorized lists of words. Some participants were recruited as part of a standard Web-based experiment. In control trials, the same experiment was run, but either in the lab, with an experimenter present, or outside in campus public areas, again with an experimenter present.
Participants did no better in the lab-based condition than in the Web-based condition, though in the word-memorization experiment, participants performed more poorly in the public areas (as might be expected), than in either the Web-based or lab-based conditions.
In a final, third experiment, participants described short videos. They actually produced more comprehensive descriptions in the Web-based condition than in the lab-based or public-area conditions.
——–
Stanley Milgram (1963). Behavioral Study of obedience. The Journal of Abnormal and Social Psychology, 67 (4), 371-378 DOI: 10.1037/h0040525
Heike Ollesch, Edgar Heineken, Frank P. Schulte (2006). Physical or virtual presence of the experimenter: Psychological online-experiments in different settings International Journal of Internet Science, 1 (1), 71-81
Comments are closed.