Home Measuring Social Preferences on Amazon Mechanical Turk
Chapter
Licensed
Unlicensed Requires Authentication

Measuring Social Preferences on Amazon Mechanical Turk

  • Marc Höglinger and Stefan Wehrli
Become an author with De Gruyter Brill
How to publish with us
Social dilemmas, institutions, and the evolution of cooperation
This chapter is in the book Social dilemmas, institutions, and the evolution of cooperation

Abstract

Social preferences are receiving increased attention in the social sciences, especially in behavioral economics and social psychology. From this arises the need to measure individuals’ social preferences in both the laboratory and in surveys of the broader population. The recently proposed SVO slider measure (Murphy et al. 2011) is supposed to be feasible for laboratory as well as for survey research. Our aim is to evaluate this measure using an online survey distributed on AmazonMechanical Turk (MTurk). We compare the elicited social preferences on MTurk to those found in laboratory settings, look at sociodemographic variation in measured social preferences and evaluate the measure’s test-retest reliability. In addition, we investigate how the standard dictator game performs as an alternative (and shorter) measure of prosocial preferences. Finally, we explore the correlation of these two incentivized measures with established survey items on self-reported prosocial behavior. Results show that social preferences elicited with the SVO-Slider on MTurk have a similar distribution to those found in laboratory settings. Also, the SVO slider turns out to have a high test-retest reliability (Pearson’s r = 0.79). However, the SVO measure correlates only weakly with self-reported prosocial behavior items but, interestingly, considerably with the survey response time.

Abstract

Social preferences are receiving increased attention in the social sciences, especially in behavioral economics and social psychology. From this arises the need to measure individuals’ social preferences in both the laboratory and in surveys of the broader population. The recently proposed SVO slider measure (Murphy et al. 2011) is supposed to be feasible for laboratory as well as for survey research. Our aim is to evaluate this measure using an online survey distributed on AmazonMechanical Turk (MTurk). We compare the elicited social preferences on MTurk to those found in laboratory settings, look at sociodemographic variation in measured social preferences and evaluate the measure’s test-retest reliability. In addition, we investigate how the standard dictator game performs as an alternative (and shorter) measure of prosocial preferences. Finally, we explore the correlation of these two incentivized measures with established survey items on self-reported prosocial behavior. Results show that social preferences elicited with the SVO-Slider on MTurk have a similar distribution to those found in laboratory settings. Also, the SVO slider turns out to have a high test-retest reliability (Pearson’s r = 0.79). However, the SVO measure correlates only weakly with self-reported prosocial behavior items but, interestingly, considerably with the survey response time.

Chapters in this book

  1. Frontmatter I
  2. Preface VII
  3. Contents IX
  4. Part I: Foundations
  5. Introduction 3
  6. Micro-Macro Models in Sociology: Antecedents of Coleman’s Diagram 11
  7. Part II: Institutions
  8. The Kula Ring of Bronislaw Malinowski: Simulating the Co-Evolution of an Economic and Ceremonial Exchange System 39
  9. From the Savannah to the Magistrate’s Court 61
  10. The Dependence of Human Cognitive and Motivational Processes on Institutional Systems 85
  11. Social Dilemmas and Solutions in Immunizations 107
  12. Part III: Social Norms
  13. When Do People Follow Norms and When Do They Pursue Their Interests? 119
  14. Personal Exposure to Unfavorable Environmental Conditions: Does it Stimulate Environmental Activism? 143
  15. Cooperation and Career Chances in Science 165
  16. Social Dilemmas in Science: Detecting Misconduct and Finding Institutional Solutions 189
  17. The Interplay of Social Status and Reciprocity 215
  18. Part IV: Peer-Sanctioning
  19. Types of Normative Conflicts and the Effectiveness of Punishment 239
  20. Social Status and Peer-Punishment: Findings from Two Road Traffic Field Experiments 259
  21. The Double Edge of Counter-Sanctions. Is Peer Sanctioning Robust to Counter-Punishment but Vulnerable to Counter-Reward? 279
  22. Diffusion of Responsibility in Norm Enforcement 303
  23. Endogenous Peer Punishment Institutions in Prisoner’s Dilemmas: The Role of Noise 327
  24. Part V: Trust and Trustworthiness
  25. Cooperation and Distrust – a Contradiction? 357
  26. Signaling Theory Evolving: Signals and Signs of Trustworthiness in Social Exchange 373
  27. Trust and Promises as Friendly Advances 393
  28. Online Reputation in eBay Auctions: Damaging and Rebuilding Trustworthiness Through Feedback Comments from Buyers and Sellers 421
  29. Part VI: Game Theory
  30. Nash Dynamics, Meritocratic Matching, and Cooperation 447
  31. A Note on the Strategic Determination of the Required Number of Volunteers 471
  32. Is No News Bad News? A Hostage Trust Game with Incomplete Information and Fairness Considerations of the Trustee 481
  33. Part VII: Experimental Methods
  34. When Prediction Fails 505
  35. Measuring Social Preferences on Amazon Mechanical Turk 527
  36. Repetition Effects in Laboratory Experiments 547
  37. Notes on the Editors and Contributors 567
https://doi.org/10.1515/9783110472974-025

Chapters in this book

  1. Frontmatter I
  2. Preface VII
  3. Contents IX
  4. Part I: Foundations
  5. Introduction 3
  6. Micro-Macro Models in Sociology: Antecedents of Coleman’s Diagram 11
  7. Part II: Institutions
  8. The Kula Ring of Bronislaw Malinowski: Simulating the Co-Evolution of an Economic and Ceremonial Exchange System 39
  9. From the Savannah to the Magistrate’s Court 61
  10. The Dependence of Human Cognitive and Motivational Processes on Institutional Systems 85
  11. Social Dilemmas and Solutions in Immunizations 107
  12. Part III: Social Norms
  13. When Do People Follow Norms and When Do They Pursue Their Interests? 119
  14. Personal Exposure to Unfavorable Environmental Conditions: Does it Stimulate Environmental Activism? 143
  15. Cooperation and Career Chances in Science 165
  16. Social Dilemmas in Science: Detecting Misconduct and Finding Institutional Solutions 189
  17. The Interplay of Social Status and Reciprocity 215
  18. Part IV: Peer-Sanctioning
  19. Types of Normative Conflicts and the Effectiveness of Punishment 239
  20. Social Status and Peer-Punishment: Findings from Two Road Traffic Field Experiments 259
  21. The Double Edge of Counter-Sanctions. Is Peer Sanctioning Robust to Counter-Punishment but Vulnerable to Counter-Reward? 279
  22. Diffusion of Responsibility in Norm Enforcement 303
  23. Endogenous Peer Punishment Institutions in Prisoner’s Dilemmas: The Role of Noise 327
  24. Part V: Trust and Trustworthiness
  25. Cooperation and Distrust – a Contradiction? 357
  26. Signaling Theory Evolving: Signals and Signs of Trustworthiness in Social Exchange 373
  27. Trust and Promises as Friendly Advances 393
  28. Online Reputation in eBay Auctions: Damaging and Rebuilding Trustworthiness Through Feedback Comments from Buyers and Sellers 421
  29. Part VI: Game Theory
  30. Nash Dynamics, Meritocratic Matching, and Cooperation 447
  31. A Note on the Strategic Determination of the Required Number of Volunteers 471
  32. Is No News Bad News? A Hostage Trust Game with Incomplete Information and Fairness Considerations of the Trustee 481
  33. Part VII: Experimental Methods
  34. When Prediction Fails 505
  35. Measuring Social Preferences on Amazon Mechanical Turk 527
  36. Repetition Effects in Laboratory Experiments 547
  37. Notes on the Editors and Contributors 567
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 3.10.2025 from https://www.degruyterbrill.com/document/doi/10.1515/9783110472974-025/html?lang=en
Scroll to top button