Startseite Wirtschaftswissenschaften Assessing the Impact of Political Involvement on the Reliability and Validity of Virtualized Real-time-response Measurement
Artikel
Lizenziert
Nicht lizenziert Erfordert eine Authentifizierung

Assessing the Impact of Political Involvement on the Reliability and Validity of Virtualized Real-time-response Measurement

  • Thomas Waldvogel EMAIL logo , Thomas Metz , Uwe Wagschal , Bernd Becker , Linus Feiten und Samuel Weishaupt
Veröffentlicht/Copyright: 18. Mai 2021
Veröffentlichen auch Sie bei De Gruyter Brill

Abstract

How does political involvement impact on data quality of virtualized Real-Time-Response (RTR) Measurement? The article addresses this issue, drawing on a large field study (n = 5660) conducted during the 2017 German general election campaign. Since it is unclear how heterogeneous characteristics of individuals influence data quality of RTR-Measurement from audiences surveyed outside the lab, we assess the impact of political involvement on the reliability and validity of virtualized real-time-response data. We show that political involvement shapes reliability and validity systematically but to a degree that does not compromise established standards of data quality. Thus, we conclude that virtualized RTR outside the laboratory with limited means of control is an appropriate method to survey heterogeneous samples in large N-field studies and therefore offers new paths of data collection.


Corresponding author: Thomas Waldvogel, Political Science, Albert-Ludwigs-Universitat Freiburg, Freiburg im Breisgau, Germany, E-mail:

Appendix

Appendix A1: Measurement Instructions to the participants

Please note the following:

  • – The Debat-O-Meter records your ratings exact to the second, which means one rating is recorded per second.

  • – Why you rate a person and what exactly you consider good or bad is left entirely to you.

  • – If no button gets pressed, no data gets transmitted, which will be interpreted as a “neutral” rating.

  • – Please do not use the device when the host is speaking.

Appendix A2: Description of the sample

Age Gender Education Political interest
Category Frequency (%) Category Frequency (%) Category Frequency (%) Category Frequency (%)
<20 403 (9.9) Female 1715 (42.5) No degree 19 (0.5) No interest 14 (0.3)
20–29 1035 (25.3) Male 2319 (57.5) Still in school 95 (2.3) Weak 199 (4.9)
30–39 924 (22.6) Lower secondary school 288 (7.1) Medium 1285 (31.6)
40–49 653 (15.9) Mid-level secondary school 1040 (25.5) Strong 1670 (41.1)
50–59 611 (15.0) Higher-level secondary school 1215 (30.2) Very strong 893 (23.8)
60–69 327 (8.0) Tertiary education 1422 (34.9)
>70 139 (3.2)

References

Bachl, M. 2013. “Die Wirkung des TV-Duells auf die Bewertung der Kandidaten und die Wahlabsicht.” In Das TV-Duell in Baden-Württemberg 2011. Inhalte, Wahrnehmungen und Wirkungen, edited by M. Bachl, F. Brettschneider, and S. Ottler, 171–98. Wiesbaden: Springer VS.10.1007/978-3-658-00792-8_8Suche in Google Scholar

Bachl, M. 2014. Analyse rezeptionsbegleitend gemessener Kandidatenbewertungen in TV-Duellen Erweiterung etablierter Verfahren und Vorschlag einer Mehrebenenmodellierung. Berlin: Universität Hohenheim.Suche in Google Scholar

Benoit, W. L., G. J. Hansen, and R. M. Verser. 2003. “A Meta-Analysis of the Effects Viewing U.S. Presidential Debates.” Communication Monographs 70 (4): 335–50, https://doi.org/10.1080/0363775032000179133.Suche in Google Scholar

Biocca, F., P. David, and M. West. 1994. “Continuous Response Measurement (CRM): A Computerized Tool for Research on the Cognitive Processing of Communication Messages.” In Measuring Psychological Responses to Media Messages, edited by A. Lang, 15–64. Hillsdale: Erlbaum.Suche in Google Scholar

Bortz, J., and N. Döring. 2006. Forschungsmethoden und Evaluation für Human-und Sozialwissenschaftler., 4., überarbeitete Auflage. Heidelberg: Springer, https://doi.org/10.1007/978-3-540-33306-7.Suche in Google Scholar

Boyd, T. C., and G. D. Hughes. 1992. “Validating Realtime Response Measures.” NA-Advances in Consumer Research 19.Suche in Google Scholar

Boydstun, A. E., R. A. Glazier, M. T. Pietryka, and P. Resnik. 2014. “Real-Time Reactions to a 2012 Presidential Debate A Method for Understanding Which Messages Matter.” Public Opinion Quarterly 78 (Special issue): 330–43, https://doi.org/10.1093/poq/nfu007.Suche in Google Scholar

Campbell, A., P. Converse, W. Miller, and D. Stokes. 1960. The American Voter. New York, NY: Wiley.Suche in Google Scholar

Druckman, J. N. 2003. “The Power of Television Images: The First Kennedy-Nixon Debate Revisited.” The Journal of Politics 65 (2): 559–71, https://doi.org/10.1111/1468-2508.t01-1-00015.Suche in Google Scholar

Eulau, H., and P. Schneider. 1956. “Dimensions of Political Involvement.” Public Opinion Quarterly 20 (1): 128–42, https://doi.org/10.1086/266603.Suche in Google Scholar

Faas, T., and J. Maier. 2011. “Medienwahlkampf. Sind TV-Duelle nur Show und damit nutzlos?” In Der unbekannte Wähler? Mythen und Fakten über das Wahlverhalten der Deutschen, edited by E. Bytzek, and S. Roßteutscher, 99–114. Frankfurt/New York: Campus.Suche in Google Scholar

Faas, T., J. Maier, and M. Maier. 2017. Merkel gegen Steinbrück: Analysen zum TV-Duell vor der Bundestagswahl 2013. Wiesbaden: Springer-Verlag.10.1007/978-3-658-05432-8Suche in Google Scholar

Fenwick, I., and M. D. Rice. 1991. “Reliability of Continuous Measurement Copy-Testing Methods.” Journal of Advertising Research 31 (1): 23–9, http://psycnet.apa.org/psycinfo/1991-26085-001.Suche in Google Scholar

Green, D., B. Palmquist, and E. Schickler. 2002. Partisan Hearts and Minds. Political Parties and the Social Identity of Voters. New Haven, CT: Yale University Press.Suche in Google Scholar

Hallonquist, T., and J. G. Peatman. 1947. “Diagnosing your Radio Program, or the Program Analyzer at Work.” In Education on the Air. Yearbook of the Institute for Education by Radio, edited by Institute for Education by Radio, 463–74. OH: Ohio State University Press.Suche in Google Scholar

Hallonquist, T., and E. E. Suchmann. 1944. “Listening to the Listener. Experiences with the Lazarsfeld-Stanton Program Analyzer.” In Radio Research 1942-1943, edited by P. F. Lazarsfeld, and F. Stanton, 265–334. New York: Arno Press.Hughes.Suche in Google Scholar

Holbrook, T. M. 2002. “Presidential Campaigns and the Knowledge Gap.” Political Communication 19 (4): 437–54, https://doi.org/10.1080/10584600290109997.Suche in Google Scholar

Hughes, G. D., and R. Lennox. 1990. “Realtime Response Research: Construct Validation and Reliability Assessment.” In Enhancing Knowledge Development in Marketing, edited by W. Bearden, et al.., 284–8. Chicago, IL: American Marketing Association.Suche in Google Scholar

Lazarsfeld, P. F., B. Berelson, and H. Gaudet. 1944. The People’s Choice. New York, NY: Columbia University Press.Suche in Google Scholar

Maier, J. 2007a. “Eine Basis für rationale Wahlentscheidungen? Die Wirkungen des TV-Duells auf politische Kenntnisse.” In Schröder gegen Merkel. Wahrnehmung und Wirkung des TV-Duells 2005 im Ost-West-Vergleich, edited by M. Maurer, C. Reinemann, J. Maier, and M. Maier, 129–43. Wiesbaden: VS Verlag.10.1007/978-3-531-90709-3_7Suche in Google Scholar

Maier, J. 2007b. “Erfolgreiche Überzeugungsarbeit. Urteile über den Debattensieger und die Veränderung der Kanzlerpräferenz.” In Schröder gegen Merkel. Wahrnehmung und Wirkung des TV-Duells 2005 im Ost-West-Vergleich, edited by M. Maurer, C. Reinemann, J. Maier, and M. Maier, 91–109. Wiesbaden: VS Verlag.10.1007/978-3-531-90709-3_5Suche in Google Scholar

Maier, J., and T. Faas. 2011. “‘Miniature Campaigns’ in Comparison: The German Televised Debates, 2002–09.” German Politics 20 (1): 75–91.10.1080/09644008.2011.554102Suche in Google Scholar

Maier, J., T. Faas, and M. Maier. 2014. “Aufgeholt, aber nicht aufgeschlossen: Ausgewählte Befunde zur Wahrnehmung und Wirkung des TV-Duells 2013 zwischen Angela Merkel und Peer Steinbrück.” Zeitschrift für Parlamentsfragen 45 (1): 38–54.10.5771/0340-1758-2014-1-38Suche in Google Scholar

Maier, J., T. Faas, B. Rittberger, J. Fortin-Rittberger, K. A. Josifides, S. Banducci, P. Bellucci, M. Blomgren, I. Brikse, and K. Chwedczuk-Szulc. 2018. “This Time It’s Different? Effects of the Eurovision Debate on Young Citizens and its Consequence for EU Democracy–Evidence from a Quasi-Experiment in 24 Countries.” Journal of European Public Policy 25 (4): 606–29.10.1080/13501763.2016.1268643Suche in Google Scholar

Maier, J., J. F. Hampe, and N. Jahn. 2016a. “Breaking Out of the Lab Measuring Real-Time Responses to Televised Political Content in Real-World Settings.” Public Opinion Quarterly 80 (2): 542–53.10.1093/poq/nfw010Suche in Google Scholar

Maier, J., M. Maier, M. Maurer, C. Reinemann, and V. Meyer, eds. 2009. Real-Time Response Measurement in the Social Sciences. Methodological Perspectives and Applications. Peter Lang.Suche in Google Scholar

Maier, J., M. Maurer, C. Reinemann, and T. Faas. 2007. “Reliability and Validity of Real-Time Response Measurement: A Comparison of Two Studies of a Televised Debate in Germany.” International Journal of Public Opinion Research 19 (1): 53–73.10.1093/ijpor/edl002Suche in Google Scholar

Maier, J., B. Rittberger, and T. Faas. 2016b. “Debating Europe: Effects of the “Eurovision Debate” on EU Attitudes of Young German Voters and the Moderating Role Played by Political Involvement.” Politics and Governance 4 (1): 55–68.10.17645/pag.v4i1.456Suche in Google Scholar

Maier, M., and J. Strömbäck. 2009. “Advantages and Limitations of Comparative Audience Responses to Televised Debates: A Comparative Study of Germany and Sweden.” In Real-Time Response Measurement in the Social Sciences. Methodological Perspectives and Applications, edited by J. Maier, M. Maier, M. Maurer, C. Reinemann, and V. Meyer, 97–116. Frankfurt: Peter Lang.Suche in Google Scholar

Maurer. 2009. Sagen Bilder mehr als tausend Worte? https://doi.org/10.5771/1615-634x-2009-2-198.Suche in Google Scholar

Maurer, M., and C. Reinemann. 2003. Schröder gegen Stoiber: Nutzung, Wahrnehmung und Wirkung der TV-Duelle. Wiesbaden: Westdeutscher Verlag.10.1007/978-3-322-80456-3Suche in Google Scholar

Maurer, M., and C. Reinemann. 2015. “Do Uninvolved Voters Rely on Visual MessageElements? A Test of a Central Assumption of the ELM in the Context of Televised Debates.” Politische Psychologie 4 (2): 235–51.Suche in Google Scholar

Maurer, M., C. Reinemann, J. Maier, and M. Maier, eds. 2007. Schröder gegen Merkel. Wahrnehmung und Wirkung des TV-Duells 2005 im Ost-West-Vergleich. Wiesbaden: VS Verlag.Suche in Google Scholar

McKinney, M. S., and D. B. Carlin. 2004. “Political Campaign Debates.” In Handbook of Political Communication Research, edited by L. L. Kaid, 203–34. Mahwah: Lawrence Erlbaum Associates.Suche in Google Scholar

Metz, T., U. Wagschal, T. Waldvogel, M. Bachl, L. Feiten, and B. Becker. 2016. “Das Debat-O-Meter: ein neues Instrument zur Analyse von TV-Duellen. ZSE Zeitschrift für Staats- und Europawissenschaften.” Journal for Comparative Government and European Policy 14 (1): 124–49, doi:https://doi.org/10.5771/1610-7780-2016-1-124.10.5771/1610-7780-2016-1-124Suche in Google Scholar

Mullinix, K. J. 2015. “Presidential Debates, Partisan Motivations, and Political Interest.” Presidential Studies Quarterly 45 (2): 270–88.10.1111/psq.12187Suche in Google Scholar

Nagel, F. 2012. Die Wirkung verbaler und nonverbaler Kommunikation in TV-Duellen. Wiesbaden: VS Verlag.10.1007/978-3-531-93497-6Suche in Google Scholar

Otto, L., M. Maier, and I. Glogger. 2015. “Image- or Issue-Orientation? How the Presentation Modality Influences the Perception of Candidates in Televised Debates.” Politische Psychologie 4 (2): 215–34.Suche in Google Scholar

Papastefanou, G. 2013. Reliability and Validity of RTR Measurement Device. (Working Paper No. 27). Mannheim: GESIS - Leibniz-Institut für Sozialwissenschaften.Suche in Google Scholar

Petty, R. E., and J. T. Cacioppo. 1986. Communication and Persuasion: Central and Peripheral Routes to Attitude Change. New York, NY: Springer.10.1007/978-1-4612-4964-1Suche in Google Scholar

Range, J. 2017. “Wissens-und Partizipations-Gaps: Führte das TV-Duell 2013 zu einer politischen und kognitiven Mobilisierung?” In Merkel gegen Steinbrück, edited by T. Faas, J. Maier, and M. Maier, 75–86. Wiesbaden: Springer VS.10.1007/978-3-658-05432-8_5Suche in Google Scholar

Reinemann, C., J. Maier, T. Faas, and M. Maurer. 2005. “Reliabilität und Validität von RTR-Messungen.” Publizistik 50 (1): 56–73.10.1007/s11616-005-0118-4Suche in Google Scholar

Reinemann, C., and M. Maurer. 2005. “Unifying or Polarizing? Short-Term Effects and Postdebate Consequences of Different Rhetorical Strategies in Televised Debates.” Journal of Communication 55 (4): 775–94.10.1111/j.1460-2466.2005.tb03022.xSuche in Google Scholar

Reinemann, C., and M. Maurer. 2010. “Leichtgläubig und manipulierbar? Die Rezeption persuasiver Wahlkampfbotschaften durch politisch Interessierte und Desinteressierte.” In Information – Wahrnehmung – Emotion, edited by T. Faas, K. Arzheimer, and S. Roßteutscher, 239–57. Wiesbaden: VS Verlag für Sozialwissenschaften.10.1007/978-3-531-92336-9_12Suche in Google Scholar

Schill, D., R. Kirk, and A. E. Jasperson. 2016. Political Communication in Real Time: Theoretical and Applied Research Approaches. New York, NY/London: Routledge/ Taylor & Francis Group.10.4324/9781315669083Suche in Google Scholar

Schwerin, H. 1940. “An Exploratory Study of the Reliability of the “Program Analyzer”.” Journal of Applied Psychology 24 (6): 742–5.10.1037/h0058363Suche in Google Scholar

Waldvogel, T., and T. Metz. 2020. “Measuring Real-Time Responses in Real-Life Settings.” International Journal of Public Opinion Research, https://doi.org/10.1093/ijpor/edz050.Suche in Google Scholar

Wagschal, U. 1999. Statistik für Politikwissenschaftler. München/Wien: Oldenbourg.10.1515/9783486791204Suche in Google Scholar

Weiber, R., and D. Mühlhaus. 2014. Strukturgleichungsmodellierung. Heidelberg: Springer.10.1007/978-3-642-35012-2Suche in Google Scholar

Received: 2020-11-26
Accepted: 2021-04-26
Published Online: 2021-05-18
Published in Print: 2021-06-25

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Heruntergeladen am 29.1.2026 von https://www.degruyterbrill.com/document/doi/10.1515/spp-2020-0014/html
Button zum nach oben scrollen