Home Challenging Valuations: How Rankings Navigate Contestation
Article
Licensed
Unlicensed Requires Authentication

Challenging Valuations: How Rankings Navigate Contestation

  • Leopold Ringel

    Leopold Ringel holds a PhD from Heinrich-Heine-University Duesseldorf and is a Lecturer in Sociology at Bielefeld University. His research focuses on the emergence of transparency as a global norm, how organizations enact visibility, the social construction of competition, and the production, institutionalization, and multiple impacts of rankings. Selected publications: Ringel, L., 2019: Unpacking the Transparency-Secrecy Nexus: Frontstage and Backstage Behaviour in a Political Party. Organization Studies 40: 705–723; Ringel, L., 2021: Stepping Into the Spotlight: How Rankings Become Public Performances. S. 53–76 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing; Ringel, L., J. Brankovic & T. Werron, 2020: The Organizational Engine of Rankings: Connecting “New” and “Old” Institutionalism. Politics & Governance 8: 36–47; Ringel, L. & T. Werron, 2021: Serielle Vergleiche: Zum Unterschied, den Wiederholung macht. Anhand der Geschichte von Kunst- und Hochschulrankings. Kölner Zeitschrift für Soziologie und Sozialpsychologie 73: 301–331.

    EMAIL logo
Published/Copyright: November 27, 2021
Become an author with De Gruyter Brill

Abstract

Accounts of why rankings are pervasive features of the modern world focus mostly on their properties as valuation devices that, upon entering the public sphere, exert pressure on the ranked. In doing so, however, research tends to overlook the important role played by the different types of organizations that produce rankings. To remedy this, the article draws from a qualitative study consisting of semi-structured interviews with members of these organizations to show that they put a great deal of effort into addressing and responding to different kinds of criticism. Working towards building and maintaining the credibility of rankings is thus revealed to require constant attention by their producers, who devise multiple procedures and rhetorical strategies to this end.

Zusammenfassung

Die Allgegenwart von Rankings wurde bisher vor allem in Zusammenhang mit ihren Eigenschaften als Praktiken der quantitativen öffentlichen Leistungsbewertung diskutiert. Infolgedessen hat der Beitrag jener Instanzen, die für die Herstellung von Rankings verantwortlich sind, deutlich weniger Aufmerksamkeit erhalten. Der Artikel nimmt auf diese Lücke in der Forschungsliteratur Bezug und präsentiert die zentralen Ergebnisse einer qualitativen Studie, die auf Interviews mit Mitgliedern von rankingproduzierenden Organisationen basiert. Es wird gezeigt, dass diese Organisationen erhebliche Mengen an Zeit und Aufwand darauf verwenden, um auf Kritik einzugehen und diese zu entkräften. Aktivitäten dieses Typs erscheinen folglich aus Sicht der Produzenten als Daueraufgabe, der sie sich durch den Einsatz verschiedener organisationaler Verfahren und rhetorischer Strategien widmen.


Article note

I am grateful to Walter Bartl, Jelena Brankovic, Sebastian Büttner, Simon Egbert, Maximilian Heimstädt, Thomas Laux, Clelia Minnetian, Kathia Serrano-Velarde, Tobias Werron, Stefan Wilbers, and the journal’s editors and reviewers for helpful comments on earlier versions of the article.


About the author

Leopold Ringel

Leopold Ringel holds a PhD from Heinrich-Heine-University Duesseldorf and is a Lecturer in Sociology at Bielefeld University. His research focuses on the emergence of transparency as a global norm, how organizations enact visibility, the social construction of competition, and the production, institutionalization, and multiple impacts of rankings. Selected publications: Ringel, L., 2019: Unpacking the Transparency-Secrecy Nexus: Frontstage and Backstage Behaviour in a Political Party. Organization Studies 40: 705–723; Ringel, L., 2021: Stepping Into the Spotlight: How Rankings Become Public Performances. S. 53–76 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing; Ringel, L., J. Brankovic & T. Werron, 2020: The Organizational Engine of Rankings: Connecting “New” and “Old” Institutionalism. Politics & Governance 8: 36–47; Ringel, L. & T. Werron, 2021: Serielle Vergleiche: Zum Unterschied, den Wiederholung macht. Anhand der Geschichte von Kunst- und Hochschulrankings. Kölner Zeitschrift für Soziologie und Sozialpsychologie 73: 301–331.

References

Arnold, N., 2021: Avoiding Competition: The Effects of Rankings in the Food Waste Field. S. 112–130 in: S. Arora-Jonsson, N. Brunsson, R. Hasse & K. Lagerström (Hrsg.), Competition: What It is and Why It Happens. Oxford: Oxford University Press.10.1093/oso/9780192898012.003.0007Search in Google Scholar

Arora-Jonsson, S., N. Brunsson & R. Hasse, 2020: Where Does Competition Come From? The Role of Organization. Organization Theory 1: 1–24.10.1177/2631787719889977Search in Google Scholar

Bandola-Gill, J. Grek & M. Ronzani, 2021: Beyond Winners and Losers: Ranking Visualizations as Alignment Devices in Global Public Policy. S. 27–52 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing.10.1108/S0733-558X20210000074027Search in Google Scholar

Barron, G., 2017: The Berlin Principles on Ranking Higher Education Institutions: Limitations, Legitimacy, and Value Conflict. Higher Education 73: 317–333.10.1007/s10734-016-0022-zSearch in Google Scholar

Bartl, W., C. Papilloud & A. Terracher-Lipinski, 2019: Governing by Numbers – Key Indicators and the Politics of Expectations. An Introduction. Historical Social Research 44: 7–43.Search in Google Scholar

Basel Institute of Governance, 2018: Basel AML Index 2018 Report. Basel: Basel Institute of Governance.Search in Google Scholar

Baty, P., 2016. THE World University Rankings 2016–2017 passes independent audit. https://www.timeshighereducation.com/news/world-university-rankings-2016-2017-passes-independent-auditSearch in Google Scholar

Blank, G., 2007: Critics, Ratings and Society. New York: Rowman and Littlefield Publishers.Search in Google Scholar

Blumer, H., 1954: What is Wrong with Social Theory? American Sociological Review 19: 3–10.10.4324/9781315129945-8Search in Google Scholar

Bowers, A. & M. Prato, 2019: The Role of Third-Party Rankings in Status Dynamics: How Does the Stability of Rankings Induce Status Changes? Organization Science 30: 1146–1164.10.1287/orsc.2019.1316Search in Google Scholar

Brankovic, J., 2021: “Measure of Shame”: Media Career of the Global Slavery Index. S. 103–125 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald.10.1108/S0733-558X20210000074030Search in Google Scholar

Brankovic, J., L. Ringel & T. Werron, 2018: How Rankings Produce Competition: The Case of Global University Rankings. Zeitschrift für Soziologie 47: 270–88.10.1515/zfsoz-2018-0118Search in Google Scholar

Brunsson, N., 1989: The Organization of Hypocrisy: Talk, Decisions and Action in Organizations. New York: Wiley.Search in Google Scholar

Burchard, A., A. Kühne & T. Warnecke, 2014: Europas superkompliziertes Ranking. Der Tagesspiegel. Abgerufen von: https://www.tagesspiegel.de/wissen/hochschulranking-u-multirank-europas-superkompliziertes-ranking/9887734.htmlSearch in Google Scholar

Cameron, E., J. Nuzzo & J. Bell, 2019: Global Health Security Index: Building Collective Action and Accountability. www.ghsindex.org/wp-content/uploads/2020/04/2019-Global-Health-Security-Index.pdf.Search in Google Scholar

Chong, P. & A. Bourgoin, 2020: Communicating Credibility by Expert Service Workers: The Credibility Tactics of Fiction Critics and Management Consultants. Valuation Studies 7: 65–100.10.3384/VS.2001-5992.2020.7.1.65Search in Google Scholar

Clune, C. & B. O’Dwyer, 2020: Organizing Dissonance Through Institutional Work: The Embedding of Social and Environmental Accountability in an Investment Field. Accounting, Organizations and Society 85: 101–130.10.1016/j.aos.2020.101130Search in Google Scholar

Dobusch, L. & G. Müller-Seitz, 2019: Closing for the Benefit of Openness: Lessons from Wikimedia’s Open Strategy Process. Organization Studies 40: 343–370.10.1177/0170840617736930Search in Google Scholar

Emey, J. & P. Cowley, 2020: Report Card on Alberta’s Elementary Schools 2020. Abgerufen von: https://www.compareschoolrankings.org/pdf/report-card-on-alberta-elementary-schools-2020-13568.pdfSearch in Google Scholar

Espeland, W. & M. Sauder, 2007: Rankings and Reactivity: How Public Measures Recreate Social Worlds. American Journal of Sociology 113: 1–40.10.1086/517897Search in Google Scholar

Espeland, W. & M. Sauder, 2016: Engines of Anxiety: Academic Rankings, Reputation, and Accountability. New York: Russell Sage Foundation.Search in Google Scholar

Espeland, W. & M. Stevens, 2008: A Sociology of Quantification. European Journal of Sociology 49: 401–436.10.1017/S0003975609000150Search in Google Scholar

Esposito, E. & D. Stark, 2019: What’s Observed in a Rating? Rankings as Orientation in the Face of Uncertainty. Theory, Culture & Society 36: 3–26.10.1177/0263276419826276Search in Google Scholar

Flyverbom, M., 2019: The Digital Prism: Transparency and Managed Visibilities in a Datafied World. Cambridge: Cambridge University Press.10.1017/9781316442692Search in Google Scholar

Fourcade, M., 2016: Ordinalization: Lewis A. Coser Memorial Award for Theoretical Agenda Setting 2014. Sociological Theory 34: 175–195.10.1177/0735275116665876Search in Google Scholar

Gray, G. & S. Silbey, 2014: Governing Inside the Organization: Interpreting Regulation and Compliance. American Journal of Sociology 120: 96–145.10.1086/677187Search in Google Scholar

Hamann, J., 2016: The Visible Hand of Research Performance Assessment. Higher Education 72: 761–779.10.1007/s10734-015-9974-7Search in Google Scholar

Hansen, H. & M. Flyverbom, 2015: The Politics of Transparency and the Calibration of Knowledge in the Digital Age. Organization 22: 872–889.10.1177/1350508414522315Search in Google Scholar

Heimstädt, M. & L. Dobusch, 2020: Transparency and Accountability: Causal, Critical and Constructive Perspectives. Organization Theory. https://doi.org/10.1177/263178772096421610.1177/2631787720964216Search in Google Scholar

Heintz, B., 2010: Numerische Differenz. Überlegungen zu einer Soziologie des (quantitativen) Vergleichs. Zeitschrift für Soziologie 39: 162–181.10.1515/zfsoz-2010-0301Search in Google Scholar

Heintz, B., 2019: Good – Better – Best. Comparisons and the Power of Ranking Orders. S. 306–332 in: W. Steinmetz (Hrsg.), The Force of Comparison. A New Perspective on Modern European History and the Contemporary World. New York, NY: Berghahn.10.2307/j.ctv1850gn8.16Search in Google Scholar

Honig, D. & C. Weaver, 2019: A Race to the Top? The Aid Transparency Index and the Social Power of Global Performance Indicators. International Organization 73: 579–610.10.1017/9781108763493.006Search in Google Scholar

Jeacle, I. & C. Carter, 2011: In TripAdvisor We Trust: Rankings, Calculative Regimes and Abstract Systems. Accounting, Organizations and Society 36: 293–309.10.1016/j.aos.2011.04.002Search in Google Scholar

Kelle, U., (2014): Theorization from Data. S. 554–568 in: U. Flick (Hrsg.), The Sage Handbook of Qualitative Data Analysis. London: Sage.10.4135/9781446282243.n38Search in Google Scholar

Kellogg, K.C., 2011): Challenging Operations: Medical Reform and Resistance in Surgery. Chicago, IL: University of Chicago Press.10.7208/chicago/9780226430010.001.0001Search in Google Scholar

Kiviat, B., 2019: The Art of Deciding with Data: Evidence from How Employers Translate Credit Reports into Hiring Decisions. Socio-Economic Review 17: 283–309.10.1093/ser/mwx030Search in Google Scholar

Kornberger, M., Justesen, L., A. Koed Madsen & J. Mouritsen (Hrsg.), 2015: Making Things Valuable. Oxford: Oxford University Press.10.1093/acprof:oso/9780198712282.001.0001Search in Google Scholar

Krüger, A.K. & F. Hesselmann, 2020: Sichtbarkeit und Bewertung. Zeitschrift für Soziologie 49: 145–163.10.1515/zfsoz-2020-0015Search in Google Scholar

Landahl, J., 2020: The PISA Calendar: Temporal Governance and International Large-Scale Assessments. Educational Philosophy and Theory 52: 625–639.10.4324/9781003247500-4Search in Google Scholar

Lim, M., 2018: The Building of Weak Expertise. The Work of Global University Rankers. Higher Education 75: 415–430.10.1007/s10734-017-0147-8Search in Google Scholar

Mau, S., 2019: The Metric Society: On the Quantification of the Social. Cambridge: Polity Press.Search in Google Scholar

Mehrpouya, A., & R. Samiolo, 2016: Performance Measurement in Global Governance: Ranking and the Politics of Variability. Accounting, Organizations and Society 55: 12–31.10.1016/j.aos.2016.09.001Search in Google Scholar

Merry, S.E., 2016: The Seductions of Quantification: Measuring Human Rights, Gender Violence, and Sex Trafficking. Chicago: University of Chicago Press.10.7208/chicago/9780226261317.001.0001Search in Google Scholar

Orlikowski, W. & S. Scott, 2014: What Happens When Evaluation Goes Online? Exploring Apparatuses of Valuation in the Travel Sector. Organization Science 25: 868–889.10.1287/orsc.2013.0877Search in Google Scholar

Pollock, N. & L. D’Adderio, 2012: Give Me a Two-by-Two Matrix and I Will Create the Market: Rankings, Graphic Visualisations and Sociomateriality. Accounting, Organizations and Society 37: 565–586.10.1016/j.aos.2012.06.004Search in Google Scholar

Porter, T., 1995: Trust in Numbers. The Pursuit of Objectivity in Science and Public Life. Princeton: Princeton University Press.10.1515/9781400821617Search in Google Scholar

Rindova, V., L. Martins Srinivas & D. Chandler, 2018: The Good, the Bad, and the Ugly of Organizational Rankings: A Multidisciplinary Review of the Literature and Directions for Future Research. Journal of Management 44: 2175–2208.10.1177/0149206317741962Search in Google Scholar

Ringel, L., 2021: Stepping Into the Spotlight: How Rankings Become Public Performances. S. 53–76 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing.10.1108/S0733-558X20210000074028Search in Google Scholar

Ringel, L., J. Brankovic & T. Werron, 2020: The Organizational Engine of Rankings: Connecting “New” and “Old” Institutionalism. Politics & Governance 8: 36–47.10.17645/pag.v8i2.2576Search in Google Scholar

Ringel, L., Espeland, W., M. Sauder & T. Werron, 2021: Worlds of Rankings. S. 1–23 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing.10.1108/S0733-558X20210000074026Search in Google Scholar

Ringel, L. & T. Werron, 2020: Where do Rankings Come From? A Historical-sociological Perspective on the History of Modern Rankings. S. 137–170 in: A. Epple, W. Erhart & J. Grave (Hrsg.), Practices of Comparing: Ordering and Changing the Worlds. Bielefeld: Bielefeld University Press.10.1515/9783839451663-006Search in Google Scholar

Ringel, L. & T. Werron, 2021: Serielle Vergleiche: Zum Unterschied, den Wiederholung macht. Anhand der Geschichte von Kunst- und Hochschulrankings. Kölner Zeitschrift für Soziologie und Sozialpsychologie 73: 301–331.10.1007/s11577-021-00750-2Search in Google Scholar

Samiolo, R. & A. Mehrpouya, 2021: Between Stakeholders and Third Parties: Regulatory Rankings and the Organization of Competition. S. 77–100 in: L. Ringel, W. Espeland, M. Sauder & T. Werron (Hrsg.), Research in the Sociology of Organizations: Worlds of Rankings. Bingley: Emerald Publishing.10.1108/S0733-558X20210000074029Search in Google Scholar

Sauder, M., 2008: Interlopers and Field Change: The Entry of U.S. News into the Field of Legal Education. Administrative Science Quarterly 53: 209–234.10.2189/asqu.53.2.209Search in Google Scholar

Sauder, M. & G. Fine, 2008: Arbiters, Entrepreneurs, and the Shaping of Business School Reputations. Sociological Forum 23: 699–723.10.1111/j.1573-7861.2008.00091.xSearch in Google Scholar

Seabrooke, L. & D. Wigan, 2015: How Activists Use Benchmarks. Reformist and Revolutionary Benchmarks for Global Economic Justice. Review of International Studies 41: 887–904.10.1017/S0260210515000376Search in Google Scholar

Scott, M. & S. Lyman, 1968: Accounts. American Sociological Review 33: 46–62.10.2307/2092239Search in Google Scholar

U-Multirank, 2019: Frequently Asked Questions. FAQ 2019 Release. Abgerufen von: https://www.umultirank.org/export/sites/default/press-media/documents/FAQ_U-Multirank_2019.pdfSearch in Google Scholar

Transparency International, 2021: Corruption Perceptions Index 2020: Technical Methodology Note. https://www.transparency.de/fileadmin/Redaktion/Publikationen/2021/CPI2020_TechnicalMethodologyNote.pdfSearch in Google Scholar

Velthuis, O., 2014: ArtRank and the Flippers: Apocalypse Now? Texte zur Kunst 24: 34–49.Search in Google Scholar

Waibel, D., T. Peetz & F. Meier, 2021: Valuation Constellations. Valuation Studies 8: 33–66.10.3384/VS.2001-5992.2021.8.1.33-66Search in Google Scholar

Werron, T. & L. Ringel, 2017: Rankings in a Comparative Perspective. Conceptual Remarks. S. 1–14 in: S. Lessenich (Hrsg.), Geschlossene Gesellschaften. Verhandlungen des 38. Kongresses der Deutschen Gesellschaft für Soziologie. Essen: DGS.Search in Google Scholar

Wilbers & J. Brankovic, 2021: The Emergence of University Rankings in the United States: A Historical-Sociological Account. Higher Education.Search in Google Scholar

Published Online: 2021-11-27
Published in Print: 2021-11-26

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 28.10.2025 from https://www.degruyterbrill.com/document/doi/10.1515/zfsoz-2021-0020/html
Scroll to top button