OER Statistics Framework: Recognizing and rewarding Open Educational Resources in Academic Evaluation
-
Sylvia Kullmann
Sylvia Kullmann, M.Sc., conducts research at the DIPF | Leibniz Institute for Research and Information in Education in Frankfurt am Main. Her main areas of focus include Open Educational Resources (OER), scientometric assessment of teaching performance, responsible research evaluation, and open science infrastructures. Her work concentrates on developing a framework for the evaluation of OER in the context of open scholarship and on advancing the visibility and recognition of teaching within academia.
und
Verena Weimer
Verena Weimer conducts research at DIPF | Leibniz Institute for Research and Information in Education on the scientometric recording of open educational resources (OER) and the inclusion of teaching performance in the academic recognition and reward system. She studied education at the University of Siegen (Germany) and Gender Studies at Humboldt-Universität zu Berlin (Germany) and is currently pursuing her doctorate at CWTS at Leiden University (Netherlands).
Abstract
Academic teaching is an essential part of science. Nevertheless, teaching achievements are not recognized and rewarded to the same extent in academic evaluations as other achievements, in particular research activities. Against the backdrop of the Responsible Research Assessment (RRA) movement, the OER Statistics Framework was developed to make teaching performance visible in scientific evaluations by treating OER as a scientometric measurement object. OER-based assessments are carried out at the individual level in the case of scientists and at the institutional level for larger units like research groups, institutes or universities. The assessments comprise classical productivity, cooperation, resonance, openness, and altmetric indicators, as well as innovative indicators representing knowledge transfer between teaching and research (transfer indicators). At the institutional level, newly introduced support indicators are used to record performance in terms of an OER-promoting ecosystem.
Zusammenfassung
Die akademische Lehre ist ein wesentlicher Bestandteil wissenschaftlicher Arbeit. Doch werden Lehrleistungen bei akademischen Bewertungen nicht in gleichem Maße anerkannt und honoriert wie etwa Forschungsaktivitäten. Um die Lehrleistung in wissenschaftlichen Bewertungen sichtbar zu machen wurde vor dem Hintergrund der Responsible Research Assessment (RRA)-Bewegung das OER Statistics Framework entwickelt, mit dem OER als scientometrische Messobjekte behandelt werden können. OER-basierte Bewertungen werden auf individueller Ebene und auf institutioneller Ebene für größere Einheiten wie Forschungsgruppen, Institute oder Universitäten durchgeführt. Die Bewertungen umfassen klassische Indikatoren für Produktivität, Zusammenarbeit, Reichweite, Offenheit und Altmetrik sowie innovative Indikatoren, die den Wissenstransfer zwischen Lehre und Forschung kennzeichnen (Transferindikatoren). Auf institutioneller Ebene werden neu eingeführte Unterstützungsindikatoren verwendet, um die Leistung in Bezug auf ein OER-förderndes Ökosystem zu erfassen.
Résumé
L’enseignement universitaire est une composante essentielle du travail scientifique. Cependant, les prestations d’enseignement ne sont pas reconnues et récompensées de la même manière que les activités de recherche dans les évaluations universitaires. Afin de rendre visibles les prestations d’enseignement dans les évaluations scientifiques, le cadre statistique REL a été développé dans le contexte du mouvement Responsible Research Assessment (RRA), qui permet de traiter les REL comme des objets de mesure scientométriques. Les évaluations basées sur les REL sont réalisées au niveau individuel et au niveau institutionnel pour des entités plus importantes telles que des groupes de recherche, des instituts ou des universités. Les évaluations comprennent des indicateurs classiques de productivité, de collaboration, de portée, d’ouverture et d’altmétrie, ainsi que des indicateurs innovants qui caractérisent le transfert de connaissances entre l’enseignement et la recherche (indicateurs de transfert). Au niveau institutionnel, de nouveaux indicateurs de soutien sont utilisés pour mesurer les performances en termes d’écosystème favorisant les REL.
Article Note
Authorship is evenly split at 50/50 %. The order of authorship is alphabetical and has no further meaning.
Über die Autoren
Sylvia Kullmann, M.Sc., conducts research at the DIPF | Leibniz Institute for Research and Information in Education in Frankfurt am Main. Her main areas of focus include Open Educational Resources (OER), scientometric assessment of teaching performance, responsible research evaluation, and open science infrastructures. Her work concentrates on developing a framework for the evaluation of OER in the context of open scholarship and on advancing the visibility and recognition of teaching within academia.
Verena Weimer conducts research at DIPF | Leibniz Institute for Research and Information in Education on the scientometric recording of open educational resources (OER) and the inclusion of teaching performance in the academic recognition and reward system. She studied education at the University of Siegen (Germany) and Gender Studies at Humboldt-Universität zu Berlin (Germany) and is currently pursuing her doctorate at CWTS at Leiden University (Netherlands).
References
Coalition for Advancing Research Assessment (CoARA), 2022. Agreement on Reforming Research Assessment. https://coara.eu/agreement/the-agreement-full-text/#:~:text=The%20Agreement%20on%20Reforming%20Research%20Assessment%20sets%20a,to%20maximise%20the%20quality%20and%20impact%20of%20research [28.10.2025].Suche in Google Scholar
Creative Commons (CC), 2019. About CC Licenses. https://creativecommons.org/share-your-work/cclicenses/ [28.10.2025].Suche in Google Scholar
Creative Commons (CC), 2022. Best practices for attribution. https://wiki.creativecommons.org/wiki/Best_practices_for_attribution#Attributing_text [28.10.2025].Suche in Google Scholar
Deutsches Zentrum für Hochschul- und Wissenschaftsforschung (DZHW), 2021. Methodik. Bibliometrischer Indikatorbericht für Institutionen. Version 20211007. https://bibliometrie.info/downloads/Methodikanhang.pdf [28.10.2025].Suche in Google Scholar
Hicks, Diana, Wouters, Paul, Waltman, Ludo, de Rijke, Sarah & Rafols, Ismael, 2015. The Leiden Manifesto for research metrics: use these 10 principles to guide research evaluation. Nature, April 23, 520:429–431. https://doi.org/10.1038/520429a [28.10.2025].10.1038/520429aSuche in Google Scholar
Humboldt, Wilhelm, 2022; around 1810. Über die innere und äußere Organisation der höheren wissenschaftlichen Anstalten in Berlin. In: Lauer, G.: Wilhelm von Humboldt. Schriften zur Bildung. Stuttgart: Reclam, S. 152–165.Suche in Google Scholar
Kerres, Michael, & Heinen, Richard, 2015. Open informational ecosystems: The missing link for sharing resources for education. International Review of Research in Open and Distributed Learning, 16(1). https://doi.org/10.19173/irrodl.v16i1.2008 [28.10.2025].10.19173/irrodl.v16i1.2008Suche in Google Scholar
Kullmann, Sylvia, & Rasulzade, Shahla, 2025. What is a recognizable contribution? On the characteristics of OER authorship. 18. Internationales Symposium für Informationswissenschaft (ISI 2025). Chemnitz, Deutschland. https://doi.org/10.5281/zenodo.14925594 [28.10.2025].Suche in Google Scholar
Kullmann, Sylvia & Weimer, Verena, 2023. Teaching as Part of Open Scholarship – Scientometric Indicators for Open Educational Resources. Proceedings of ISSI 2023 – the 19th International Conference of the International Society for Scientometrics and Informetrics, 1, 667–683. https://doi.org/10.5281/zenodo.8246995 [28.10.2025].Suche in Google Scholar
Kullmann, Sylvia & Weimer, Verena, 2024. Teaching as part of Open Scholarship – Developing a scientometric framework for Open Educational Resources. Scientometrics. https://doi.org/10.1007/s11192-024-05007-1 [28.10.2025].10.1007/s11192-024-05007-1Suche in Google Scholar
Kullmann, Sylvia, 2025a. Teaching counts! Open Educational Resources as an object of measurement for scientometric analysis. Quantitative Science Studies 6, 216–237. https://doi.org/10.1162/qss_a_00346 [28.10.2025].10.1162/qss_a_00346Suche in Google Scholar
Kullmann, Sylvia, 2025b. Towards OER Statistics: Data Requirements for the Scientometric Analysis of Open Educational Resources. Open Information Science, 9(1), 20250017. https://doi.org/10.1515/opis-2025-0017 [28.10.2025].10.1515/opis-2025-0017Suche in Google Scholar
Mayrberger, Kerstin & Thiemann, Stefan, 2018. Jenseits von Selbstreferenzialität. Awareness for Openness @ UHH. In: Synergie – Fachmagazin für Digitalisierung in der Lehre (5). https://www.synergie.uni-hamburg.de/de/media/ausgabe05/synergie05-beitrag18-mayrberger-thiemann.pdf [28.10.2025].Suche in Google Scholar
Müskens, Wolfgang, Zawacki-Richter, Olaf, Dolch, Carina, 2022. Instrument zur Qualitätssicherung von OER – IQOer – Entwicklungsversion 17. https://www.researchgate.net/publication/359534360_Instrument_zur_Qualitatssicherung_von_OER_-_IQOer_-_Entwicklungsversion_17 [28.10.2025].Suche in Google Scholar
OER Repo-AG, 2021. LOM for Higher Education OER Repositories. https://dini-ag-kim.github.io/hs-oer-lom-profil/latest/ [28.10.2025].Suche in Google Scholar
Open Science Assessment Framework (OSAF), 2023. GraspOS Open Research Assessment Dataspace. https://graspos.eu/open-science-assessment-framework [28.10.2025].Suche in Google Scholar
Tunger, Dirk, Clermont, Marcel, Dirksen, Alexander, & Scheidt, Barbara, 2017. Citation metrics as an additional indicator for evaluating research performance? An analysis of their correlations and validity. Business Research, 10(2), 1–28. https://doi.org/10.1007/s40685-017-0044-0 [28.10.2025].10.1007/s40685-017-0044-0Suche in Google Scholar
Schön, Sandra, Ebner, Martin, Berger, Elfriede, Brandhofer, Gerhard, Gröblinger, Ortrun, Jadin, Tanja, Kopp, Michael, Steinbacher, Hans-Peter & Zwiauer, Charlotte, 2021. OER Certification of Individuals and Organizations in Higher Education: Implementations Worldwide. Open Praxis, 13(3), 264–278. https://doi.org/10.5944/openpraxis.13.3.265 [28.10.2025].10.5944/openpraxis.13.3.265Suche in Google Scholar
Shaddim, 2016. Creative commons license spectrum (Graphic). https://commons.wikimedia.org/wiki/File:Creative_commons_license_spectrum.svg [28.10.2025].Suche in Google Scholar
UNESCO, 2019. Recommendation on open educational resources (OER). https://unesdoc.unesco.org/ark:/48223/pf0000373755/PDF/373755eng.pdf.multi.page=10 [28.10.2025].Suche in Google Scholar
VSNU, NFU, KNAW, NWO and ZonMw, 2019. Room for everyone’s talent. Towards a new balance in the recognition and rewards of academics’. https://recognitionrewards.nl/wp-content/uploads/2020/12/position-paper-room-for-everyones-talent.pdf [28.10.2025].Suche in Google Scholar
Weimer, Verena, Heck, Tamara, van Leeuwen, Thed & Rittberger, Marc, 2023. The quantification of open scholarship—a mapping review. Quantitative Science Studies, 4 (3): 650–670. https://doi.org/10.1162/qss_a_00266 [28.10.2025].10.1162/qss_a_00266Suche in Google Scholar
Weimer, Verena, Alt, Rebecca & Hiebl, Johannes, 2024. Wissenschaftswettbewerb = Wettbewerb in Forschung und Lehre? In: Webler, W.-D. [Hrsg.]: Überzogener und überhitzter Wettbewerb in der Wissenschaft II. Herausforderungen bei der Ausgestaltung von Wettbewerb – Theoretische und analytische Perspektiven. Bielefeld: UVW UniversitätsverlagWebler.Suche in Google Scholar
Wiley, David, 2021. Defining the “Open” in open content and open educational resources. In Y. Arts, H. Call, M. Cavan, T. P. Holmes, R. Rogers, S. Tuiloma, L. West, & R. Kimmons (Eds.), An introduction to open education. EdTech Books. https://edtechbooks.org/open_education/defining_the_open [28.10.2025].Suche in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
