Abstract
The study of dialectal microvariation in spoken Spanish faces challenges due to the absence of an adequate morpho-syntactically annotated and parsed corpus. Therefore, this article introduces a novel technique, a game-based approach, for creating resources for non-standard Spanish language varieties. The article provides an overview of the progress in designing three Games With A Purpose (GWAPs) prototypes, to wit, Agentes, Tesoros, and Anotatlón. These games aim to facilitate the confirmation and correction of the morpho-syntactic tagging task of the COSER-AP (Corpus Oral y Sonoro del Español Rural-Anotado y Parseado, ‘Annotated and Parsed Audible Corpus of Spoken Rural Spanish’). First, the article presents the methodology used to build the games. Second, it offers a detailed description of the implemented Game Design Elements (GDEs). Finally, the article discusses the results of a pilot evaluation that assesses player enjoyment and the linguistic accuracy. Findings are promising, with Tesoros and Anotatlón demonstrating high levels of enjoyment. Additionally, Agentes proves to be effective in collecting a large number of annotations. The linguistic accuracy also shows potential benefits of gamified approaches in linguistic annotation tasks. However, it also emphasizes the importance of considering regional in player assessment and training them in multidialectal contexts.
Referencias
Ahn, Luis von & Laura Dabbish. 2004. Labeling Images with a Computer Game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 319–326. New York: ACM. https://doi.org/10.1145/985692.98573310.1145/985692.985733Search in Google Scholar
Ahn, Luis von & Laura Dabbish. 2008. Designing Games With A Purpose. Communications of the ACM 51: 58–67. https://doi.org/10.1145/1378704.137871910.1145/1378704.1378719Search in Google Scholar
Bonilla, Johnatan E., Rosa Lilia Segundo Díaz & Miriam Bouzouita. En prensa. Using GWAPs for Verifying PoS Tagging of Spoken Dialectal Spanish. In 10th International Conference on Behavioural and Social Computing (BESC) 2023. IEEE.10.1109/BESC59560.2023.10386542Search in Google Scholar
Bonilla, Johnatan E, Miriam Bouzouita & Rosa Lilia Segundo Díaz. 2022. La construcción del Corpus Oral y Sonoro del Español Rural – Anotado y Parseado (COSER- AP): avances en el etiquetado de partes del discurso. Revista Internacional de Lingüística Iberoamericana 20: 77–96. https://doi.org/10.31819/rili-2022-20400610.31819/rili-2022-204006Search in Google Scholar
Bouzouita, Miriam, Johnatan E. Bonilla & Rosa Lilia Segundo Díaz. En prep. Gaming for Dialects: Creating an Annotated and Parsed Corpus of Rural Spanish Dialects through GWAPs. In Linguistic Corpora and Big Data in Spanish and Portuguese, eds. Miguel Calderón Campos & Gael Vaamonde Berlin: Mouton de Gruyter.Search in Google Scholar
Bouzouita, Miriam, Mónica Castillo Lluch & Enrique Pato. 2018. Dialectos del español: Una nueva aplicación para conocer la variación actual y el cambio en las variedades del español. Dialectología 20: 61–83.Search in Google Scholar
Bouzouita, Miriam, Mónica Castillo Lluch & Enrique Pato. 2021. Dialectos del español: Une application pour l’étude de la variation linguistique dans le monde hispanophone. Nouveaux regards sur la variation dialectale: 291–303.Search in Google Scholar
Bouzouita, Miriam, Mónica Castillo Lluch & Enrique Pato. 2022a. Dialectos del español: Apresentação da aplicação e primeiros resultados. Estudos em variação lingüística nas línguas románicas 209–227, eds. Lurdes de Castro Moutinho, Alber-to Gómez Bautista, Elisa Fernández Rei, Helena Rebelo, Rosa-Lídia Coimbra & Xulio Sousa. Universidade de Aveiro Editora.Search in Google Scholar
Bouzouita, Miriam, Mónica Castillo Lluch & Enrique Pato. 2022b. Dialectos del español: presentación de la app y primeros resultados. Revista Internacional de Lingüística Iberoamericana 20:59–76. https://doi.org/10.31819/rili-2022-20400510.31819/rili-2022-204005Search in Google Scholar
Boyle, Elizabeth A., Thomas M. Connolly, Thomas Hainey & James M. Boyle. 2012. Engagement in digital entertainment games: A systematic review. Computers in Human Behavior. Pergamon 28: 771–780. https://doi.org/10.1016/j.chb 2011.11.0 20.10.1016/j.chb.2011.11.020Search in Google Scholar
Caroux, Loïc, Katherine Isbister, Ludovic Le Bigot & Nicolas Vibert. 2015. Player-video game interaction: A systematic review of current concepts. Computers in Human Behavior, 366–381, ed. Robert D. Tennyson. https://doi.org/10.1016/j.chb.2015.01.06610.1016/j.chb.2015.01.066Search in Google Scholar
Chamberlain, Jon, Massimo Poesio & Udo Kruschwitz. 2008. Phrase Detectives: A Web-based Collaborative Annotation Game. Proceedings of I-Semantics, 42–49. Graz, Austria: ACM Press.Search in Google Scholar
Chklovski, Timothy. 2005. Collecting paraphrase corpora from volunteer contributors. In Proceedings of the 3rd International Conference on Knowledge Capture, K-CAP’05: 115–120. https://doi.org/10.1145/1088622.108864410.1145/1088622.1088644Search in Google Scholar
Cooper, Seth, Firas Khatib, Adrien Treuille, Janos Barbero, Jeehyung Lee, Michael Beenen, Andrew Leaver-Fay, David Baker, Zoran Popović & Foldit Players. 2010. Predicting protein structures with a multiplayer online game. Nature. Nature Publishing Group 466(7307): 756–760. https://doi.org/10.1038/nature0930410.1038/nature09304Search in Google Scholar
COSER- UD = Bonilla, Johnatan E. 2022. COSER- UD. https://github.com/johnatanebonilla/UD_Spanish-COSER (October, 2023).Search in Google Scholar
COSER = Fernández-Ordóñez, Inés (dir.). 2005–. Corpus Oral y Sonoro del Español Rural. http://www.corpusrural.es (September, 2023).Search in Google Scholar
Curtis, Vickie. 2018. Motivation for Participation: From General Volunteerism to Online Citizen Science. In Online Citizen Science and the Widening of Academia: Distributed Engagement with Research and Knowledge Production, 69–92. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-77664-4_410.1007/978-3-319-77664-4_4Search in Google Scholar
Dialectos del español = Bouzouita, Miriam, Mónica Castillo Lluch & Enrique Pato. 2019. Dialectos del español. http://www.dialectosdelespanol.org (October 2023).Search in Google Scholar
Entringer, Nathalie, Peter Gilles, Sara Martin & Christoph Purschke. 2021. Schnëssen. Surveying language dynamics in Luxembourgish with a mobile research app. Linguistics Vanguard. de Gruyter Mouton. https://doi.org/10.1515/lingvan-2019-003110.1515/lingvan-2019-0031Search in Google Scholar
Finquelievich, Susana & Celina Fischnaller. 2014. Ciencia ciudadana en la Sociedad de la Información: nuevas tendencias a nivel mundial. CTS. Ciencia, tecnología y sociedad. REDES Centro de Estudios sobre Ciencia, Desarrollo y Educación Superior 9: 11–31.Search in Google Scholar
Fort, Karën. 2016. Collaborative Annotation for Reliable Natural Language Processing. Hoboken, NJ, USA: Wiley. https://doi.org/10.1002/978111930669610.1002/9781119306696Search in Google Scholar
Fort, Karën, Bruno Guillaume & Hadrien Chastant. 2014. Creating Zombilingo, a game with a purpose for dependency syntax annotation. In ACM International Conference Proceeding Series, 2–6. Association for Computing Machinery. https://doi.org/10.1145/2594776.259477710.1145/2594776.2594777Search in Google Scholar
Gaiser, Leonie Elisa & Yaron Matras. 2021. Using smartphones to document linguistic landscapes: The LinguaSnapp mobile app. Linguistics Vanguard 7, no. s1, pp. 20190012. de Gruyter Mouton. https://doi.org/10.1515/lingvan-2019-001210.1515/lingvan-2019-0012Search in Google Scholar
Guillaume, Bruno, Karën Fort & Nicolas Lefebvre. 2016. Crowdsourcing Complex Language Resources: Playing to Annotate Dependency Syntax. In COLING 2016 –26th International Conference on Computational Linguistics, Proceedings of COLING 2016: Technical Papers: 3041–3052, eds. Yuji Matsumoto & Rashmi Prasad.Search in Google Scholar
Haklay, Mordechai, Daniel Dörler, Florian Heigl, Marina Manzoni, Susanne Hecker & Katrin Vohland. 2021. What Is Citizen Science? The Challenges of Definition. In The Science of Citizen Science, 13–33, eds. Katrin Vohland, Anne Land-Zandstra, Luigi Ceccaron et al. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-58278-4_210.1007/978-3-030-58278-4_2Search in Google Scholar
Hilton, Nanna Haug. 2021. Stimmen: A citizen science approach to minority language sociolinguistics. Linguistics Vanguard 7, no. s1, pp. 20190017. de Gruyter Mouton. https://doi.org/10.1515/lingvan-2019-001710.1515/lingvan-2019-0017Search in Google Scholar
Hladká, Barbora, Jiří Mírovský & Pavel Schlesinger. 2009. Play the language: Play coreference. In ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf., 209–212.10.3115/1667583.1667648Search in Google Scholar
Honnibal, Matthew, Ines Montani, Sofie Van Landeghem & Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. Zenodo, Honolulu, HI, USA. https://doi.org/10.5281/zenodo.1212303Search in Google Scholar
Juegos del español = Bouzouita, Miriam, Johnatan E. Bonilla, Rosa Lilia Segundo Díaz, Véronique Hoste, Karin Coninx & Gustavo Rovelo Ruiz. 2022. Juegos del español. www.juegosdelespanol.com (October 2023).Search in Google Scholar
Kawrykow, Alexander, Gary Roumanis, Alfred Kam, Daniel Kwak, Clarence Leung, Chu Wu, Eleyine Zarour, Luis Sarmenta, Mathieu Blanchette & Jérôme Waldispühl. 2012. Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment. Ed. Pawel Michalak. PLoS ONE. Public Library of Science 7(3). e31362. https://doi.org/10.1371/journal.pone.003136210.1371/journal.pone.0031362Search in Google Scholar
Kolly, Marie-José & Adrian Leemann. 2015. Dialäkt Äpp: Communicating dialectology to the public crowdsourcing dialects from the public. In Trends in Phonetics and Phonology: Studies from German speaking Europe, 271–285, eds. Adrian; Leemann, Marie-José; Kolly, Stephan; Schmid & Volker Dellwo. Peter Lang. https://doi.org/https://doi.org/10.5167/uzh-117114Search in Google Scholar
Lafourcade, Mathieu. 2007. Making people play for Lexical Acquisition with the JeuxDeMots prototype. 7th International Symposium on Natural Language Processing (SNLP’07) 7. https://hal-lirmm.ccsd.cnrs.fr/lirmm-00200883(December, 2021).Search in Google Scholar
Lafourcade, Mathieu, Alain Joubert & Nathalie Le Brun. 2015. GWAPs for Natural Language Processing. In Games with a Purpose (Gwaps), 47–72, eds. Joseph Mariani & Patrick Paroubek Wiley. https://doi.org/10.1002/9781119136309.ch310.1002/9781119136309.ch3Search in Google Scholar
Leemann, Adrian, Marie-José Kolly & David Britain. 2018. The English Dialects App: The creation of a crowdsourced dialect corpus. Ampersand. Elsevier 5: 1–17. https://doi.org/10.1016/j.amper.2017.11.00110.1016/j.amper.2017.11.001Search in Google Scholar
Madge, Chris. 2019. Gamifying Language Resource Acquisition. https://qmro.qmul.ac.uk/xmlui/handle/123456789/68617Search in Google Scholar
Madge, Chris, Richard Bartle, Jon Chamberlain, Udo Kruschwitz & Massimo Poesio. 2019. Incremental Game Mechanics applied to Text Annotation. In CHI PLAY 2019 – Proceedings of the Annual Symposium on Computer-Human Interaction in Play, 545–558. Association for Computing Machinery, Inc. https://doi.org/10.1145/3311350.334718410.1145/3311350.3347184Search in Google Scholar
Mekler, Elisa D., Julia Ayumi Bopp, Alexandre N. Tuch & Klaus Opwis. 2014. A systematic review of quantitative studies on the enjoyment of digital entertainment games. In Conference on Human Factors in Computing Systems –Proceedings, 927–936. New York, New York, USA: Association for Computing Machinery. https://doi.org/10.1145/2556288.255707810.1145/2556288.2557078Search in Google Scholar
Millour, Alice & Karën Fort. 2018. Toward a lightweight solution for less-resourced languages: Creating a POS tagger for Alsatian using voluntary crowdsourcing. In LREC 2018 –11th International Conference on Language Resources and Evaluation, 455–460, eds. Nicoletta Calzolari, Khalid Choukri, Christopher Cieri et al.Search in Google Scholar
Möller, Robert. 2021. An online atlas of colloquial German: The Atlas zur deutschen Alltagssprache. https://dialnet.unirioja.es/servlet/articulo?codigo=8323125. (March, 2022).Search in Google Scholar
Nivre, Joakim, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers & Daniel Zeman. 2020. Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. In LREC 2020 –12th International Conference on Language Resources and Evaluation, Conference Proceedings, 4034–4043. European Language Resources Association (ELRA). https://arxiv.org/abs/2004.10643v1 (December, 2021).Search in Google Scholar
Poesio, Massimo, Jon Chamberlain & Udo Kruschwitz. 2017. Crowdsourcing. Handbook of Linguistic Annotation 277–295. Dordrecht: Springer. https://doi.org/10.1007/978-94-024-0881-2_1010.1007/978-94-024-0881-2_10Search in Google Scholar
Poesio, Massimo, Jon Chamberlain, Udo Kruschwitz, Livio Robaldo & Luca Ducceschi. 2013. Phrase detectives: Utilizing collective intelligence for internet-scale language resource creation. ACM Transactions on Interactive Intelligent Systems. Association for Computing Machinery 3(1): 1–44. https://doi.org/10.1145/2448116.2448119Search in Google Scholar
Poesio, Massimo, Jon Chamberlain, Udo Kruschwitz, Livio Robaldo & Luca Ducceschi. 2015. Phrase Detectives: Utilizing Collective Intelligence for Internet-Scale Language Resource Creation (Extended Abstract). Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015). https://doi.org/10.1145/2448116.244811910.1145/2448116.2448119Search in Google Scholar
Purschke, Christoph. 2021. Crowdscapes. Participatory research and the collaborative (re)construction of linguistic landscapes with Lingscape. Linguistics Vanguard 7, no. s1, pp. 20190032. de Gruyter Mouton. https://doi.org/10.1515/lingvan-2019-003210.1515/lingvan-2019-0032Search in Google Scholar
Segundo Díaz, Rosa Lilia, Gustavo Rovelo Ruiz, Miriam Bouzouita & Karin Coninx. 2022. Building blocks for creating enjoyable games – A systematic literature review. International Journal of Human-Computer Studies. Elsevier 159. 102758. https://doi.org/10.1016/j.ijhcs.2021.10275810.1016/j.ijhcs.2021.102758Search in Google Scholar
Segundo Díaz, Rosa Lilia, Gustavo Rovelo Ruiz, Miriam Bouzouita, Véronique Hoste & Karin Coninx. 2023a. The Influence of Personality Traits and Game Design Elements on Player Enjoyment: An Empirical Study on GWAPs for Linguistics. In: Games and Learning Alliance 12th International Conference, GALA 2023 November 29–December 1, 2023, Proceedings, vol. 14475: 204–213, eds. Pierpaolo Dondio, Mariana Rocha, Attracta Brennan, Avo Schönbohm, Francesca de Rosa, Antti Koskinen & Francesco Bellotti. Dublin: Springer Nature Switzerland AG.10.1007/978-3-031-49065-1_20Search in Google Scholar
Segundo Díaz, Rosa Lilia, Gustavo Rovelo Ruiz, Miriam Bouzouita, Véronique Hoste & Karin Coninx. 2023b. The Influence of Personality Traits and Game Design Elements on Player Enjoyment: A Demo on GWAPs for Part-of-Speech Tagging. In Serious Games. JCSG 2023. Lecture Notes in Computer Science, vol. 14309: 353–361, eds. Stefan Göbel, Mads Haahr & Alberto Rojas-Salazar. Dublin: Springer International Publishing https://doi.org/10.1007/978-3-031-44751-8_2810.1007/978-3-031-44751-8_28Search in Google Scholar
Stöckle, Philipp. 2021. Wörterbuch der Bairischen Mundarten in Österreich (WBÖ). Germanistische Dialektlexikographie zu Beginn des 21 Jahrhunderts, 11–46, eds. Alexandra Lenz & Philipp Stöckle. Stuttgart: Franz Steiner.Search in Google Scholar
Vaux, Bert & Scott Golder. 2003. The Harvard dialect survey. Cambridge, MA: Harvard University Linguistics Department.Search in Google Scholar
Venhuizen, Noortje J., Valerio Basile, Kilian Evang, Johan Bos, Valerio Basile, Johan Bos & Noortje J. Venhuizen. 2013. Gamification for word sense labeling. In Proceedings of the 10th International Conference on Computational Semantics (IWCS’13)-Short Papers, 397–403, eds. Kartin Erk & Alexander Koller. Potsdam: University of Groningen.Search in Google Scholar
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Artikel
- Prepositional Usage in Modern Irish: Range and Variation
- The North Ukrainian Dialect of Vyšneve in the East Slavic Context: Towards a Final Description (part 2)
- Geographical Variation of Systems of Sibling Terms in Asia and Africa
- Diversidad denominativa del Pisum sativum en español: una historia de variedad y confusión
- Quantitative Perspectives on the Geolinguistic Structures of Dialect Morphology in Austria
- A synchronic and diachronic reappraisal of Indo-European *dʱug̑ʱh2ter- ‘daughter’ and *suhxnú- ‘son’ in Celtic dialects, Insular and Continental
- Juegos con propósito para la anotación del Corpus Oral Sonoro del Español rural
- Besprechungen
- Besprechungen – Comptes rendus – Reseñas – Reviews
Articles in the same Issue
- Frontmatter
- Artikel
- Prepositional Usage in Modern Irish: Range and Variation
- The North Ukrainian Dialect of Vyšneve in the East Slavic Context: Towards a Final Description (part 2)
- Geographical Variation of Systems of Sibling Terms in Asia and Africa
- Diversidad denominativa del Pisum sativum en español: una historia de variedad y confusión
- Quantitative Perspectives on the Geolinguistic Structures of Dialect Morphology in Austria
- A synchronic and diachronic reappraisal of Indo-European *dʱug̑ʱh2ter- ‘daughter’ and *suhxnú- ‘son’ in Celtic dialects, Insular and Continental
- Juegos con propósito para la anotación del Corpus Oral Sonoro del Español rural
- Besprechungen
- Besprechungen – Comptes rendus – Reseñas – Reviews