Startseite Vergleich der menschlichen, maschinellen und Post-Editing-Übersetzung aus dem Slowakischen ins Deutsche mittels automatischer Evaluation
Artikel
Lizenziert
Nicht lizenziert Erfordert eine Authentifizierung

Vergleich der menschlichen, maschinellen und Post-Editing-Übersetzung aus dem Slowakischen ins Deutsche mittels automatischer Evaluation

  • Daša Munková EMAIL logo , Oľga Wrede und Jakub Absolon
Veröffentlicht/Copyright: 28. Mai 2019
Veröffentlichen auch Sie bei De Gruyter Brill

Summary

The aim of the study is to compare translation quality and effectiveness (translator productivity) using measures of the automatic evaluation of machine translation output. The examined translation(s) was a legal text, translated from Slovak (mother tongue) into German. We distinguish human translation (HT), machine translation (MT) and post-edited MT (PEMT). For the evaluation we used our own tool, wherein were implemented the metrics of automatic MT evaluation. Analysis of Variance and Multiply Comparisons (Tukey HSD test) were applied to textual data. The results proved that in terms of productivity, regarding the expert and terminological knowledge of human translators, post-editing of MT is more effective than HT. On the other hand, in terms of quality (sentence structure or stylistics), regardless of time, HT is of a higher quality then MT, but similar to PEMT. The post-editors were affected by grammar and by stylistics of MT output, but post-edited a larger volume of MT than human translators translated.

Literaturangaben

Arnold, Doug J., Lorna Balkan, Siety Meijer, R. Lee Humphreys & Louisa Sadler. 1994. Machine Translation: An Introductory Guide. London: Blackwells-NCC.Suche in Google Scholar

Aziz, Wilker, Maarit Koponen & Lucia Specia. 2014. Sub-sentence level analysis of machine translation post-editing effort. In Sharon O’Brien, Laura Winther Balling, Michael Carl, Michel Simard & Lucia Specia (eds.), Post-editing of machine translation: processes and applications, 170–199. Newcastle upon Tyne: Cambridge Scholars Publishing.Suche in Google Scholar

Banerjee, Satanjeev & Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, June, 65–72. Michigan: Ann Arbor.Suche in Google Scholar

Bernth, Arendse & Claudia Gdaniec. 2001. MTranslatability. In Machine Translation 16(3). 175–218.10.1023/A:1019867030786Suche in Google Scholar

Carl, Michael. 2009. Grounding translation tools in translator’s activity data. In MT Summit 2009 Workshop: Beyond Translation Memories: New Tools for Translators. Ottawa, Canada. http://www.mt-archive.info/MTS-2009-TOC.htm. Letzter Zugriff: 02.02.2017.Suche in Google Scholar

Carl, Michael, Barbara Dragsted, Jakob Elming, Daniel Hardt & Arnt Lykke Jakobsen. 2011. The process of post-editing: A pilot study. In Proceedings of the 8th international NLPSC workshop. Special theme: Human-machine interaction in translation, 131–142. Frederiksberg: Samfundsliteratur.Suche in Google Scholar

de Almeida, Giselle & Sharon O’Brien. 2010. Analysing Post-Editing Performance: Correlations with Years of Translation Experience. In EAMT 2010. St Raphael, France. http://www.mtarchive.info/EAMT-2010-Almeida.pdf. Letzter Zugriff: 02.02.2017.Suche in Google Scholar

DePalma, Donald, Vijayalaxmi Hegde & Hélène Pielmeier. 2014. Common Sense Advisory’s annual report. commonsenseadvisory.com. http://www.commonsenseadvisory.com/Default.aspx?Aid=21545&Contenttype=ArticleDetAD&moduleId=390&tabID=63. Letzter Zugriff: 02.02.2017.10.20432/prd201763Suche in Google Scholar

DePalma, Donald, Hélène Pielmeier, Robert G. Stewart & Stephen Henderson. 2016. Common Sense Advisory’s annual report. commonsenseadvisory.com. http://www.commonsenseadvisory.com/AbstractView/tabid/74/ArticleID/36540/Title/TheLanguageServicesMarket2016/Default.aspx. Letzter Zugriff: 02.02.2017.Suche in Google Scholar

Doddington, George. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of the second international conference on Human Language Technology Research (HLT '02), San Diego, California, March 24–27, 138–145. San Francisco: Morgan Kaufmann Publishers Inc.10.3115/1289189.1289273Suche in Google Scholar

Doherty, Stephen, Sharon O’Brien & Michael Carl. 2010. Eye tracking as an MT evaluation technique. In Machine Translation 24(1). 1–13.10.1007/s10590-010-9070-9Suche in Google Scholar

Dragsted, Barbara & Inge Gorm Hansen. 2009. Exploring Translation and Interpreting Hybrids. The Case of Sight Translation. In Meta 54(3). 588–604.10.7202/038317arSuche in Google Scholar

Ehrensberger-Dow, Maureen. 2015. An Ergonomic Perspective of Professional Translation. In Meta 60(2). 328.10.7202/1032879arSuche in Google Scholar

Federico, Marcello, Alessandro Cattelan & Marco Trombetti. 2012. Measuring user productivity in machine translation enhanced computer assisted translation. In Proceedings of the Tenth Conference of the Association for Machine Translation in the Americas (AMTA), San Diego, California. http://amta2012.cloudapp.net/AMTA2012Files/papers/123.pdf. Letzter Zugriff: 26.01.2018.Suche in Google Scholar

Gaspari, Federico, Hala Almaghout & Stephen Doherty. 2015. A survey of machine translation competences: Insights for translation technology educators and practitioners. In Perspectives Studies in Translatology 23. 1–26. 10.1080/0907676X.2014.979842Suche in Google Scholar

Guerberof, Ana. 2009. Productivity and quality in MT post-editing. In MT Summit 2009 Workshop: Beyond Translation Memories: New Tools for Translators, August 26–30, Ottawa, Canada. http://www.mt-archive.info/MTS-2009-Guerberof.pdf. Letzter Zugriff: 02.02.2017.Suche in Google Scholar

Guerberof, Ana. 2014. The role of professional experience in post-editing from a quality and productivity perspective. In Sharon O'Brien, Laura Winther Balling, Michael Carl, Michel Simard & Lucia Specia (eds.), Post-editing of machine translation: processes and applications, 51–76. Newcastle upon Tyne: Cambridge Scholars Publishing. 10.1007/s10590-014-9155-ySuche in Google Scholar

House, Juliane. 1997. Translation Quality Asseement: a model revisited. Tubingen: Gunter Narr Verlag.Suche in Google Scholar

Interpreting and translating for Europe. Directorate General for INTERPRETATION. European Commission. 2016. https://ec.europa.eu/info/sites/info/files/en_print_2016.pdf. Letzter Zugriff: 02.02.2017.Suche in Google Scholar

Koehn, Philipp. 2009. A process study of computer-aided translation. In Machine Translation 23(2). 10.1007/s10590-010-9076-3Suche in Google Scholar

Koehn, Philipp & Ulrich Germann. 2014. The Impact of Machine Translation Quality on Human Post-editing. In Workshop on Humans and Computer-assisted Translation, 38–46. Gothenburg: Association for Computational Linguistics.10.3115/v1/W14-0307Suche in Google Scholar

Koponen, Maarit. 2012 a. Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. In The Journal of Specialised Translation 9(25). 131–148. http://www.jostrans.org/issue25/art_koponen.pdf. Letzter Zugriff: 26.01.2018.Suche in Google Scholar

Koponen, Maarit. 2012 b. Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations. In Proceedings of the Seventh Workshop on Statistical Machine Translation, 181–190. Montréal: Association for Computational Linguistics.Suche in Google Scholar

Krings, Hans P. 2001. Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Process. Ohio: Kent State University Press.Suche in Google Scholar

Läubli, Samuel, Mark Fishel, Gary Massey, Maureen Ehrensberger-Dow & Martin Volk. 2013. Assessing Post-Editing Efficiency in a Realistic Translation Environment. In Sharon O’Brien, Michael Simard & Lucia Specia (eds.), Proceedings of MT Summit XIV Workshop on Post-editing Technology and Practice, Nice, 2 September 2013, 83–91. http://www.mt-archive.info/10/MTS-2013-W4-Laubli.pdf. Letzter Zugriff: 16.03.2018.Suche in Google Scholar

Mariniello, Elanna & Afaf Steiert. 2016. The human role in a machine-translated world. In TCworld. http://www.tcworld.info/e-magazine/translation-and-localization/article/the-human-role-in-a-machine-translated-world/e-magazine. Letzter Zugriff: 14.01.2018.Suche in Google Scholar

Moorkens, Joss & Sharon O’Brien. 2015. Post-editing evaluations: Trade-offs between novice and professional participants. In Ilknur Durgar El-Kahlout, Mehmed Özkan, Felipe Sánchez Martínez, Gema Ramírez-Sánchez, Fred Hollowood & Andy Way (eds.), Proceedings of the 18thAnnual Conference of the European Association for Machine Translation (EAMT) 2015, 75–81. Antalya.Suche in Google Scholar

Munk, Michal, Daša Munková & Ľubomír Benko. 2016. Identification of relevant and redundant automatic metrics for MT evaluation. In MIWAI 2016: Multi-disciplinary Trends in Artificial Intelligence – 10th International Workshop. Chiang Mai, December 7–9, 141–152. Cham: Springer.10.1007/978-3-319-49397-8_12Suche in Google Scholar

Munková, Daša. 2013. Prístupy k strojovému prekladu: modely, metódy a problémy strojového prekladu. Nitra: UKF Nitra.Suche in Google Scholar

Munková, Daša, Jozef Kapusta & Martin Drlík. 2016. System for Post-Editing and Automatic Error Classification on Machine Translation. In DIVAI 2016: 11th International Scientific Conference on Distance Learning in Applied Informatics, May 2–4, Štúrovo, 571–581. Nitra: UKF Nitra.Suche in Google Scholar

Munková, Daša & Michal Munk. 2014. An automatic evaluation of machine translation and Slavic languages. In Proceedings of 8th International Conference on Application of Information and Communication Technologies (AICT 2014), October, 15–17, Astana, 447–451. Astana: IEEE.10.1109/ICAICT.2014.7035992Suche in Google Scholar

Munková, Daša & Michal Munk. 2016. Automatic metrics for machine translation evaluation and minority languages. In Proceedings of the Mediterranean Conference on Information & Communication Technologies (MedICT’15: Lecture Notes in Electrical Engineering, Volume 381), May 7–9, Saïdia, 631–636. Cham: Springer.10.1007/978-3-319-30298-0_69Suche in Google Scholar

Munková, Daša, Michal Munk & Ľudmila Adamová. 2013 a. Modelling of Language Processing Dependence on Morphological Features. In Advances in Intelligent Systems and Computing. 77–86. Heidelberg: Springer.10.1007/978-3-319-01466-1_7Suche in Google Scholar

Munková, Daša, Michal Munk & Martin Vozár. 2013 b. Data Pre-processing Evaluation for Text Mining: Transaction/Sequence model. In Procedia Computer Science 18. 1198–1207. 10.1016/j.procs.2013.05.286.10.1016/j.procs.2013.05.286Suche in Google Scholar

Munková, Daša, Michal Munk & Martin Vozár. 2013 c. Influence of Stop-Words Removal on Sequence Patterns Identification within Comparable Corpora. In Advances in Intelligent Systems and Computing. 67–76. Heidelberg: Springer.10.1007/978-3-319-01466-1_6Suche in Google Scholar

O´Brien, Sharon. 2006. Pauses as Indicators of Cognitive Effort in Post-editing Machine Translation Output. In Across Languages and Cultures 7(1). 1–21. https://core.ac.uk/download/pdf/11310916.pdf. Letzter Zugriff: 16.01.2018.10.1556/Acr.7.2006.1.1Suche in Google Scholar

O’Brien, Sharon. 2007. An Empirical Investigation of Temporal and Technical Post-Editing Effort. In Translation and Interpreting Studies 2(1). 83–136. 10.1075/tis.2.1.03obSuche in Google Scholar

O’Brien, Sharon. 2011. Towards Predicting Post-Editing Productivity. In Machine Translation 25. 197–215.10.1007/s10590-011-9096-7Suche in Google Scholar

Offersgaard, Lene, Claus Povlsen, Lisbeth Almsten & Bente Maegaard. 2008. Domain specific MT in use. In Proceedings of the 12th Annual Conference of the European Association for Machine Translation (EAMT 2008), September, 22–23, Hamburg University, 150–159. Hamburg: HITEC e.V.Suche in Google Scholar

Papineni, Kishore, Salim Roukos, Todd Ward & WeiJing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL), July, 311–318. Philadelphia: PA.10.3115/1073083.1073135Suche in Google Scholar

Plitt, Mirko. 2012. Observations on MTQuality: Results from Postediting and Raw MT Usability Tests at Autodesk. In 3e Journée d'études 'Traduction et Qualité'. University of Lille.Suche in Google Scholar

Plitt, Mirko & François Masselot. 2010. A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context. In The Prague Bulletin of Mathematical Linguistics 93. 7–16.10.2478/v10108-010-0010-xSuche in Google Scholar

Pym, Anthony, Claudio Sfreddo, Andy L. J. Chan & François Grin. 2014.The Status of the Translation Profession in the European Union (= The Anthem-European Union Series). London & New York: Anthem Press.Suche in Google Scholar

Ramos, Prieto Fernando. 2011. Developing Legal Translation Competence: An Integrative Process-Oriented Approach. In Comparative Linguistics 5. 7–21.10.14746/cl.2011.5.01Suche in Google Scholar

Sandrini, Peter. 2009. Der transkulturelle Vergleich von Rechtsbegriffen. In S. Šarčević (ed.), Legal Language in Action: Translation, Terminology, Drafting and Procedural Issues, 151–165. Zagreb: Globus.Suche in Google Scholar

Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla & John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA-2006), 8–12 August, 223–231. Cambridge, USA.Suche in Google Scholar

Teixeira, Cardoso Carlos da Silva. 2014. Data Collection Methods for Researching the Interaction between Translators and Translation Tools – An ‘Ecological’ Approach. In John W. Schwieter & ‎Aline Ferreira (eds.), The Development of Translation Competence: Theories and Methodologies from Psycholinguistics and Cognitive Science, 269–286. Newcastle upon Tyne: Cambridge Scholars Publishing.Suche in Google Scholar

Toury, Gideon. 1980. In Search of a Theory of Translation. Tel Aviv: The Poter Institute for Poetics and Semiotics.Suche in Google Scholar

Toury, Gideon. 1995. Descriptive Translation Studies and Beyond. Amsterdam & Philadelphia: John Benjamins Publishing Company.10.1075/btl.4Suche in Google Scholar

Vanden Bulcke, Patricia & Armand Hérogue. 2011. Quality Issues in the Field of Legal Translation. In Ilse Depraetere (ed.), Perspectives on Translation Quality, 211–248. Berlin: Walter de Gruyter.10.1515/9783110259889.211Suche in Google Scholar

Witczak, Olga. 2016. Incorporating post-editing into a computer-assisted translation course. A study of student attitudes. In Journal of Translator Education and Translation Studies 1(1). 35–55.Suche in Google Scholar

Wrede, Oľga. 2013. Law – language – translation = Právo – jazyk – translácia. In XLinguae 6(4). 20–36. Suche in Google Scholar

Wrede, Oľga. 2016. Legal text in terms of didactics in specialized translation = Právny text z pohľadu didaktiky odborného prekladu. In XLinguae 9(1). 59–82. 10.18355/XL.2016.09.01.59-82Suche in Google Scholar

Published Online: 2019-05-28
Published in Print: 2019-05-28

© 2018 Walter de Gruyter GmbH, Berlin/Boston

Heruntergeladen am 3.11.2025 von https://www.degruyterbrill.com/document/doi/10.1515/slaw-2018-0042/html?lang=de
Button zum nach oben scrollen