Abstract
Motivated by an increasing use of social media for the expression of personal stance towards a certain target, we analyse the language used to produce such opinionated content with expressions of sentiment, which represents the main data source for sentiment analysis. We use the first manually annotated corpus for sentiment analysis of the Serbian language developed for the service sector of higher education. Our study focuses on how various linguistic constructions, used in different context, influence the sentiment polarity of a text. Our findings indicate that sentiment expressions and negation have a most significant role in determining whether the text conveys positive, neutral, or negative sentiment, while intensifiers (words which either increase or decrease sentiment) have a considerable influence on sentiment intensity. We also present an analysis of the impact of conjunctions, conditional sentences, comparative and modal verbs, and pronouns on sentiment polarity and intensity. Based on the derived observations, we propose a set of rules that could be integrated with machine learning algorithms into an automated sentiment analysis system for the Serbian language. Our findings also make a much-needed contribution to the few currently available resources for natural language processing of Serbian.
References
Altrabsheh, N., M. Cocea and S. Fallahkha. 2014. “Learning sentiment from students’ feedback for real-time interventions in classrooms”. Paper presented at the Adaptive and Intelligent Systems – Third International Conference, volume 8779, Bournemouth, UK, 8–9 September. 40–49.Suche in Google Scholar
Altrabsheh, N., M.G. Medhat and M. Cocea. 2013. “SA-E: Sentiment analysis for education”. Paper presented at the 5th KES International Conference on Intelligent Decision Technologies, Sesimbra, Portugal, June. 255.Suche in Google Scholar
Balahur, A., J.M. Hermida and A. Montoyo. 2011. “Detecting implicit expressions of sentiment in text based on commonsense knowledge”. Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, ACL-HLT 2011, Portland, Oregon, USA. 53–60.Suche in Google Scholar
Blanco, E. and D. Moldovan. 2013. “Retrieving implicit positive meaning from negated statements”. Natural Language Engineering 20. 501–535.10.1017/S1351324913000041Suche in Google Scholar
Bollen, J., H. Mao and X.J. Zeng. 2011. “Twitter mood predicts the stock market”. Journal of Computational Science 2(1). 1–8.10.1016/j.jocs.2010.12.007Suche in Google Scholar
Bošnjak, Z., O. Grljević and M. Dimitrijević. 2018. “Primena inteligentnih tehnologija u visokom obrazovanju” [Application of intelligent technologies in higher education]. Anali Ekonomskog fakulteta u Subotici 54(39/2018). 291–303.10.5937/AnEkSub1839291BSuche in Google Scholar
Boucher, J. and C.E. Osgood. 1969. “The Pollyanna hypothesis”. Journal of Verbal Learning and Verbal Behaviour 8. 1–8.10.1016/S0022-5371(69)80002-2Suche in Google Scholar
Broß, J. 2013. Aspect-oriented sentiment analysis of customer reviews using distant supervision techniques. (PhD dissertation, Freie Universität Berlin.)Suche in Google Scholar
Caffi, C. and R.W. Janney. 1994. “Towards a pragmatics of emotive communication”. Journal of Pragmatics 22. 325–373.10.1016/0378-2166(94)90115-5Suche in Google Scholar
Calders, T. and M. Pechenizkiy. 2012. “Introduction to the special section on educational data mining”. SIGKDD Explorations 13(2). 3–6.10.1145/2207243.2207245Suche in Google Scholar
Carrillo de Albornoz, J. and L. Plaza. 2013. “An emotion-based model of negation, intensifiers, and modality for polarity and intensity classification”. Journal of the American Society for Information Science and Technology 64. 1618–1633.10.1002/asi.22859Suche in Google Scholar
Chapman, W., B. Will, H. Paul, C.F. Gregory and B.G. Buchanan. 2001. “A simple algorithm for identifying negated findings and diseases in discharge summaries”. Journal of Biomedical Informatics 34(5). 301–310.10.1006/jbin.2001.1029Suche in Google Scholar
Choi, Y. and C. Cardie. 2008. “Learning with compositional semantics as structural inference for subsentential sentiment analysis”. Proceedings of EMNLP 200810.3115/1613715.1613816Suche in Google Scholar
Chung, S.F. 2011. “Uses of ter- in Malay: A corpus-based study”. Journal of Pragmatics 43(3). 799–813.10.1016/j.pragma.2010.10.004Suche in Google Scholar
Collins, C. 2016. “Not even”. Natural Language Semantics 24(4). 291–303.10.1007/s11050-016-9124-5Suche in Google Scholar
Dalal, M.K. and M.A. Zaveri. 2014. “Opinion mining from online user reviews using fuzzy linguistic hedges”. Applied Computational Intelligence and Soft Computing 2014, Article ID 735942.Suche in Google Scholar
D’Avanzo, E. and G. Pilato. 2015. “Mining social network users opinions’ to aid buyers’ shopping decisions”. Computers in Human Behavior 51. 1284–1294.10.1016/j.chb.2014.11.081Suche in Google Scholar
Dave, K., S. Lawrence and D.M. Pennock. 2003. “Mining the Peanut Gallery: Opinion extraction and semantic classification of product reviews”. Paper presented at the 12th International Conference on World Wide Web, Budapest, Hungary, 20–24 May.10.1145/775152.775226Suche in Google Scholar
Fleiss, J. L. 1971. “Measuring nominal scale agreement among many raters”. Psychological Bulletin 76(5). 378–382.10.1037/h0031619Suche in Google Scholar
Fleiss, J.L., B. Levin and M.C. Paik. 2003. Statistical methods for rates and proportions Oxford: Wiley.10.1002/0471445428Suche in Google Scholar
Ganu, G., N. Elhadad and A. Marian. 2009. “Beyond the stars: Improving rating predictions using Review Text Content”. Paper presented at the 12th International Workshop on the Web and Databases (WebDB 2009), Providence, RI, USA, 28 June.Suche in Google Scholar
Garrote, M., C. Kimura, K. Matsui, M.A. Sandoval and E. Takamori. 2013. “C-ORAL-JAPON: Corpus of spontaneous spoken Japanese”. Corpus Linguistics and Linguistic Theory 11(2). 373–392.10.1515/cllt-2013-0004Suche in Google Scholar
González, A.O. and A.M. Ramos. 2013. “A comparative study of collocations in a native corpus and a learner corpus of Spanish”. Procedia – Social and Behavioral Sciences 95. 563–570.10.1016/j.sbspro.2013.10.683Suche in Google Scholar
Grafsgaard, J.F., J.B. Wiggins, E.K. Boyer, E.N. Wiebe and J.C. Lester. 2013. “Embodied affect in tutorial dialogue: Student gesture and posture”. Paper presented at the16th International Conference on Artificial Intelligence in Education AIED Memphis, TN, USA, 9–13 July.10.1007/978-3-642-39112-5_1Suche in Google Scholar
Grimmer, J. and B.M. Stewart. 2013. “Text as data: The promise and pitfalls of automatic content analysis methods for political texts”. Political Analysis. 1–31.10.1093/pan/mps028Suche in Google Scholar
Grefenstette, G., Q. Yan, S.G. James and D.A. Evans. 2004. “Coupling niche browsers and affect analysis for an opinion mining application”. Proceedings of RIAO 2004Suche in Google Scholar
Grljević, O. 2016. Sentiment u sadržajima sa društvenih mreža kao instrument unapređenja poslovanja visokoškolskih institucija [Sentiment in social networks as means of business improvement of higher education institutions]. (PhD dissertation, University of Novi Sad, Faculty of Economics in Subotica.)Suche in Google Scholar
Grljević, O. and Z. Bošnjak. 2015. “Development of Serbian Higher Education Corpus”. Paper presented at the 16th IEEE International Symposium on Computational Intelligence and Informatics (CINTI), Budapest, Hugary, 19–21 November.10.1109/CINTI.2015.7382918Suche in Google Scholar
Grljević, O., Z. Bošnjak and A. Kovačević. Forthcoming. “Opinion mining in higher education: A corpus-based approach”. Enterprise Information Systems, SI: The Artificial Intelligence-enabled Enterprise Information SystemsSuche in Google Scholar
Hammond, M. 2015. “Predicting the gender of Welsh nouns”. Corpus Linguistics and Linguistic Theory 12(2). 221–261.10.1515/cllt-2015-0001Suche in Google Scholar
Henricson, S. and M. Nelson. 2017. “Giving and receiving advice in higher education. Comparing Sweden-Swedish and Finland-Swedish supervision”. Journal of Pragmatics 109. 105–120.10.1016/j.pragma.2016.12.013Suche in Google Scholar
Hu, M. and B. Liu. 2004. “Mining and summarizing customer reviews”. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04 Seattle, WA, USA, 22-25 August. 168–177.Suche in Google Scholar
Huang, T.H., Y. Ho-Cheng and H.H. Chen. 2012. “Modeling Polyanna phenomena in Chinese sentiment analysis”. Proceedings of COLING 2012: Demonstration papers Mumbai, India. 231–238.Suche in Google Scholar
Hunston, S. 2011. Corpus approaches to evaluation: Phraseology and evaluative language New York: Routledge.10.4324/9780203841686Suche in Google Scholar
Hutto, C.J. and E. Gilbert. 2014. “VADER: A parsimonious rule-based model for sentiment analysis of social media text”. Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media Ann Arbor, MI.10.1609/icwsm.v8i1.14550Suche in Google Scholar
Iosif, E., I. Klasinas, G. Athanasopoulou, E. Palogiannidi, S. Georgiladakis, K. Louka and A. Potamianos. 2018. “Speech understanding for spoken dialogue systems: From corpus harvesting to grammar rule induction”. Computer Speech and Language 47. 272–297.10.1016/j.csl.2017.08.002Suche in Google Scholar
Iruskieta, M., A. Ilarraza deDiaz and M. Lersundi. 2013. “Establishing criteria for RST-based discourse segmentation and annotation for texts in Basque”. Corpus Linguistics and Linguistic Theory 11(2). 303–334.10.1515/cllt-2013-0008Suche in Google Scholar
Jia, L., C. Yu and W. Meng. 2009. “The effect of negation on sentiment analysis and retrieval effectiveness”. Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009 Hong Kong, China. 1827–1830.Suche in Google Scholar
Jing-Schmidt, Z. 2007. “Negativity bias in language: A cognitive-affective model of emotive intensifiers”. Cognitive Linguistics 18. 417–443.10.1515/COG.2007.023Suche in Google Scholar
Kayabaşi, D. and M. Özgen. 2018. “A phase-based account on NPI-licensing in Turkish”. Poznań Studies in Contemporary Linguistics 54(1). 83–113.10.1515/psicl-2018-0003Suche in Google Scholar
Kennedy, A. and D. Inkpen. 2006. “Sentiment classification of movie reviews using contextual valence shifters”. Computational Intelligence 22. 110–125.10.1111/j.1467-8640.2006.00277.xSuche in Google Scholar
Kessler, J.S. and N. Nicolov. 2009. “Targeting sentiment expressions through supervised ranking of linguistic configurations”. Proceedings of the Third International AAAI Conference on Weblogs and Social Media 90–97.10.1609/icwsm.v3i1.13948Suche in Google Scholar
Kessler, J. S., M. Eckert, L. Clark and N. Nicolov. 2010. “The 2010 ICWSM JDPA sentiment corpus for the automotive domain”. Paper presented at the 4th International AAAI Conference on Weblogs and Social Media Data Workshop Challenge (ICWSM-DWC 2010), Washington, DC, USA, 23–26 May.Suche in Google Scholar
Kim, E. and R. Klinger. 2018. “Who feels what and why? Annotation of a literature corpus with semantic roles of emotions”. Proceedings of the 27th International Conference on Computational Linguistics Santa Fe, NM, USA. 1345–1359.Suche in Google Scholar
Kim, J., E. Shaw, S. Wyner, T. Kim and J. Li. 2010. “Discerning affect in student discussions”. Paper presented at the Annual Meeting of the Cognitive Science Society, Portland, Oregon, USA, 11–14 August.Suche in Google Scholar
Kim, J., D. Choi, M. Hwang and P. Kim. 2014. “Analysis on smartphone related Twitter reviews by using opinion mining techniques”. Advanced Approaches to Intelligent Information and Database Systems, Studies in Computational Intelligence 551. 205–212.10.1007/978-3-319-05503-9_20Suche in Google Scholar
Landis, R. J. and G. G. Koch. 1977. “The measurement of observer agreement for categorical data”. Biometrics 33(1). 159–174.10.2307/2529310Suche in Google Scholar
Lin, Y.L. 2017. “Co-occurrence of speech and gestures: A multimodal corpus linguistic approach to intercultural interaction”. Journal of Pragmatics 117. 155–167.10.1016/j.pragma.2017.06.014Suche in Google Scholar
Litman, D.J. and K. Forbes-Riley. 2004. “Annotating student emotional states in spoken tutoring dialogues”. Paper presented at the 5th SIGdial Workshop on Discourse and Dialogue, Cambridge, MA, USA, April 30–May 1. 144–153.Suche in Google Scholar
Liu, B. 2012. Sentiment analysis and opinion mining. Morgan & Claypool Publishers.10.2200/S00416ED1V01Y201204HLT016Suche in Google Scholar
Liu, Y., X. Huang, A. An and X. Yu. 2007. “ARSA: A sentiment-aware model for predicting sales performance using blogs”. Paper presented at the SIGIR’07, Amsterdam, Netherlands, 23–27 July. 607–614.Suche in Google Scholar
Martin, J.R. and P.R.R. White. 2005. The language of evaluation: Appraisal in English London & New York: Palgrave Macmillan.10.1057/9780230511910Suche in Google Scholar
Medhat, W., A. Hassan and H. Korashy. 2014. “Sentiment analysis algorithms and applications: A survey”. Ain Shams Engineering Journal 5. 1093–1113.10.1016/j.asej.2014.04.011Suche in Google Scholar
Morsy, A.S. and A. Rafea. 2012. “Improving document-level sentiment classification using contextual valence shifters”. In Bouma, G., A. Ittoo, E. Métais and H. Wortmann (eds.), Natural language processing and information systems: Proceedings of the 17th International Conference on Applications of Natural Language to Information Systems Groningen, The Netherlands: Springer. 253–258.10.1007/978-3-642-31178-9_30Suche in Google Scholar
Nikolić, M. 2014. Kategorija stepena u srpskom jeziku (složena rečenica) [Category of degrees in Serbian (complex sentence)]. Beograd: Institut za srpski jezik SANU.Suche in Google Scholar
Pang, B., L. Lee and S. Vaithyanathan. 2002. “Thumbs up?: Sentiment classification using machine learning techniques”. Paper presented at the ACL-02 Conference on Empirical Methods in Natural Language Processing, University of Pennsylvania, Philadelphia, PA, USA, 6–7 July.10.3115/1118693.1118704Suche in Google Scholar
Pang, B. and L. Lee. 2008. “Opinion mining and sentiment analysis”. Foundations and Trends in Information Retrieval 2(1–2). 1–135.10.1561/9781601981516Suche in Google Scholar
Piper, P., I. Antonić, V. Ružić, S. Tanasić, Lj. Popović and B. Tošović. 2005. Sintaksa savremenog srpskog jezika – Prosta rečenica [Syntax of contemporary Serbian – The simple sentence]. Beograd: Institut za srpski jezik SANU, Beogradska knjiga, Matica srpska.Suche in Google Scholar
Polanyi, L. and A. Zaenen. 2006. “Contextual valence shifters”. In Shanahan, J.G., Y. Qu and J. Wiebe (eds.), Computing attitude and affect in text: Theory and applications Dordrecht: Springer. 1–10.Suche in Google Scholar
Poria, S., E. Cambria, L.W. Ku, C. Gui and A. Gelbukh. 2014. “A rule-based approach to aspect extraction from product reviews”. Proceedings of the Second Workshop on Natural Language Processing for Social Media (SocialNLP) Association for Computational Linguistics and Dublin City University. 28–37.10.3115/v1/W14-5905Suche in Google Scholar
Poulos, A. and M.J. Mahony. 2008. “Effectiveness of feedback: The students perspective”. Assessment & Evaluation in Higher Education 33(2). 143–154.10.1080/02602930601127869Suche in Google Scholar
Pustejovsky, J. and A. Stubbs. 2013. Natural language annotation for machine learning. O’Reilly Media, Inc.Suche in Google Scholar
Rumenapp, J.C. 2016. “Analyzing discourse analysis: Teachers’ views of classroom discourse and student identity”. Linguistics and Education 35. 26–36.10.1016/j.linged.2016.04.002Suche in Google Scholar
Sagum, R.A., G.J.M. de Vera, J.P.S. Lansang, S.D.R. Narciso and J.K. Respeto. 2015. “Application of language modelling in sentiment analysis for faculty comment evaluation”. Paper presented at the International Multi Conference of Engineers and Computer Scientists 2015 – IMECS 2015 I, Hong Kong, 18–20 March.Suche in Google Scholar
Saurí, R. 2008. A factuality profiler for eventualities in text. (PhD dissertation thesis, Brandeis University, Waltham.)Suche in Google Scholar
Sayeed, A.B., J. Boyd-Graber, B. Rusk and A. Weinberg. 2012. “Grammatical structures for word-level sentiment detection”. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Montreal, Canada, June 3–8, 2012. 667–676.Suche in Google Scholar
Schoenmueller, V., O. Netzer, F. Stahl. 2018. “The extreme distribution of online reviews: Prevalence, drivers and implications”. Columbia Business School Research Paper No. 18-10. 68.10.2139/ssrn.3100217Suche in Google Scholar
Stanojčić, Ž. and Lj. Popović. 2002. Gramatika srpskog jezika [A grammar of the Serbian language]. Beograd: Zavod za udžbenike i nastavna sredstva.Suche in Google Scholar
Su, Q., K. Xiang, H. Wang, B. Sun and S. Yu. 2006. “Using pointwise mutual information to identify implicit features in customer reviews”. In: Matsumoto Y., R.W. Sproat, K.F. Wong, M. Zhang (eds.), Computer processing of Oriental languages. Beyond the Orient: The research challenges ahead. ICCPOL 2006 (Lecture Notes in Computer Science, vol 4285.) Berlin: Springer.10.1007/11940098_3Suche in Google Scholar
Taboada, M. 2016. “Sentiment analysis: An overview from linguistics”. Annual Review of Linguistics 2. 325–347.10.1146/annurev-linguistics-011415-040518Suche in Google Scholar
Taboada, M., J. Brooke, M. Tofiloski, K. Voll and M. Stede. 2011. “Lexicon-based methods for sentiment analysis”. Computational Linguistics 37. 267–307.10.1162/COLI_a_00049Suche in Google Scholar
Tang, X. 2016. “Lexeme-based collexeme analysis with DepCluster”. Corpus Linguistics and Linguistic Theory 13(1). 165–202.10.1515/cllt-2015-0007Suche in Google Scholar
Tomasello, M. 2006. “Acquiring linguistic constructions”. In Siegler, R. and D. Kuhn (eds.), Handbook of child psychology: Cognitive development Oxford: Wiley. 255– 299.Suche in Google Scholar
Van de Kauter, M., D. Breesch and V. Hoste. 2015. “Fine-grained analysis of explicit and implicit sentiment in financial news articles”. Expert Systems with Applications 42: 4999–5010.10.1016/j.eswa.2015.02.007Suche in Google Scholar
Vass, H. 2017. “Lexical verb hedging in legal discourse: The case of law journal articles and Supreme Court majority and dissenting opinions”. English for Specific Purposes 48. 17–31.10.1016/j.esp.2017.07.001Suche in Google Scholar
Wachsmuth, H., M. Trenkmann, B. Stein, G. Engels and T. Palakarska. 2014. “A review corpus for argumentation analysis”. Paper presented at the15th International Conference on Computational Linguistics and Intelligent Text Processing, Kathmandu, Nepal, 06–12 April.10.1007/978-3-642-54903-8_10Suche in Google Scholar
Wen, M., D. Yang and C.R. Penstein. 2014. “Sentiment analysis in MOOC discussion forums: What does it tell us?” Paper presented at the 7th International Conference on Educational Data Mining, London, UK, 4–7 July. 130–137.Suche in Google Scholar
Wiebe, J., T. Wilson and C. Cardie. 2005. “Annotating expressions of opinions and emotions in language”. Language Resources and Evaluation 39(2–3). 165–210.10.1007/s10579-005-7880-9Suche in Google Scholar
Wyner, S., E. Shaw, T. Kim, J. Li and J. Kim. 2009. “Sentiment analysis of a student Q&A board for computer science”. Paper presented at the IJCAI workshop on Computational Models of Natural Argument, Pasadena, CA, USA, 13 July.Suche in Google Scholar
Yang, C.S., C.H. Chen and P.C. Chang. 2015. “Harnessing consumer reviews for marketing intelligence: a domain-adapted sentiment classification approach”. Information Systems and e-Business Management 13(3). 403–419.10.1007/s10257-014-0266-zSuche in Google Scholar
Yang, K. 2008. “WIDIT in TREC 2008 blog track: Leveraging multiple sources of opinion evidence”. Proceedings of The Seventeenth Text REtrieval Conference, TREC 2008 Gaithersburg, Maryland, USA, November 18–21, 2008.Suche in Google Scholar
Yoo, J. and J. Kim. 2014. “Capturing difficulty expressions in student online Q&A discussions”. Paper presented at the Twenty-Eighth {AAAI} Conference on Artificial Intelligence, Québec City, Québec, Canada, 27–31 July. 208–214.Suche in Google Scholar
© 2020 Faculty of English, Adam Mickiewicz University, Poznań, Poland
Artikel in diesem Heft
- Frontmatter
- The linguistic construction of sentiment expressions in student opinionated content: A corpus-based study
- Syntax meets discourse: Locative and deictic (directional) inversion in English
- A cognitive approach to semantic approximations in monolingual English-speaking children
- Shifting genres: Rendering bad language in the Polish voice-over of the Canadian drama American Heist
- Verb valency in interlanguage: An extension to valency theory and new perspective on L2 learning
- Introducing corpus-based translation studies
Artikel in diesem Heft
- Frontmatter
- The linguistic construction of sentiment expressions in student opinionated content: A corpus-based study
- Syntax meets discourse: Locative and deictic (directional) inversion in English
- A cognitive approach to semantic approximations in monolingual English-speaking children
- Shifting genres: Rendering bad language in the Polish voice-over of the Canadian drama American Heist
- Verb valency in interlanguage: An extension to valency theory and new perspective on L2 learning
- Introducing corpus-based translation studies