Home The role of syntactic dependencies in compositional distributional semantics
Article
Licensed
Unlicensed Requires Authentication

The role of syntactic dependencies in compositional distributional semantics

  • Pablo Gamallo EMAIL logo
Published/Copyright: January 24, 2017

Abstract

This article provides a preliminary semantic framework for Dependency Grammar in which lexical words are semantically defined as contextual distributions (sets of contexts) while syntactic dependencies are compositional operations on word distributions. More precisely, any syntactic dependency uses the contextual distribution of the dependent word to restrict the distribution of the head, and makes use of the contextual distribution of the head to restrict that of the dependent word. The interpretation of composite expressions and sentences, which are analyzed as a tree of binary dependencies, is performed by restricting the contexts of words dependency by dependency in a left-to-right incremental way. Consequently, the meaning of the whole composite expression or sentence is not a single representation, but a list of contextualized senses, namely the restricted distributions of its constituent (lexical) words. We report the results of two large-scale corpus-based experiments on two different natural language processing applications: paraphrasing and compositional translation.

Funding statement: This work is funded by Project TELPARES, Ministry of Economy and Competitiveness (FFI2014-51978-C2-1-R), and the program “Ayuda Fundación BBVA a Investigadores y Creadores Culturales 2016”.

Acknowledgments

We would like to thank the anonymous reviewers for helpful comments and suggestions.

References

Baroni, Marco. 2013. Composition in distributional semantics. Language and Linguistics Compass 7. 511–522.10.1111/lnc3.12050Search in Google Scholar

Baroni, Marco, Raffaella Bernardi & Roberto Zamparelli. 2014. Frege in space: A program for compositional distributional semantics. LiLT 9. 241–346.10.33011/lilt.v9i.1321Search in Google Scholar

Baroni, Marco, Silvia Bernardini, Adriano Ferraresi & Eros Zanchetta. 2009. The wacky wide web: A collection of very large linguistically processed webcrawled corpora. Language Resources and Evaluation 43(3). 209–226.10.1007/s10579-009-9081-4Search in Google Scholar

Baroni, Marco & Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP’10, 1183–1193. Stroudsburg, PA, USA.Search in Google Scholar

Barwise, Jon. 1987. Recent developments in situation semantics. Language and Artificial Intelligence. Berlin: Springer-Verlag.Search in Google Scholar

Coecke, B., M. Sadrzadeh & S. Clark. 2010. Mathematical foundations for a compositional distributional model of meaning. Linguistic Analysis 36(1–4). 345–384.Search in Google Scholar

Copestake, Ann & Aurelie Herbelot. 2012. Lexicalised compositionality. In http://www.cl.cam.ac.uk/ah433/lc-semprag.pdf.Search in Google Scholar

Costa, F., V. Lombardo, P. Frasconi & G. Soda. 2001. Wide coverage incremental parsing by learning attachment preferences. In Conference of the Italian Association for Artificial Intelligence (AIIA).10.1007/3-540-45411-X_30Search in Google Scholar

Davidson, Donald. 1969. The individuation of events, 216–234. Dordrecht: Springer Netherlands. ISBN 978-94-017-1466–2.10.1007/978-94-017-1466-2_11Search in Google Scholar

Delpech, Estelle, Béatrice Daille, Emmanuel Morin & Claire Lemaire. 2012. Extraction of domain-specific bilingual lexicon from comparable corpora: Compositional translation and ranking. In COLING2012, 24th International Conference on Computational Linguistics, Mumbai, India, 745–762.Search in Google Scholar

Dinu, G., N. Pham & M. Baroni. 2013a. Dissect: Distributional semantics composition toolkit. In ACL 2013 Workshop on Continuous Vector Space Models and their Compositionality (CVSC 2013), 31–36. East Stroudsburg, PA.Search in Google Scholar

Dinu, G., N. Pham & M. Baroni. 2013b. General estimation and evaluation of compositional distributional semantic models. In ACL 2013 Workshop on Continuous Vector Space Models and their Compositionality (CVSC 2013), 50–58. East Stroudsburg, PA.Search in Google Scholar

Erk, Katrin. 2013. Towards a semantics for distributional representations. In IWCS-2013.Search in Google Scholar

Erk, Katrin & Sebastian Padó. 2008. A structured vector space model for word meaning in context. In Proceedings of EMNLP. Honolulu, HI.10.3115/1613715.1613831Search in Google Scholar

Fellbaum, C. 1998. A semantic network of English: The mother of all WordNets. Computer and the Humanities 32. 209–220.10.1023/A:1001181927857Search in Google Scholar

Fung, Pascale & Lo Yuen Yee. 1998. An IR approach for translating new words from nonparallel, comparable texts. In Coling’98, 414–420. Montreal, Canada.10.3115/980451.980916Search in Google Scholar

Gamallo, Pablo. 2003. Cognitive characterisation of basic grammatical structures. Pragmatics and Cognition 11(2). 209–240.10.1075/pc.11.2.03oteSearch in Google Scholar

Gamallo, Pablo. 2007. Learning bilingual lexicons from comparable English and Spanish Corpora. In Machine Translation SUMMIT XI. Copenhagen, Denmark.Search in Google Scholar

Gamallo, Pablo. 2008. The meaning of syntactic dependencies. Linguistik OnLine 35(3). 33–53.10.13092/lo.35.522Search in Google Scholar

Gamallo, Pablo, Alexandre Agustini & Gabriel Lopes. 2005. Clustering syntactic positions with similar semantic requirements. Computational Linguistics 31(1). 107–146.10.1162/0891201053630318Search in Google Scholar

Gamallo, Pablo & Isaac González. 2011. A grammatical formalism based on patterns of part-of-speech tags. International Journal of Corpus Linguistics 16(1). 45–71.10.1075/ijcl.16.1.03gamSearch in Google Scholar

Gamallo, Pablo & José Ramom Pichel. 2008. Learning Spanish-Galician translation equivalents using a comparable corpus and a bilingual dictionary. LNCS 4919. 413–423.10.1007/978-3-540-78135-6_36Search in Google Scholar

Grefenstette, Gregory. 1996. Evaluation techniques for automatic semantic extraction: Comparing syntactic and window based approaches. In B. Boguraev & J. Pustejovsky (eds.), Corpus processing for lexical acquisition, 205–216. Cambridge, MA: The MIT Press.Search in Google Scholar

Grefenstette, Gregory. 1999. The World Wide Web as a resource for example-based machine translation tasks. In Translating and the Computer 21: Proceedings of the 21st International Conference on Translating and the Computer.Search in Google Scholar

Grefenstette, Edward, Mehrnoosh Sadrzadeh, Stephen Clark, Bob Coecke & Stephen Pulman. 2011. Concrete sentence spaces for compositional distributional models of meaning. In Proceedings of the Ninth International Conference on Computational Semantics, IWCS ’11, 125–134.Search in Google Scholar

Groenendijk, J. & M. Stokhof. 1991. Dynamic predicate logic. Linguistics and Philosophy 14. 39–100.10.1007/BF00628304Search in Google Scholar

Guevara, Emiliano. 2010. A regression model of adjective-noun compositionality in distributional semantics. In Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics, GEMS ’10.Search in Google Scholar

Hanks, Patrick. 2013. Lexical analysis: Norms and exploitations. Cambridge, MA: MIT Press.10.7551/mitpress/9780262018579.001.0001Search in Google Scholar

Harris, Zellig. 1954. Distributional structure. Word 10(23). 146–162.10.1080/00437956.1954.11659520Search in Google Scholar

Hudson, Richard. 2003. The psychological reality of syntactic dependency relations. In MTT 2003. Paris.Search in Google Scholar

Jezek, Elisabetta & Patrick Hanks. 2010. What lexical sets tell us about conceptual categories. Lexis [Online], 4 | 2010, Online since 14 April 2010. http://lexis.revues.org/555 (accessed 16 January 2017), DOI: 10.4000/lexis.555.10.4000/lexis.555Search in Google Scholar

Kahane, Sylvain. 2003. Meaning-text theory. In V. Ágel et al. (eds.), Dependency and valency: An international handbook of contemporary research. Berlin: De Gruyter.Search in Google Scholar

Kamp, H. & U. Reyle. 1993. From discourse to logic: Introduction to model-theoretic semantics of natural language. Formal logic and discourse representation theory. Dordrecht: Kluwer Academic Publisher.Search in Google Scholar

Kempson, R., W. Meyer-Viol & D. Gabbay. 1997. Language understanding: A procedural perspective. In C. Retore (ed.), First international conference on logical aspects of computational linguistics, 228–247. Lecture Notes in Artificial Intelligence Vol. 1328. Springer Verlag.10.1007/BFb0052160Search in Google Scholar

Kempson, R., W. Meyer-Viol & D. Gabbay. 2001. Dynamic syntax: The flow of language understanding. Oxford: Blackwell.Search in Google Scholar

Koehn, Philipp. 2009. Statistical machine translation. Cambridge: Cambridge University Press.10.1017/CBO9780511815829Search in Google Scholar

Krishnamurthy, Jayant & Tom Mitchell. 2013. Proceedings of the workshop on continuous vector space models and their compositionality, chap. Vector Space Semantic Parsing: A Framework for Compositional Vector Space Models, 1–10. Association for Computational Linguistics.Search in Google Scholar

Langacker, Ronald W. 1991. Foundations of cognitive grammar: Descriptive applications, vol. 2. Stanford: Stanford University Press.Search in Google Scholar

McRae, K., T.R. Ferreti & L. Amoyte. 1997. Thematic roles as verb-specific concepts. In M. MacDonald (ed.), Lexical representations and sentence processing, 137–176. Sussex, UK: Psychology Press.10.1080/016909697386835Search in Google Scholar

Meillet, Antoine. 1921. Linguistique historique et linguistique générale. Paris: La Société Linguistique de Paris.Search in Google Scholar

Milward, David. 1992. Dynamics, dependency grammar and incremental interpretation. In 14th Conference on Computational Linguistics (Coling92), 1095–1099. Nantes.10.3115/992424.992430Search in Google Scholar

Mitchell, Jeff & Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL-08: HLT, 236–244.Search in Google Scholar

Mitchell, Jeff & Mirella Lapata. 2009. Language models based on semantic composition. In Proceedings of EMNLP, 430–439.Search in Google Scholar

Mitchell, Jeff & Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive Science 34(8). 1388–1439.10.1111/j.1551-6709.2010.01106.xSearch in Google Scholar PubMed

Montague, Richard. 1970. Universal grammar. theoria. Theoria 36. 373–398.10.1111/j.1755-2567.1970.tb00434.xSearch in Google Scholar

Morin, Emmanuel & Béatrice Daille. 2012. Revising the compositional method for terminology acquisition from comparable corpora. In COLING2012, 24th International Conference on Computational Linguistics, Mumbai, India, 1797–1810.Search in Google Scholar

Navigli, Roberto. 2009. Word sense disambiguation: A survey. ACM Computing Surveys 41(2). 1–69.10.1145/1459352.1459355Search in Google Scholar

Partee, Barbara. 2007. Private adjectives: Subsective plus coercion. In R. Bäuerle, U. Reyle & T. E. Zimmermann (eds.), Presuppositions and discourse. Amsterdam: Elsevier.Search in Google Scholar

Pustejovsky, James. 1995. The generative lexicon. Cambridge: MIT Press.Search in Google Scholar

Rapp, Reinhard. 1999. Automatic identification of word translations from unrelated English and German Corpora. In ACL’99, 519–526.Search in Google Scholar

Schlesewsky, M. & I. Bornkessel. 2004. On incremental interpretation: Degrees of meaning accessed during sentence comprehension. Lingua 114. 1213–1234.10.1016/j.lingua.2003.07.006Search in Google Scholar

Schütze, Hinrich. 1998. Automatic word sense discrimination. Computational Linguistics 24(1). 97–124.Search in Google Scholar

Sperber, Dan & Deirdre Wilson. 1995. Relevance: Communication and cognition, 2nd edn. Oxford: Blackwell.Search in Google Scholar

Steedman, Mark. 1996. Surface structure and interpretation. Cambridge, MA: The MIT Press.Search in Google Scholar

Studtmann, Paul. 2014. Aristotle’s categories. In E. N. Zalta (ed.), The Stanford encyclopedia of philosophy. Summer 2014 edn.Search in Google Scholar

Tanaka, Takaaki & Timothy Baldwin. 2003. Noun-noun compound machine translation a feasibility study on shallow processing. In Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment, 17–24. Sapporo, Japan.10.3115/1119282.1119285Search in Google Scholar

Tanenhaus, M.K. & G.N. Carlson. 1989. Lexical structure and language comprehension. In W. Marslen-Wilson (ed.), Lexical representation and process, 530–561. Cambridge, MA: The MIT Press.Search in Google Scholar

Tesniére, Lucien. 1959. Eléments de syntaxe structurale. Paris: Klincksieck.Search in Google Scholar

Thater, Stefan, Hagen Fürstenau & Manfred Pinkal. 2010. Contextualizing semantic representations using syntactically enriched vector models. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 948–957. Stroudsburg, PA, USA.Search in Google Scholar

Truswell, J.C., M.K. Tanenhaus & S.M. Garnsey. 1994. Semantic influences on parsing: use of thematic role information in syntactic ambiguity resolution. Journal of Memory and Language 33. 285–318.10.1006/jmla.1994.1014Search in Google Scholar

Turney, Peter D. 2013. Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research (JAIR) 44. 533–585.10.1613/jair.3640Search in Google Scholar

Zanzotto, Fabio Massimo, Ioannis Korkontzelos, Francesca Fallucchi & Suresh Manandhar. 2010. Estimating linear models for compositional distributional semantics. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’10, 1263–1271.Search in Google Scholar


Supplemental Material

The online version of this article offers supplementary material (https://doi.org/10.1515/cllt-2016-0038).


Published Online: 2017-1-24
Published in Print: 2017-9-26

© 2017 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 11.11.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cllt-2016-0038/pdf?lang=en
Scroll to top button