Incremental parsing in a continuous dynamical system: sentence processing in Gradient Symbolic Computation
-
Pyeong Whan Cho
,
Matthew Goldrick
Abstract
Any incremental parser must solve two computational problems: (1) maintaining all interpretations consistent with the words that have been processed so far and (2) excluding all globally-incoherent interpretations. While these problems are well understood, it is not clear how the dynamic, continuous mechanisms that underlie human language processing solve them. We introduce a Gradient Symbolic Computation (GSC) parser, a continuous-state, continuous-time stochastic dynamical-system model of symbolic processing, which builds up a discrete symbolic structure gradually by dynamically strengthening a discreteness constraint. Online, interactive tutorials with open-source software are presented in a companion website. Our results reveal that the GSC parser can solve the two computational problems by moving to a non-discrete blend state that evolves exclusively to discrete states representing contextually-appropriate globally-coherent interpretations. In a simulation study using a simple formal grammar, we show that successful parsing requires appropriate control of the discreteness constraint strength (a quantization policy). With inappropriate quantization policies, the GSC parser makes mistakes that mirror those made in natural language comprehension (garden-path or local-coherence errors). These findings suggest that the GSC model offers a neurally plausible solution to these two core problems.
Acknowledgement
We thank Géraldine Legendre, Akira Omaki, Kyle Rawlins, Ben Van Durme, and Colin Wilson for their contributions to this work, and gratefully acknowledge the support of NSF INSPIRE grant BCS-1344269. We thank Paul Tupper for suggesting the form of the HQ function used in this work.
References
Altmann, Gerry T. M & Yuki Kamide. 1999. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 73(3). 247–264.10.1016/S0010-0277(99)00059-1Suche in Google Scholar
beim Graben, Peter & Sabrina Gerth. 2012. Geometric representations for minimalist grammars. Journal of Logic, Language and Information 21. 393–432.10.1007/s10849-012-9164-2Suche in Google Scholar
beim Graben, Peter, Sabrina Gerth & Shravan Vasishth. 2008. Towards dynamical system models of language-related brain potentials. Cognitive Neurodynamics 2(3). 229–255.10.1007/s11571-008-9041-5Suche in Google Scholar
Bever, Thomas G. 1970. The cognitive basis for linguistic structures. In John R. Hayes (eds.), Cognition and the development of language, 279–362. New York: John Wiley.Suche in Google Scholar
Borges, Jorge Luis. 1962. The garden of forking paths (1941). In Anthony Kerrigan (eds.), Ficciones. New York, NY: Grove Press.Suche in Google Scholar
Carmantini, Giovanni S., Peter beim Graben, Mathieu Desroches & Serafim Rodrigues. 2017. A modular architecture for transparent computation in recurrent neural networks. Neural Networks 85. 85–105.10.1016/j.neunet.2016.09.001Suche in Google Scholar
Cho, Pyeong Whan & Paul Smolensky. 2016. Bifurcation analysis of a Gradient Symbolic Computation model of incremental processing. In A. Papafragou, D. Grodner, D. Mirman & J. C. Trueswell (eds.), Proceedings of the 38th annual conference of the Cognitive Science Society. Cognitive Science Society: Austin, TX.Suche in Google Scholar
Elman, Jeffrey L. 1990. Finding structure in time. Cognitive Science 14(2). 179–211.10.4324/9781315784779-11Suche in Google Scholar
Frazier, Lyn. 1987. Sentence processing: A tutorial review. In M. Coltheart (eds.), Attention and performance XII: The psychology of reading, 559–586. Lawrence Erlbaum Associates: Mahwah, NJ.Suche in Google Scholar
Geman, Stuart & Donald Geman. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6(6). 721–741.10.1016/B978-0-08-051581-6.50057-XSuche in Google Scholar
Gibson, Edward. 2006. The interaction of top–down and bottom–up statistics in the resolution of syntactic category ambiguity. Journal of Memory and Language 54(3). 363–388.10.1016/j.jml.2005.12.005Suche in Google Scholar
Hale, John. 2001. A probabilistic Earley parser as a psycholinguistic model. NAACL '01: Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies. 1–8.10.3115/1073336.1073357Suche in Google Scholar
Hale, John. 2011. What a rational parser would do. Cognitive Science 35(3). 399–443.10.1111/j.1551-6709.2010.01145.xSuche in Google Scholar
Hale, John & Paul Smolensky. 2006. Harmonic Grammars and harmonic parsers for formal languages. In Paul Smolensky & Géraldine Legendre (eds.), The harmonic mind: From neural computation to optimality-theoretic grammar. Volume I: Cognitive architecture, 393–416. The MIT Press: Cambridge, MA.Suche in Google Scholar
Hockett, Charles F. 1960. The origin of speech. Scientific American 203. 89–96.10.1038/scientificamerican0960-88Suche in Google Scholar
Jurafsky, Daniel. 1996. A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science 20(2). 137–194.10.1207/s15516709cog2002_1Suche in Google Scholar
Konieczny, Lars. 2005. The psychological reality of local coherences in sentence processing. Proceedings of the 27th annual conference of the cognitive science society . 1178–1183.Suche in Google Scholar
Levy, Roger. 2008. Expectation-based syntactic comprehension. Cognition 106(3). 1126–1177.10.1016/j.cognition.2007.05.006Suche in Google Scholar
Levy, Roger, Florencia Reali & Thomas L. Griffiths. 2008. Modeling the effects of memory on human online sentence processing with particle filters. In D. Koller, D. Schuurmans, Y. Bengio & L. Bottou (eds.), Proceedings of the 21st international conference on Neural Information Processing Systems, 937–944. Red Hook, NY, USA: Curran Associates Inc.Suche in Google Scholar
Levy, Roger, Klinton Bicknell, Tim Slattery & Keith Rayner. 2009. Eye movement evidence that readers maintain and act on uncertainty about past linguistic input. Proceedings of the National Academy of Sciences 106(50). 21086–21090.10.1073/pnas.0907664106Suche in Google Scholar
Lewis, Richard L., Shravan Vasishth & Julie A. van Dyke. 2006. Computational principles of working memory in sentence comprehension. Trends in Cognitive Sciences 10(10). 447–454.10.1016/j.tics.2006.08.007Suche in Google Scholar
Marr, David. 1982. Vision: A computational investigation into the human representation and processing of visual information. W.H. Freeman: San Francisco.Suche in Google Scholar
Smolensky, Paul. 1990. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46(1). 159–216.10.1016/0004-3702(90)90007-MSuche in Google Scholar
Smolensky, Paul & Géraldine Legendre (eds.). 2006. The harmonic mind: From neural computation to optimality-theoretic grammar. Volume I: Cognitive architecture. Cambridge, MA: The MIT Press.Suche in Google Scholar
Smolensky, Paul, Matthew Goldrick & Donald Mathis. 2014. Optimization and quantization in gradient symbol systems: a framework for integrating the continuous and the discrete in cognition. Cognitive Science 38(6). 1102–1138.10.1111/cogs.12047Suche in Google Scholar
Tabor, Whitney & Sean Hutchins. 2004. Evidence for self-organized sentence processing: Digging-in effects. Journal of Experimental Psychology: Learning, Memory, and Cognition 30(2). 431–450.10.1037/0278-7393.30.2.431Suche in Google Scholar
Tabor, Whitney, Cornell Juliano & Michael K. Tanenhaus. 1997. Parsing in a dynamical system: An attractor-based account of the interaction of lexical and structural constraints in sentence processing. Language and Cognitive Processes 12(2–3). 211–271.10.1080/016909697386853Suche in Google Scholar
Tabor, Whitney, Bruno Galantucci & Daniel Richardson. 2004. Effects of merely local syntactic coherence on sentence processing. Journal of Memory and Language 50(4). 355–370.10.1016/j.jml.2004.01.001Suche in Google Scholar
Vosse, Theo & Gerard Kempen. 2000. Syntactic structure assembly in human parsing: A computational model based on competitive inhibition and a lexicalist grammar. Cognition 75(2). 105–143.10.1016/S0010-0277(00)00063-9Suche in Google Scholar
© 2017 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Editorial
- Editorial
- Phonetics & Phonology
- Acoustic correlates of word stress: A cross-linguistic survey
- A two-decade-interval variation in vowel insertion after word-final English and French postvocalic plosives in Korean adaptation: A sociolinguistic account
- Methodological issues in the study of word stress correlates
- Morphology & Syntax
- From the past into the present: From case frames to semantic frames
- Valency and expectation in Bantu applicatives
- Semantics & Pragmatics
- Semantic values as latent parameters: Testing a fixed threshold hypothesis for cardinal readings of few & many
- The Role of Prosody in the Identification of Persian Sentence Types: Declarative or Wh-question?
- Language Acquisition & Language Learning
- Individual differences in second language speech perception across tasks and contrasts: The case of English vowel contrasts by Korean learners
- Language Documentation & Typology
- Topological Relations in Pohnpeian
- Psycholinguistics & Neurolinguistics
- Incremental parsing in a continuous dynamical system: sentence processing in Gradient Symbolic Computation
- Sociolinguistics & Anthropological Linguistics
- Frequency effects over the lifespan: a case study of Attenborough’s r’s
- Is like like like?: Evaluating the same variant across multiple variables
- The linguist’s Drosophila: Experiments in language change
- Seseo, ceceo, and distinción in Andalusian Spanish: Free variation or sociolinguistic variation?
Artikel in diesem Heft
- Editorial
- Editorial
- Phonetics & Phonology
- Acoustic correlates of word stress: A cross-linguistic survey
- A two-decade-interval variation in vowel insertion after word-final English and French postvocalic plosives in Korean adaptation: A sociolinguistic account
- Methodological issues in the study of word stress correlates
- Morphology & Syntax
- From the past into the present: From case frames to semantic frames
- Valency and expectation in Bantu applicatives
- Semantics & Pragmatics
- Semantic values as latent parameters: Testing a fixed threshold hypothesis for cardinal readings of few & many
- The Role of Prosody in the Identification of Persian Sentence Types: Declarative or Wh-question?
- Language Acquisition & Language Learning
- Individual differences in second language speech perception across tasks and contrasts: The case of English vowel contrasts by Korean learners
- Language Documentation & Typology
- Topological Relations in Pohnpeian
- Psycholinguistics & Neurolinguistics
- Incremental parsing in a continuous dynamical system: sentence processing in Gradient Symbolic Computation
- Sociolinguistics & Anthropological Linguistics
- Frequency effects over the lifespan: a case study of Attenborough’s r’s
- Is like like like?: Evaluating the same variant across multiple variables
- The linguist’s Drosophila: Experiments in language change
- Seseo, ceceo, and distinción in Andalusian Spanish: Free variation or sociolinguistic variation?
