When do language comprehenders mentally simulate locations?
-
Nian Liu
and
Benjamin Bergen
Abstract
Embodied approaches to comprehension propose that understanding language entails performing mental simulations of its content. The evidence, however, is mixed. Action-sentence Compatibility Effect studies (Glenberg and Kaschak 2002) report mental simulation of motor actions during processing of motion language. But the same studies find no evidence that language comprehenders perform spatial simulations of the corresponding locations. This challenges simulation-based approaches. If locations are not represented in simulation, but are still understood, then simulation may be unnecessary for understanding. We conducted a Location-sentence Compatibility experiment, to determine whether understanders mentally simulate locations. People did indeed simulate locations, but only when sentences used progressive (and not perfect) grammatical aspect. Moreover, mental simulations of locations differed for language about concrete versus abstract events. These findings substantiate the role of mental simulation in language understanding, while highlighting the importance of the grammatical form of utterances as well as their content.
References
Anderson, Sarah, Teenie Matlock & Michael J. Spivey. 2010. The role of grammatical aspect in the dynamics of spatial descriptions. In Cristoph Hölscher, Thomas F. Shipley, Marta Olivetti Belardinelli, John A. Bateman & Nora S. Newcombe (eds.), Spatial cognition, Vol. 7 (Lecture Notes in Artificial Intelligence 6222), 139–151. Berlin: Springer-Verlag.10.1007/978-3-642-14749-4_14Search in Google Scholar
Barsalou, Lawrence W. 1999. Perceptual symbol systems. Behavioral and Brain Sciences 22. 577–609.10.1017/S0140525X99002149Search in Google Scholar
Barsalou, Lawrence W. & Katja Wiemer-Hastings. 2005. Situating abstract concepts. In Diane Pecher & Rolf A. Zwaan (eds.), Grounding cognition: The role of perception and action in memory, language, and thought, 129–163. New York: Cambridge University Press.10.1017/CBO9780511499968.007Search in Google Scholar
Bergen, Benjamin K. 2007. Experimental methods for simulation semantics. In Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson & Michael J. Spivey (eds.), Methods in cognitive linguistics, 277–301. Amsterdam & Philadelphia: John Benjamins.10.1075/hcp.18.19berSearch in Google Scholar
Bergen, Benjamin K. & Nancy Chang. 2005. Embodied construction grammar in simulation-based language understanding. In Jan-Ola Östman & Mirjam Fried (eds.), Construction grammar(s): Cognitive grounding and theoretical extensions, 147–190. Amsterdam: John Benjamins.10.1075/cal.3.08berSearch in Google Scholar
Bergen, Benjamin K., Shane Lindsay, Teenie Matlock & Srini Narayanan. 2007. Spatial and linguistic aspects of visual imagery in sentence comprehension. Cognitive Science 31, 733–764.10.1080/03640210701530748Search in Google Scholar
Bergen, Benjamin K., Shweta Narayan & Jerome Feldman. 2003. Embodied verbal semantics: Evidence from an image-verb matching task. In Richard Alterman & David Kirsh (eds.), Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society, 139–144. Boston, MA: Cognitive Science Society.Search in Google Scholar
Bergen, Benjamin K. & Kathryn B. Wheeler. 2005. Sentence understanding engages motor processes. In Bruno G. Bara, Lawrence W. Barsalou & Monica Bucciarelli (eds.), Proceedings of the Twenty-Seventh Annual Meeting of the Cognitive Science Society, 238–243. New York: Lawrence Erlbaum.Search in Google Scholar
Bergen, Benjamin K. & Kathryn B. Wheeler. 2010. Grammatical aspect and mental simulation. Brain and Language 112. 150–158.10.1016/j.bandl.2009.07.002Search in Google Scholar
Brunyé, Tad T., Tali Ditman, Caroline R. Mahoney, Jason S. Augustyn & Holly A. Taylor. 2009. When you and I share perspectives: Pronouns and perspective-taking during narrative comprehension. Psychological Science 20(1). 27–32.10.1111/j.1467-9280.2008.02249.xSearch in Google Scholar
Bub, Daniel N., Michael E. J. Masson & George S. Cree. 2008. Evocation of functional and volumetric gestural knowledge by objects and words. Cognition 106. 27–58.10.1016/j.cognition.2006.12.010Search in Google Scholar
Carreiras, Manuel, Núria Carriedo, María Angeles Alonso & Angel Fernández. 1997. The role of verb tense and verb aspect in the foregrounding of information during reading. Memory and Cognition 25. 438–446.10.3758/BF03201120Search in Google Scholar
Chomsky, Noam. 1957. Syntactic structures. The Hague & Paris: Mouton.10.1515/9783112316009Search in Google Scholar
Comrie, Bernard. 1976. Aspect. Cambridge: Cambridge University Press.Search in Google Scholar
Connell, Louise. 2007. Representing object color in language comprehension. Cognition 102. 476–485.10.1016/j.cognition.2006.02.009Search in Google Scholar
Dowty, David R. 1977. Toward a semantic analysis of verb aspect and the English “imperfective” progressive. Linguistics and Philosophy 1. 45–77.10.1002/9780470758335.ch11Search in Google Scholar
Feldman, Jerome & Srinivas Narayanan. 2004. Embodied meaning in a neural theory of language. Brain and Language 89. 385–392.10.1016/S0093-934X(03)00355-9Search in Google Scholar
Ferretti, Todd R., Marta Kutas & Ken McRae. 2007. Verb aspect and the activation of event knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition 33(1). 182–196.10.1037/0278-7393.33.1.182Search in Google Scholar
Fodor, Jerry A. 1983. The modularity of mind: An essay on faculty psychology. Cambridge, MA: MIT Press.10.7551/mitpress/4737.001.0001Search in Google Scholar
Gallese, Vittorio & George Lakoff. 2005. The brain’s concepts: The role of the sensory-motor system in reason and language. Cognitive Neuropsychology 22. 455–479.10.1080/02643290442000310Search in Google Scholar
Giora, Rachel, Noga Balaban, Ofer Fein & Inbar Alkabets. 2004. Negation as positivity in disguise. In Herbert L. Colston & Albert Katz (eds.), Figurative language comprehension: Social and cultural influences, 233–258. Hillsdale, NJ: Erlbaum.Search in Google Scholar
Glenberg, Arthur M., Raymond Becker, Susann Klötzer, Lidia Kolanko, Silvana Müller & Mike Rinck. 2009. Episodic affordances contribute to language comprehension. Language and Cognition 1. 113–135.10.1037/e537052012-249Search in Google Scholar
Glenberg, Arthur & Michael Kaschak. 2002. Grounding language in action. Psychonomic Bulletin and Review 9. 558–565.10.3758/BF03196313Search in Google Scholar
Goldberg, Adele E. 1995. Constructions: A construction grammar approach to argument structure. Chicago: University of Chicago Press.Search in Google Scholar
Guan, Connie Qun, Wanjin Meng, Ru Yao & Arthur M. Glenberg. 2013. The motor system contributes to comprehension of abstract language. PloS one 8(9). e75183. DOI 10.1371/journal.pone.0075183.10.1371/journal.pone.0075183Search in Google Scholar
Kaschak, Michael P., Carol J. Madden, David J. Therriault, Richard H. Yaxley, Mark Aveyard, Adrienne A. Blanchard & Rolf A. Zwaan. 2005. Perception of motion affects language processing. Cognition 94. B79–B89.10.1037/e537052012-803Search in Google Scholar
Kaup, Barbara, Jana Lüdtke & Rolf A. Zwaan. 2007. The experiential view of language comprehension: How is negated information represented? In Franz Schmalhofer & Charles A. Perfetti (eds.), Higher level language processes in the brain: Inference and comprehension processes, 255–288. Mahwah, NJ: Erlbaum.Search in Google Scholar
Kosslyn, Stephen M., Giorgio Ganis & William L. Thompson. 2001. Neural foundations of imagery. Nature Reviews Neuroscience 2. 635–642.10.1038/35090055Search in Google Scholar
Lakoff, George & Mark Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press.Search in Google Scholar
Langacker, Ronald W. 1983. Remarks on English aspect. In Paul J. Hopper (ed.), Tense and aspect: Between semantics and pragmatics, 265–304. Amsterdam: John Benjamins.10.1075/tsl.1.17lanSearch in Google Scholar
Madden, Carol J. & David J. Therriault. 2009. Verb aspect and perceptual simulations. The Quarterly Journal of Experimental Psychology 62(7). 1294–1302.10.1080/17470210802696088Search in Google Scholar
Madden, Carol J. & Rolf A. Zwaan. 2003. How does verb aspect constrain event representations? Memory and Cognition 31. 663–672.10.3758/BF03196106Search in Google Scholar
Magliano, Joseph P. & Michelle C. Schleich. 2000. Verb aspect and situation models. Discourse Processes 29. 83–112.10.1207/S15326950dp2902_1Search in Google Scholar
Matlock, Teenie. 2011. The conceptual motivation of aspect. In Klaus-Uwe Panther & Günther Radden (eds.), Motivation in grammar and the lexicon, 133–147. Amsterdam & Philadelphia: John Benjamins.10.1075/hcp.27.09matSearch in Google Scholar
Pecher, Diane, René Zeelenberg & Lawrence W. Barsalou. 2003. Verifying properties from different modalities for concepts produces switching costs. Psychological Science 14. 119–124.10.1111/1467-9280.t01-1-01429Search in Google Scholar
Richardson, Daniel C. & Teenie Matlock. 2007. The integration of figurative language and static depictions: An eye movement study of fictive motion. Cognition 102. 129–138.10.1016/j.cognition.2005.12.004Search in Google Scholar
Richardson, Daniel C., Michael J. Spivey, Lawrence W. Barsalou & Ken McRae. 2003. Spatial representations activated during real-time comprehension of verbs. Cognitive Science 27. 767–780.10.1207/s15516709cog2705_4Search in Google Scholar
Solomon, Karen O. & Lawrence W. Barsalou. 2004. Perceptual simulation in property verification. Memory and Cognition 32. 244–259.10.3758/BF03196856Search in Google Scholar
Stanfield, Robert A. & Rolf A. Zwaan. 2001. The effect of implied orientation derived from verbal context on picture recognition. Psychological Science 12. 153–156.10.1111/1467-9280.00326Search in Google Scholar
Taylor, Lawrence J. & Rolf A. Zwaan. 2008. Motor resonance and linguistic focus. The Quarterly Journal of Experimental Psychology 61. 869–904.10.1080/17470210701625519Search in Google Scholar
Wheeler, Mark E., Steven E. Petersen & Randy L. Buckner. 2000. Memory’s echo: Vivid remembering reactivates sensory-specific cortex. Proceedings of the National Academy of Sciences USA 97. 11125–11129.10.1073/pnas.97.20.11125Search in Google Scholar
Zacks, Jeff M. & Barbara Tversky. 2001. Event structure in perception and conception. Psychological Bulletin 127(1). 3–21.10.1037/0033-2909.127.1.3Search in Google Scholar
Zwaan, Rolf A. 1999. Embodied cognition, perceptual symbols, and situation models. Discourse Processes 28. 81–88.10.1080/01638539909545070Search in Google Scholar
Zwaan, Rolf A., Robert A. Stanfield & Richard H. Yaxley. 2002. Language comprehenders mentally represent the shapes of objects. Psychological Science 13. 168–171.10.1111/1467-9280.00430Search in Google Scholar
Appendix
Critical stimuli. Only the progressive versions are shown below. Perfect versions were identical except for aspect marking. The numbers are the average and standard deviation of each sentence’s sensibility score.
| Mean | SD | Mean | SD | ||
| Noun-determined sentences | 6.5 | 0.9 | |||
| AWAY | 6.3 | 1.2 | TOWARDS | 6.7 | 0.7 |
| Shirley is brushing the couch. | 5.0 | 2.0 | Brian is pinching his chin. | 6.2 | 1.6 |
| Mildred is squeezing the mustard bottle. | 6.7 | 0.9 | Willie is lighting his cigarette. | 7.0 | 0.0 |
| Ben is feeding his child. | 6.8 | 0.6 | Kelly is scratching her head. | 7.0 | 0.0 |
| Melissa is grabbing the doorknob. | 6.2 | 1.4 | Jonathan is tucking in his shirt. | 7.0 | 0.2 |
| Chris is patting the cat. | 5.7 | 1.8 | Fred is putting in his contact lens. | 6.5 | 1.4 |
| Mary is rubbing the magic lamp. | 6.8 | 0.7 | Joan is washing her face. | 6.9 | 0.3 |
| Helen is wiping the counter. | 6.8 | 1.0 | Louis is grabbing his nose. | 6.7 | 0.9 |
| Terry is pushing the elevator button. | 7.0 | 0.0 | Lisa is adjusting her glasses. | 6.0 | 1.8 |
| Pamela is beating the drum. | 6.6 | 1.1 | Virginia is brushing her teeth. | 7.0 | 0.2 |
| Eric is washing his desk. | 5.5 | 2.0 | Jean is cleaning her ear. | 6.4 | 1.0 |
| Mean | SD | Mean | SD | ||
| Verb-determined sentences | 6.3 | 1.3 | |||
| AWAY | 6.3 | 1.4 | TOWARDS | 6.3 | 1.2 |
| Judith is closing the cupboard. | 6.3 | 1.5 | Cheryl is pulling the door. | 5.6 | 1.9 |
| Bruce is tossing out the water. | 5.9 | 1.8 | Dennis is picking up the toys. | 6.7 | 0.9 |
| Beverly is closing the drawer. | 6.5 | 1.3 | James is eating the pie. | 6.4 | 1.2 |
| Ashley is stretching her arms. | 6.6 | 1.1 | Stephen is dragging in a fish. | 5.0 | 1.8 |
| Maria is spitting out the water. | 6.2 | 1.6 | Janice is snatching the ring. | 5.8 | 1.8 |
| Joshua is tossing a Q-tip. | 5.8 | 1.9 | Donald is biting his fingernails. | 6.8 | 0.9 |
| Kimberly is hanging up the phone. | 6.6 | 1.1 | Stephanie is rubbing her belly. | 6.9 | 0.4 |
| George is taking off the jacket. | 5.5 | 1.9 | Harry is smoking a cigarette. | 7.0 | 0.0 |
| Carol is taking off her glasses. | 6.5 | 1.4 | Edward is putting in the earplugs. | 6.3 | 1.3 |
| Carl is flipping the burger. | 6.7 | 0.7 | Joyce is stealing a marshmallow. | 6.3 | 1.4 |
| Mean | SD | Mean | SD | ||
| PP-determined sentences | 6.2 | 1.5 | |||
| AWAY | 6.4 | 1.3 | TOWARDS | 6.0 | 1.6 |
| Andrew is dumping the coffee into the sink. | 6.4 | 1.8 | Nancy is tossing the cracker past her lips. | 5.9 | 2.4 |
| Rose is putting the ear-plugs on the table. | 6.9 | 1.4 | Patrick is putting a tissue to his nose. | 6.5 | 1.7 |
| Christina is pouring the water into the sink. | 6.5 | 0.3 | Nicole is spreading the lotion on her back. | 6.7 | 1.2 |
| Sharon is putting the pencil in the pencil sharpener. | 6.6 | 1.1 | Walter is putting money in his pocket. | 6.8 | 0.5 |
| Jeffrey is throwing the pills onto the floor. | 4.2 | 1.1 | Jessica is shoving her finger into her ear. | 6.4 | 1.1 |
| Sandra is running her hands through the dog’s hair. | 6.4 | 1.3 | Adam is placing a dime on his shoulder. | 5.7 | 2.2 |
| Ruth is squeezing the drops into the bowl. | 5.9 | 1.7 | Debra is putting a grape in her mouth. | 6.6 | 1.2 |
| Mark is slapping the sticker on the refrigerator. | 6.2 | 1.8 | Jose is sticking tape on his nose. | 6.0 | 1.8 |
| Samuel is putting a ring in the jewelry box. | 6.5 | 1.0 | Kenneth is driving his knuckles into his ribs. | 5.4 | 2.3 |
| Charles is wiping the sweat off the bench. | 6.6 | 0.9 | Jane is putting her finger under her nose. | 6.0 | 1.6 |
| Mean | SD | Mean | SD | ||
| Abstract sentences | 5.9 | 1.7 | |||
| AWAY | 6.4 | 1.2 | TOWARDS | 5.3 | 2.1 |
| Darlene is transmitting the orders to the front lines. | 6.6 | 1.0 | Bill is tearing his heart out of the relationship. | 4.0 | 2.4 |
| Bertha is posting her wedding date to the newsgroup. | 6.1 | 1.6 | Oscar is receiving the message from headquarters. | 6.4 | 1.5 |
| Lloyd is donating a kidney to the biology department. | 6.7 | 1.0 | Michele is withdrawing her proposal from the running. | 5.4 | 2.1 |
| Dan is confessing his secret to the courtroom. | 6.5 | 1.0 | Jill is withdrawing her time from charity. | 4.9 | 2.0 |
| Andy is pitching the idea to the publishing firm. | 6.9 | 0.4 | Jane is collecting praise from the children. | 4.9 | 2.3 |
| Alicia is transferring responsibility to a law firm. | 5.8 | 2.0 | Jim is receiving the honor from the teacher. | 5.4 | 2.1 |
| Jeff is encoding the information on a computer disk. | 6.4 | 1.4 | Megan is removing her true name from her diary. | 5.9 | 1.7 |
| Calvin is submitting the request to the committee. | 6.7 | 1.0 | Juan is extracting state secrets from the enemy. | 5.8 | 1.9 |
| Bonnie is returning a sense of decorum to the proceedings. | 5.7 | 2.0 | Darlene is taking the idea away from the conversation. | 4.6 | 2.5 |
| Ronnie is selling the land to a corporation. | 6.8 | 0.6 | Tom is stealing the match from his opponent. | 5.1 | 2.3 |
Sample filler sentences. Only the progressive versions are shown below. Perfect versions were identical except for aspect marking.
| Louise is stretching the apple. | Dawn is typing her dinner. |
| Vincent is blowing a lesson to Liz. | Nathan is opening the plate. |
| Crystal is scratching us a clock. | Sherry is mowing the drum. |
| Stanley is grabbing him to the vase. | Leonard is washing the air. |
| Jesse is teaching his time to Anna. | Grace is pouring the moon. |
| Diana is devoting the song Jenni. | Jeffery is fertilizing his clips. |
| Peggy is eating Sally the tea cup. | Emily is plugging the railing. |
| Allen is drinking the house to Joe. | Norman is turning on the candy. |
| Annie is pouring the horse to him. | Tiffany is bicycling the steel using the keyboard. |
| Jimmy is thinking him the ice cream. | Tracy is drinking the backpack throughout the calendar. |
©2016 by De Gruyter Mouton
Articles in the same Issue
- Frontmatter
- Research Articles
- Metaphor in culture: LIFE IS A SHOW in Chinese
- When do language comprehenders mentally simulate locations?
- With the future coming up behind them: Evidence that Time approaches from behind in Vietnamese
- When variables align: A Bayesian multinomial mixed-effects model of English permissive constructions
- The role of DO-auxiliary in subject-auxiliary inversion: Developing Langacker’s notion of existential negotiation
- Book Reviews
- Raffaele Simone and Francesca Masini: Word classes: Nature, typology and representations
- Michel Achard: Impersonals and Other Agent Defocusing Constructions in French
Articles in the same Issue
- Frontmatter
- Research Articles
- Metaphor in culture: LIFE IS A SHOW in Chinese
- When do language comprehenders mentally simulate locations?
- With the future coming up behind them: Evidence that Time approaches from behind in Vietnamese
- When variables align: A Bayesian multinomial mixed-effects model of English permissive constructions
- The role of DO-auxiliary in subject-auxiliary inversion: Developing Langacker’s notion of existential negotiation
- Book Reviews
- Raffaele Simone and Francesca Masini: Word classes: Nature, typology and representations
- Michel Achard: Impersonals and Other Agent Defocusing Constructions in French
