Construction grammar and procedural semantics for human-interpretable grounded language processing

Alomari, Muhannad, Fangjun Li, David C. Hogg & Anthony G. Cohn. 2022. Online perceptual learning and natural language acquisition for autonomous robots. Artificial Intelligence 303. 103637. https://doi.org/10.1016/j.artint.2021.103637.Search in Google Scholar

Andreas, Jacob, Marcus Rohrbach, Trevor Darrell & Dan Klein. 2016. Learning to compose neural networks for question answering. In Kevin Knight, Ani Nenkova & Owen Rambow (eds.), Proceedings of the 2016 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, 1545–1554. San Diego, CA: Association for Computational Linguistics.10.18653/v1/N16-1181Search in Google Scholar

Beuls, Katrien & Paul Van Eecke. 2023. Fluid construction grammar: State of the art and future outlook. In Claire Bonial & Harish Tayyar Madabushi (eds.), Proceedings of the first International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), 41–50. Washington, D.C.: Association for Computational Linguistics.Search in Google Scholar

Beuls, Katrien & Paul Van Eecke. 2024. Construction grammar and artificial intelligence. In Mirjam Fried & Kiki Nikiforidou (eds.), The Cambridge handbook of construction grammar. Forthcoming. Cambridge, United Kingdom: Cambridge University Press.Search in Google Scholar

Beuls, Katrien, Paul Van Eecke & Vanja Sophie Cangalovic. 2021. A computational construction grammar approach to semantic frame extraction. Linguistics Vanguard 7(1). 20180015. https://doi.org/10.1515/lingvan-2018-0015.Search in Google Scholar

Bleys, Joris. 2016. Language strategies for the domain of colour. Berlin: Language Science Press.10.26530/OAPEN_603341Search in Google Scholar

Chen, Kan, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu & Nevatia Ram. 2015. Abc-cnn: An attention based convolutional neural network for visual question answering. arXiv preprint arXiv:1511.05960. https://doi.org/10.48550/arXiv.1511.05960.Search in Google Scholar

Cirik, Volkan, Taylor Berg-Kirkpatrick & Louis-Philippe Morency. 2018. Using syntax to ground referring expressions in natural images. In Sheila McIlraith & Kilian Q. Weinberger (eds.), Proceedings of the thirty-second AAAI Conference on Artificial Intelligence, 6756–6764. Washington, D.C.: AAAI Press.10.1609/aaai.v32i1.12343Search in Google Scholar

Das, Abhishek, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José M. F. Moura, Devi Parikh & Dhruv Batra. 2017. Visual dialog. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1080–1089. Washington, D.C.: IEEE Computer Society.10.1109/CVPR.2017.121Search in Google Scholar

Doumen, Jonas, Katrien Beuls & Paul Van Eecke. 2023. Modelling language acquisition through syntactico-semantic pattern finding. In Andreas Vlachos & Isabelle Augenstein (eds.), Findings of the association for computational linguistics: EACL 2023, 1317–1327. Dubrovnik: Association for Computational Linguistics.10.18653/v1/2023.findings-eacl.99Search in Google Scholar

Frank, Anette, Hans-Ulrich Krieger, Feiyu Xu, Hans Uszkoreit, Berthold Crysmann, Brigitte Jörg & Schäfer Ulrich. 2007. Question answering from structured knowledge sources. Journal of Applied Logic 5(1). 20–48. https://doi.org/10.1016/j.jal.2005.12.006.Search in Google Scholar

Garcez, Artur d’Avila, Tarek R. Besold, Luc De Raedt, Földiak Peter, Pascal Hitzler, Thomas Icard, Kai-Uwe Kühnberger, Luis C. Lamb, Risto Miikkulainen & Daniel L. Silver. 2015. Neural-symbolic learning and reasoning: Contributions and challenges. In 2015 AAAI Spring symposium series, 18–21. Washington, D.C.: AAAI Press.Search in Google Scholar

Guo, Chuan, Geoff Pleiss, Yu Sun & Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Doina Precup & Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning (ICML), 1321–1330. Sydney: JMLR.org.Search in Google Scholar

Hu, Ronghang, Andreas Jacob, Trevor Darrell & Kate Saenko. 2018. Explainable neural computation via stack neural module networks. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu & Yair Weiss (eds.), European conference on computer vision (eccv 2018), 53–69. Cham: Springer.10.1007/978-3-030-01234-2_4Search in Google Scholar

Hu, Ronghang, Jacob Andreas, Marcus Rohrbach, Trevor Darrell & Kate Saenko. 2017. Learning to reason: End-to-end module networks for visual question answering. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 804–813. Washington, D.C.: IEEE Computer Society.10.1109/ICCV.2017.93Search in Google Scholar

Hudson, Drew A. & Christopher D. Manning. 2018. Compositional attention networks for machine reasoning. In 6th International Conference on Learning Representations (ICLR 2018), 1–20. Vancouver.Search in Google Scholar

Jang, Yunseok, Yale Song, Youngjae Yu, Youngjin Kim & Gunhee Kim. 2017. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2758–2766. Washington, D.C.: IEEE Computer Society.10.1109/CVPR.2017.149Search in Google Scholar

Johnson, Justin, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick & Ross Girshick. 2017a. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2901–2910. Washington, D.C.: IEEE Computer Society.10.1109/CVPR.2017.215Search in Google Scholar

Johnson, Justin, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick & Ross Girshick. 2017b. Inferring and executing programs for visual reasoning. In Rita Cucchiara, Yasuyuki Matsushita, Nicu Sebe & Stefano Soatto (eds.), 2017 IEEE International Conference on Computer Vision (ICCV), 2989–2998. Washington, D.C.: IEEE Computer Society.10.1109/ICCV.2017.325Search in Google Scholar

Kazemzadeh, Sahar, Vicente Ordonez, Mark Matten & Tamara Berg. 2014. Referitgame: Referring to objects in photographs of natural scenes. In Alessandro Moschitti, Bo Pang & Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 787–798. Doha: Association for Computational Linguistics.10.3115/v1/D14-1086Search in Google Scholar

Liang, Percy. 2016. Learning executable semantic parsers for natural language understanding. Communications of the ACM 59(9). 68–76. https://doi.org/10.1145/2866568.Search in Google Scholar

Loetzsch, Martin. 2015. Lexicon formation in autonomous robots. Berlin: Humboldt-Universität zu Berlin dissertation.Search in Google Scholar

Lu, Jiasen, Jianwei Yang, Dhruv Batra & Devi Parikh. 2016. Hierarchical question-image co-attention for visual question answering. In Daniel Lee, Masashi Sugiyama, Ulrike Von Luxburg, Isabelle Guyon & Roman Garnett (eds.), Advances in neural information processing systems 29 (NIPS 2016), 289–297. Red Hook, NY: Curran Associates.Search in Google Scholar

Manhaeve, Robin, Sebastijan Dumančić, Angelika Kimmig, Thomas Demeester & Luc De Raedt. 2021. Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence 298. 103504. https://doi.org/10.1016/j.artint.2021.103504.Search in Google Scholar

Mao, Jiayuan, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum & Jiajun Wu. 2019. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In 7th International Conference on Learning Representations (ICLR 2019). New Orleans, LA.Search in Google Scholar

Marcus, Gary. 2018. Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631. https://doi.org/10.48550/arXiv.1801.00631.Search in Google Scholar

Marques, Tânia & Katrien Beuls. 2016. Evaluation strategies for computational construction grammars. In Yuji Matsumoto & Rashmi Prasad (eds.), Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical papers, 1137–1146. Osaka, Japan: International Committee on Computational Linguistics.Search in Google Scholar

Massiceti, Daniela, Puneet K. Dokania, Narayanaswamy Siddharth & Philip Torr. 2018. Visual dialogue without vision or dialogue. Critiquing and correcting trends in machine learning workshop: NeurIPS 2018. Montreal, Canada.Search in Google Scholar

McFetridge, Paul, Fred Popowich & Dan Fass. 1996. An analysis of compounds in HPSG (Head-driven Phrase Structure Grammar) for database queries. Data & Knowledge Engineering 20(2). 195–209. https://doi.org/10.1016/s0169-023x(96)00033-x.Search in Google Scholar

Mitchell, Melanie. 2020. On crashing the barrier of meaning in artificial intelligence. AI Magazine 41(2). 86–92. https://doi.org/10.1609/aimag.v41i2.5259.Search in Google Scholar

Mitchell, Melanie. 2021. Abstraction and analogy-making in artificial intelligence. Annals of the New York Academy of Sciences 1505(1). 79–101. https://doi.org/10.1111/nyas.14619.Search in Google Scholar

Mooney, Raymond J. 2008. Learning to connect language and perception. In Dieter Fox & Carla Gomes (eds.), Proceedings of the twenty-third AAAI conference on artificial intelligence, 1598–1601. Washington, D.C.: AAAI Press.Search in Google Scholar

Nevens, Jens. 2022. Representing and learning linguistic structures on the conceptual, morphosyntactic, and semantic level. Brussels: Vrije Universiteit Brussel dissertation.Search in Google Scholar

Nevens, Jens, Jonas Doumen, Paul Van Eecke & Katrien Beuls. 2022. Language acquisition through intention reading and pattern finding. In Nicoletta Calzolari & Chu-Ren Huang (eds.), Proceedings of the 29th International Conference on Computational Linguistics, 15–25. Gyeongju, Republic of Korea: International Committee on Computational Linguistics.Search in Google Scholar

Nevens, Jens, Paul Van Eecke & Katrien Beuls. 2019a. A practical guide to studying emergent communication through grounded language games. In AISB 2019 Symposium on Language Learning for Artificial Agents, 1–8. Falmouth: AISB.Search in Google Scholar

Nevens, Jens, Paul Van Eecke & Katrien Beuls. 2019b. Computational construction grammar for visual question answering. Linguistics Vanguard 5(1). 20180070. https://doi.org/10.1515/lingvan-2018-0070.Search in Google Scholar

Nevens, Jens, Paul Van Eecke & Katrien Beuls. 2020. From continuous observations to symbolic concepts: A discrimination-based strategy for grounded concept learning. Frontiers in Robotics and AI 7(84). https://doi.org/10.3389/frobt.2020.00084.Search in Google Scholar

Persson, Andreas, Pedro Miguel Zuidberg Dos Martires, Luc De Raedt & Loutfi Amy. 2019. Semantic relational object tracking. IEEE Transactions on Cognitive and Developmental Systems 12(1). 84–97. https://doi.org/10.1109/tcds.2019.2915763.Search in Google Scholar

Spranger, Michael, Simon Pauw & Martin Loetzsch. 2010. Open-ended semantics co-evolving with spatial language. In Erica A. Cartmill, Sean Roberts, Heidi Lyn & Hannah Cornish (eds.), Proceedings of the 10th international conference (EVOLANGX), 297–304. Singapore: World Scientific.10.1142/9789814295222_0038Search in Google Scholar

Steels, Luc. 2001. Language games for autonomous robots. IEEE Intelligent Systems 16. 16–22. https://doi.org/10.1109/mis.2001.956077.Search in Google Scholar

Steels, Luc. 2012. Grounding language through evolutionary language games. In Luc Steels & Manfred Hild (eds.), Language grounding in robots, 1–22. New York, NY: Springer.10.1007/978-1-4614-3064-3_1Search in Google Scholar

Steels, Luc & Tony Belpaeme. 2005. Coordinating perceptually grounded categories through language: A case study for colour. Behavioral and Brain Sciences 28(4). 469–489. https://doi.org/10.1017/S0140525X05000087.Search in Google Scholar

Steels, Luc, Martin Loetzsch & Michael Spranger. 2016. A boy named Sue: The semiotic dynamics of naming and identity. Belgian Journal of Linguistics 30(1). 147–169. https://doi.org/10.1075/bjl.30.07ste.Search in Google Scholar

Thulasidasan, Sunil, Gopinath Chennupati, Jeff A. Bilmes, Tanmoy Bhattacharya & Sarah Michalak. 2019. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Hanna Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily Fox & Roman Garnett (eds.), Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 13843–13854. Red Hook, NY, USA: Curran Associates.10.2172/1525811Search in Google Scholar

Van den Broeck, Wouter. 2008. Constraint based compositional semantics. In Andrew D. M. Smith, Kenny Smith & Ramon Ferrer i Cancho (eds.), Proceedings of the 7th International Conference on the Evolution of Language (EVOLANG7), 338–345. World Scientific.10.1142/9789812776129_0043Search in Google Scholar

Van Eecke, Paul. 2018. Generalisation and specialisation operators for computational construction grammar and their application in evolutionary linguistics research. Brussels: Vrije Universiteit Brussel dissertation.Search in Google Scholar

Van Eecke, Paul & Katrien Beuls. 2017. Meta-layer problem solving for computational construction grammar. In The 2017 AAAI Spring symposium series, 258–265. Washington, D.C.: AAAI Press.Search in Google Scholar

Van Eecke, Paul, Jens Nevens & Katrien Beuls. 2022. Neural heuristics for scaling constructional language processing. Journal of Language Modelling 10(2). 287–314. https://doi.org/10.15398/jlm.v10i2.318.Search in Google Scholar

van Trijp, Remi, Katrien Beuls & Paul Van Eecke. 2022. The FCG Editor: An innovative environment for engineering computational construction grammars. PLoS One 17(6). e0269708. https://doi.org/10.1371/journal.pone.0269708.Search in Google Scholar

Wellens, Pieter. 2012. Adaptive strategies in the emergence of lexical systems. Brussels: Vrije Universiteit Brussel dissertation.Search in Google Scholar

Winograd, Terry. 1972. Understanding natural language. Cognitive Psychology 3(1). 1–191. https://doi.org/10.1016/0010-0285(72)90002-3.Search in Google Scholar

Yarmohammadi, Mahsa A., Mehrnoush Shamsfard, Mahshid A. Yarmohammadi & Masoud Rouhizadeh. 2008. SBUQA question answering system. In Advances in computer science and engineering: Csicc 2008, 316–323. Berlin: Springer.10.1007/978-3-540-89985-3_39Search in Google Scholar

Yi, Kexin, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli & Josh Tenenbaum. 2018. Neural-symbolic VQA: Disentangling reasoning from vision and language understanding. In Samy Bengio, Hanna Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi & Roman Garnett (eds.), Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 1031–1042. Red Hook, NY, USA: Curran Associates.Search in Google Scholar

Yu, Zhou, Jun Yu, Yuhao Cui, Dacheng Tao & Tian Qi. 2019. Deep modular co-attention networks for visual question answering. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6274–6283. Washington, D.C.: IEEE Computer Society.10.1109/CVPR.2019.00644Search in Google Scholar

Zettlemoyer, Luke & Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Fahiem Bacchus & Tommi Jaakkola (eds.), Proceedings of the twenty-first Conference on Uncertainty in Artificial Intelligence, 658–666. Edinburgh: AUAI Press.Search in Google Scholar