Program of the workshop

Venue: ESSLLI 2022, Galway, Ireland
Date: 8–12 August, 2022

Monday, August 8th


09:00	Introduction to the workshop
09:15	Keynote talk by Aurelie Herbelot (University of Trento) Title: Can truth-theoretic knowledge be learned? Abstract: Large Neural Language Models have been criticised from various perspectives. One aspect of this critique concerns the inability of current Deep Learning paradigms to encode core semantic competences such as quantification or negation. On the other hand, logical approaches to meaning — which readily account for a wide range of semantic phenomena — lack a theory of acquisition, prompting questions about their suitability for machine learning. In this talk, I will argue it is both possible and desirable to build computational models that take meaning seriously and encode its truth-theoretical aspects. I will first show how a mapping can be automatically learned between corpus data and some underspecified set-theoretic representation of the world. Having identified the limits of this approach, I will then propose a more complete formalisation of set theory in terms of a vector space, amenable to computational treatment. Finally, I will show that such a formalisation can be automatically learned from small data, providing high levels of performance on core semantic tasks.

09:00

Introduction to the workshop

09:15

Keynote talk by Aurelie Herbelot (University of Trento)
Title: Can truth-theoretic knowledge be learned?
Abstract: Large Neural Language Models have been criticised from various perspectives. One aspect of this critique concerns the inability of current Deep Learning paradigms to encode core semantic competences such as quantification or negation. On the other hand, logical approaches to meaning — which readily account for a wide range of semantic phenomena — lack a theory of acquisition, prompting questions about their suitability for machine learning. In this talk, I will argue it is both possible and desirable to build computational models that take meaning seriously and encode its truth-theoretical aspects. I will first show how a mapping can be automatically learned between corpus data and some underspecified set-theoretic representation of the world. Having identified the limits of this approach, I will then propose a more complete formalisation of set theory in terms of a vector space, amenable to computational treatment. Finally, I will show that such a formalisation can be automatically learned from small data, providing high levels of performance on core semantic tasks.

Tuesday, August 9th


09:00	Monotonicity Reasoning as a Bridge Between Linguistic Theory and NLI Larry Moss abstract
09:30	Crowdsourcing for generating diachronic semantic annotations in presupposition triggers: a pilot on again Martin Kopf, Maryam Rajestari and Remus Gergel abstract
10:00	Probabilities in language and the world: Modeling syllogistic reasoning Luuk Suurmeijer, Hana Filip and Noortje J. Venhuizen abstract

Wednesday, August 10th


09:00	Interactive CCG Parsing with Incremental Trees Masaya Taniguchi, Satoshi Tojo and Koji Mineshima abstract
09:30	Pragmatic Structuring and Negation in Formal Grammar Kata Balogh abstract
10:00	Unitary Matrices As Compositional Word Embeddings Jean-Philippe Bernardy and Shalom Lappin abstract

Thursday, August 11th


09:00	Do contextual word embeddings represent richly subsective adjectives more diversely than intersective adjectives? Michael Goodale and Salvador Mascarenhas abstract
09:30	A Tool for Easily Integrating Grammars as Language Models into the Kaldi Speech Recognition Toolkit Lucía Ormaechea Grijalba, Benjamin Lecouteux, Pierrette Bouillon and Didier Schwab abstract
10:00	Automatic Detection of Copredication using Contextualized Word Embeddings Deniz Ekin Yavas, Marta Ricchiardi, Elisabetta Ježek, Laura Kallmeyer and Rainer Osswald abstract

Friday, August 12th


09:00	Keynote talk by James Pustejovsky (Brandeis University) Title: Dense Paraphrasing for Textual Enrichment: Question Answering and Inference Abstract: Much of the current computational work on inference in NLP can be associated with one of two techniques: the first focuses on a specific notion of text-based question answering (QA), using large pre-trained language models (LLMs). To examine specific linguistic properties present in the model, « probing tasks » (diagnostic classifiers) have been developed to test capabilities that the LLM demonstrates on interpretable semantic inferencing tasks, such as age and object comparisons, hypernym conjunction, antonym negation, and others. The second is Knowledge Graph-based inference and QA, where triples are mined from Wikipedia, ConceptNet, WikiData, and other non-corpus resources, and then used for answering questions involving multiple components of the KG (multi-hop QA). While quite impressive with benchmarked metrics in QA, both techniques are completely confused by (a) syntactically missing semantic content, and (b) the semantics accompanying the consequences of events and actions in narratives. In this talk, I discuss a model we have developed to enrich the surface form of texts, using type-based semantic operations to « textually expose » the deeper meaning of the corpus that was used to make the original embeddings in the language model. This model, Dense Paraphrasing, is a linguistically-motivated, textual enrichment strategy, that textualizes the compositional operations inherent in a semantic model, such as Generative Lexicon Theory or CCG. This involve broadly three kinds of interpretive processes: (i) recognizing the diverse variability in linguistic forms that can be associated with the same underlying semantic representation (paraphrases); (ii) identifying semantic factors or variables that accompany or are presupposed by the lexical semantics of the words present in the text, through dropped, hidden or shadow arguments; and (iii) interpreting or computing the dynamic consequences of actions and events in the text. After performing these textual enrichment algorithms, we fine-tune the LLM which allows more robust inference and QA task performance.
10:15	Outro

09:00

Keynote talk by James Pustejovsky (Brandeis University)
Title: Dense Paraphrasing for Textual Enrichment: Question Answering and Inference
Abstract: Much of the current computational work on inference in NLP can be associated with one of two techniques: the first focuses on a specific notion of text-based question answering (QA), using large pre-trained language models (LLMs). To examine specific linguistic properties present in the model, « probing tasks » (diagnostic classifiers) have been developed to test capabilities that the LLM demonstrates on interpretable semantic inferencing tasks, such as age and object comparisons, hypernym conjunction, antonym negation, and others. The second is Knowledge Graph-based inference and QA, where triples are mined from Wikipedia, ConceptNet, WikiData, and other non-corpus resources, and then used for answering questions involving multiple components of the KG (multi-hop QA). While quite impressive with benchmarked metrics in QA, both techniques are completely confused by (a) syntactically missing semantic content, and (b) the semantics accompanying the consequences of events and actions in narratives.
In this talk, I discuss a model we have developed to enrich the surface form of texts, using type-based semantic operations to « textually expose » the deeper meaning of the corpus that was used to make the original embeddings in the language model. This model, Dense Paraphrasing, is a linguistically-motivated, textual enrichment strategy, that textualizes the compositional operations inherent in a semantic model, such as Generative Lexicon Theory or CCG. This involve broadly three kinds of interpretive processes: (i) recognizing the diverse variability in linguistic forms that can be associated with the same underlying semantic representation (paraphrases); (ii) identifying semantic factors or variables that accompany or are presupposed by the lexical semantics of the words present in the text, through dropped, hidden or shadow arguments; and (iii) interpreting or computing the dynamic consequences of actions and events in the text. After performing these textual enrichment algorithms, we fine-tune the LLM which allows more robust inference and QA task performance.

10:15

Outro