Artificial intelligence research has shown dramatic improvements in performance on a variety of natural language processing tasks by feeding large corpora of natural language into machine learning systems. However, there is still not a clear and deep understanding of how these celebrated systems actually represent the meaning of language inputs, and whether they do it in ways similar to humans.

Join MLK Scholar Jamie Macbeth in discussing his work using paraphrasing to study meaning representation in a cognitive system.

Dr. Macbeth claims that a consequential part of the meaning representation for a natural language expression is a set of language-free structures and processes that are not part of the language expression in question. To support the claim, we construct a corpus of paraphrase pairs using a system that transforms language inputs into language-free structures that are complex combinations of conceptual primitives, decoupled from a linguistic system that generates natural language from them. This corpus of paraphrase pairs is special in that it represents a full range of syntactic and lexical difference in its constituent sentences. We conduct an extensive analysis comparing the performance of a state-of-the-art neural network model against humans performing the paraphrase detection task. We find that the neural model deviates significantly from human classification performance, particularly on sentence pairs that convey the same meaning while exhibiting significant differences lexically and syntactically. As the neural network model is trained only on linguistic items, the discrepancy points to the existence and necessity of a significant non-linguistic component of meaning formation.

We are committed to making this dialogue accessible for all MIT community members. Please reach out to rornitz@mit.edu with accessibility requests.

REGISTER