research
∙
08/27/2023
An Analysis of On-the-fly Determinization of Finite-state Automata
In this paper we establish an abstraction of on-the-fly determinization ...
share
research
∙
07/27/2023
A Geometric Notion of Causal Probing
Large language models rely on real-valued representations of text to mak...
research
∙
07/07/2023
On the Efficacy of Sampling Adapters
Sampling is a common strategy for generating text from probabilistic mod...
research
∙
07/07/2023
Testing the Predictions of Surprisal Theory in 11 Languages
A fundamental result in psycholinguistics is that less predictable words...
research
∙
07/06/2023
Generalizing Backpropagation for Gradient-Based Interpretability
Many popular feature-attribution methods for interpreting deep neural ne...
research
∙
07/06/2023
Efficient Semiring-Weighted Earley Parsing
This paper provides a reference description, in the form of a deduction ...
research
∙
06/29/2023
Tokenization and the Noiseless Channel
Subword tokenization is a key part of many NLP pipelines. However, littl...
research
∙
06/29/2023
A Formal Perspective on Byte-Pair Encoding
Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data...
research
∙
06/08/2023
Hexatagging: Projective Dependency Parsing as Tagging
We introduce a novel dependency parser, the hexatagger, that constructs ...
research
∙
06/06/2023
LEACE: Perfect linear concept erasure in closed form
Concept erasure aims to remove specified features from a representation....
research
∙
06/06/2023
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
While natural languages differ widely in both canonical word order and w...
research
∙
06/06/2023
Convergence and Diversity in the Control Hierarchy
Weir has defined a hierarchy of language classes whose second member (ℒ_...
research
∙
06/05/2023
Structured Voronoi Sampling
Recently, there has been a growing interest in the development of gradie...
research
∙
06/04/2023
A Fast Algorithm for Computing Prefix Probabilities
Multiple algorithms are known for efficiently calculating the prefix pro...
research
∙
05/24/2023
Learning the String Partial Order
We show that most structured prediction problems can be solved in linear...
research
∙
05/23/2023
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus ...
research
∙
05/22/2023
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
The fixed-size context of Transformer makes GPT models incapable of gene...
research
∙
05/18/2023
Efficient Prompting via Dynamic In-Context Learning
The primary way of building AI applications is shifting from training sp...
research
∙
05/18/2023
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Several recent papers claim human parity at sentence-level Machine Trans...
research
∙
04/27/2023
Controlled Text Generation with Natural Language Instructions
Large language models generate fluent texts and can follow natural langu...
research
∙
03/30/2023
Discriminative Class Tokens for Text-to-Image Diffusion Models
Recent advances in text-to-image diffusion models have enabled the gener...
research
∙
01/17/2023
Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs
Weighted finite-state automata (WSFAs) are commonly used in NLP. Failure...
research
∙
12/20/2022
A Measure-Theoretic Characterization of Tight Language Models
Language modeling, a central task in natural language processing, involv...
research
∙
12/08/2022
The Ordered Matrix Dirichlet for Modeling Ordinal Dynamics
Many dynamical systems exhibit latent states with intrinsic orderings su...
research
∙
11/25/2022
On the Effect of Anticipation on Reading Times
Over the past two decades, numerous studies have demonstrated how less p...
research
∙
11/23/2022
Schrödinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition
Recent work has shown that despite their impressive capabilities, text-t...
research
∙
11/14/2022
On Parsing as Tagging
There have been many proposals to reduce constituency parsing to tagging...
research
∙
11/11/2022
The Architectural Bottleneck Principle
In this paper, we seek to measure how much information a component in a ...
research
∙
10/26/2022
Autoregressive Structured Prediction with Language Models
Recent years have seen a paradigm shift in NLP towards using pretrained ...
research
∙
10/26/2022
Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems
Centering theory (CT; Grosz et al., 1995) provides a linguistic analysis...
research
∙
10/26/2022
A Bilingual Parallel Corpus with Discourse Annotations
Machine translation (MT) has almost achieved human parity at sentence-le...
research
∙
10/24/2022
Mutual Information Alleviates Hallucinations in Abstractive Summarization
Despite significant progress in the quality of language generated from a...
research
∙
10/18/2022
Linear Guardedness and its Implications
Previous work on concept identification in neural representations has fo...
research
∙
10/13/2022
Algorithms for Weighted Pushdown Automata
Weighted pushdown automata (WPDAs) are at the core of many natural langu...
research
∙
10/08/2022
An Ordinal Latent Variable Model of Conflict Intensity
For the quantitative monitoring of international relations, political ev...
research
∙
09/22/2022
Equivariant Transduction through Invariant Alignment
The ability to generalize compositionally is key to understanding the po...
research
∙
09/14/2022
On the Intersection of Context-Free and Regular Languages
The Bar-Hillel construction is a classic result in formal language theor...
research
∙
08/17/2022
On the Role of Negative Precedent in Legal Outcome Prediction
Every legal case sets a precedent by developing the law in one of the fo...
research
∙
08/17/2022
Learning Transductions to Test Systematic Compositionality
Recombining known primitive concepts into larger novel combinations is a...
research
∙
08/17/2022
Visual Comparison of Language Model Adaptation
Neural language models are widely used; however, their model parameters ...
research
∙
07/04/2022
Probing via Prompting
Probing is a popular method to discern what linguistic information is co...
research
∙
06/15/2022
The SIGMORPHON 2022 Shared Task on Morpheme Segmentation
The SIGMORPHON 2022 shared task on morpheme segmentation challenged syst...
research
∙
05/31/2022
Cluster-based Evaluation of Automatically Generated Text
While probabilistic language generators have improved dramatically over ...
research
∙
05/14/2022
Naturalistic Causal Probing for Morpho-Syntax
Probing has become a go-to methodology for interpreting and analyzing de...
research
∙
05/08/2022
A Structured Span Selector
Many natural language processing tasks, e.g., coreference resolution and...
research
∙
05/07/2022
UniMorph 4.0: Universal Morphology
The Universal Morphology (UniMorph) project is a collaborative effort pr...
research
∙
05/04/2022
Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models
The success of multilingual pre-trained models is underpinned by their a...
research
∙
05/03/2022
Exact Paired-Permutation Testing for Structured Test Statistics
Significance testing – especially the paired-permutation test – has play...
research
∙
04/19/2022
Probing for the Usage of Grammatical Number
A central quest of probing is to uncover how pre-trained models encode a...
research
∙
04/04/2022