research
∙
09/18/2023
Reasoning about the Unseen for Efficient Outdoor Object Navigation
Robots should exist anywhere humans do: indoors, outdoors, and even unma...
share
research
∙
09/15/2023
MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Perception
A holistic understanding of object properties across diverse sensory mod...
research
∙
07/25/2023
MAEA: Multimodal Attribution for Embodied AI
Understanding multimodal perception for embodied AI is an open question ...
research
∙
06/20/2023
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and...
research
∙
05/24/2023
SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
Open-world survival games pose significant challenges for AI algorithms ...
research
∙
05/03/2023
Plan, Eliminate, and Track – Language Models are Good Teachers for Embodied Agents
Pre-trained large language models (LLMs) capture procedural knowledge ab...
research
∙
02/13/2023
The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment
Increased focus on the deployment of machine learning systems has led to...
research
∙
12/09/2022
Object Goal Navigation with End-to-End Self-Supervision
A household robot should be able to navigate to target locations without...
research
∙
07/06/2022
Transformers are Adaptable Task Planners
Every home is different, and every person likes things done in their par...
research
∙
05/24/2022
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization
Integrating vision and language has gained notable attention following t...
research
∙
05/19/2022
Training Vision-Language Transformers from Captions Alone
We show that Vision-Language Transformers can be learned without human l...
research
∙
03/06/2022
HEAR 2021: Holistic Evaluation of Audio Representations
What audio embedding approach generalizes best to a wide range of downst...
research
∙
12/17/2021
It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process
The computing research community needs to work much harder to address th...
research
∙
12/16/2021
KAT: A Knowledge Augmented Transformer for Vision-and-Language
The primary focus of recent work with largescale transformers has been o...
research
∙
10/14/2021
Learning When and What to Ask: a Hierarchical Reinforcement Learning Framework
Reliable AI agents should be mindful of the limits of their knowledge an...
research
∙
10/12/2021
FILM: Following Instructions in Language with Modular Methods
Recent methods for embodied instruction following are typically trained ...
research
∙
09/01/2021
WebQA: Multihop and Multimodal QA
Web search is fundamentally multimodal and multihop. Often, even before ...
research
∙
08/23/2021
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
Contrastive learning has been widely used to train transformer-based vis...
research
∙
07/26/2021
Language Grounding with 3D Objects
Seemingly simple natural language requests to a robot are generally unde...
research
∙
06/04/2021
Grounding 'Grounding' in NLP
The NLP community has seen substantial recent interest in grounding to f...
research
∙
04/18/2021
Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models
Numerous works have analyzed biases in vision and pre-trained language m...
research
∙
01/31/2021
An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games
Guessing games are a prototypical instance of the "learning by interacti...
research
∙
11/07/2020
Knowledge-driven Self-supervision for Zero-shot Commonsense Question Answering
Recent developments in pre-trained neural language modeling have led to ...
research
∙
11/05/2020
Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games
In visual guessing games, a Guesser has to identify a target object in a...
research
∙
10/08/2020
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
Given a simple request (e.g., Put a washed apple in the kitchen fridge),...
research
∙
07/29/2020
The Return of Lexical Dependencies: Neural Lexicalized PCFGs
In this paper we demonstrate that context free grammar (CFG) based metho...
research
∙
05/02/2020
RMM: A Recursive Mental Model for Dialog Navigation
Fluent communication requires understanding your audience. In the new co...
research
∙
05/02/2020
A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos
Procedural knowledge, which we define as concrete information about the ...
research
∙
04/21/2020
Experience Grounds Language
Successful linguistic communication relies on a shared experience of the...
research
∙
03/02/2020
Multi-View Learning for Vision-and-Language Navigation
Learning to navigate in a visual environment following natural language ...
research
∙
12/03/2019
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
We present ALFRED (Action Learning From Realistic Environments and Direc...
research
∙
11/26/2019
PIQA: Reasoning about Physical Commonsense in Natural Language
To apply eyeshadow without a brush, should I use a cotton swab or a toot...
research
∙
09/05/2019
Robust Navigation with Language Pretraining and Stochastic Sampling
Core to the vision-and-language navigation (VLN) challenge is building r...
research
∙
05/29/2019
Defending Against Neural Fake News
Recent progress in natural language generation has raised dual-use conce...
research
∙
05/19/2019
HellaSwag: Can a Machine Really Finish Your Sentence?
Recent work by Zellers et al. (2018) introduced a new task of commonsens...
research
∙
04/02/2019
Improving Robot Success Detection using Static Object Data
We use static object data to improve success detection for stacking obje...
research
∙
03/20/2019
Prospection: Interpretable Plans From Language By Predicting the Future
High-level human instructions often correspond to behaviors with multipl...
research
∙
03/06/2019
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
We present FAST NAVIGATOR, a general framework for action decoding, whic...
research
∙
02/02/2019
Character-based Surprisal as a Model of Human Reading in the Presence of Errors
Intuitively, human readers cope easily with errors in text; typos, missp...
research
∙
11/27/2018
From Recognition to Cognition: Visual Commonsense Reasoning
Visual understanding goes well beyond object recognition. With one glanc...
research
∙
11/21/2018
Early Fusion for Goal Directed Robotic Vision
Increasingly, perceptual systems are being codified as strict pipelines ...
research
∙
11/01/2018
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
Language-and-vision navigation and question answering (QA) are exciting ...
research
∙
08/16/2018
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Given a partial description like "she opened the hood of the car," human...
research
∙
05/28/2018
Inducing Grammars with and for Neural Machine Translation
Machine translation systems require semantic knowledge and grammatical u...
research
∙
05/20/2018
Balancing Shared Autonomy with Human-Robot Communication
Robotic agents that share autonomy with a human should leverage human do...
research
∙
01/23/2018
CHALET: Cornell House Agent Learning Environment
We present CHALET, a 3D house simulator with support for navigation and ...
research
∙
12/10/2017
Learning Interpretable Spatial Operations in a Rich 3D Blocks World
In this paper, we study the problem of mapping natural language instruct...
research
∙
11/06/2017
Synthetic and Natural Noise Both Break Neural Machine Translation
Character-based neural machine translation (NMT) models alleviate out-of...
research
∙
10/09/2017
Natural Language Inference from Multiple Premises
We define a novel textual entailment task that requires inference over m...
research
∙
09/29/2016