research
∙
05/24/2023
Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering
We train a language model (LM) to robustly answer multistep questions by...
share
research
∙
04/04/2023
I2I: Initializing Adapters with Improvised Knowledge
Adapters present a promising solution to the catastrophic forgetting pro...
research
∙
03/25/2023
Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation
The size and the computational load of fine-tuning large-scale pre-train...
research
∙
02/27/2023
Multimodal Speech Recognition for Language-Guided Embodied Agents
Benchmarks for language-guided embodied agents typically assume text-bas...
research
∙
02/11/2023
Improving Sign Recognition with Phonology
We use insights from research on American Sign Language (ASL) phonology ...
research
∙
01/30/2023
RREx-BoT: Remote Referring Expressions with a Bag of Tricks
Household robots operate in the same space for years. Such robots increm...
research
∙
11/30/2022
CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation
Household environments are visually diverse. Embodied agents performing ...
research
∙
10/26/2022
Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems
For vision-and-language reasoning tasks, both fully connectionist, end-t...
research
∙
09/22/2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Task planning can require defining myriad domain knowledge about the wor...
research
∙
08/18/2022
VAuLT: Augmenting the Vision-and-Language Transformer with the Propagation of Deep Language Representations
We propose the Vision-and-Augmented-Language Transformer (VAuLT). VAuLT ...
research
∙
07/29/2022
Curriculum Learning for Data-Efficient Vision-Language Alignment
Aligning image and text encoders from scratch using contrastive learning...
research
∙
07/01/2022
Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic
Natural language is an intuitive way for humans to communicate tasks to ...
research
∙
06/18/2022
CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks
Current state-of-the-art vision-and-language models are evaluated on tas...
research
∙
03/22/2022
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
A long-term goal of AI research is to build intelligent agents that can ...
research
∙
11/10/2021
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges
Learning-based methods for training embodied agents typically require a ...
research
∙
10/01/2021
TEACh: Task-driven Embodied Agents that Chat
Robots operating in human spaces must be able to engage in natural langu...
research
∙
08/10/2021
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion
Language-guided robots performing home and office tasks must navigate in...
research
∙
07/26/2021
Language Grounding with 3D Objects
Seemingly simple natural language requests to a robot are generally unde...
research
∙
10/23/2020
The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation
Autonomous robot systems for applications from search and rescue to assi...
research
∙
05/02/2020
RMM: A Recursive Mental Model for Dialog Navigation
Fluent communication requires understanding your audience. In the new co...
research
∙
04/21/2020
Experience Grounds Language
Successful linguistic communication relies on a shared experience of the...
research
∙
12/03/2019
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
We present ALFRED (Action Learning From Realistic Environments and Direc...
research
∙
07/10/2019
Vision-and-Dialog Navigation
Robots navigating in human environments should use language to ask for a...
research
∙
07/08/2019
Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog
Some robots can interact with humans using natural language, and identif...
research
∙
04/02/2019
Improving Robot Success Detection using Static Object Data
We use static object data to improve success detection for stacking obje...
research
∙
03/29/2019
Interpreting Black Box Models with Statistical Guarantees
While many methods for interpreting machine learning models have been pr...
research
∙
03/20/2019
Prospection: Interpretable Plans From Language By Predicting the Future
High-level human instructions often correspond to behaviors with multipl...
research
∙
03/01/2019
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Natural language understanding for robotics can require substantial doma...
research
∙
11/01/2018
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
Language-and-vision navigation and question answering (QA) are exciting ...
research
∙
10/06/2018
Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence
Efforts are underway at UT Austin to build autonomous robot systems that...
research
∙
08/06/2017