research
∙
08/28/2023
CoVR: Learning Composed Video Retrieval from Web Video Captions
Composed Image Retrieval (CoIR) has recently gained popularity as a task...
share
research
∙
08/24/2023
POCO: 3D Pose and Shape Estimation with Confidence
The regression of 3D Human Pose and Shape (HPS) from an image is becomin...
research
∙
08/21/2023
UnLoc: A Unified Framework for Video Localization Tasks
While large-scale image-text pretrained models such as CLIP have been us...
research
∙
08/10/2023
Object Goal Navigation with Recursive Implicit Maps
Object goal navigation aims to navigate an agent to locations of a given...
research
∙
07/28/2023
Robust Visual Sim-to-Real Transfer for Robotic Manipulation
Learning visuomotor policies in simulation is much safer and cheaper tha...
research
∙
07/17/2023
Does Visual Pretraining Help End-to-End Reasoning?
We aim to investigate whether end-to-end learning of visual reasoning ca...
research
∙
06/20/2023
Dense Video Object Captioning from Disjoint Supervision
We propose a new task and model for dense video object captioning – dete...
research
∙
06/20/2023
How can objects help action recognition?
Current state-of-the-art video models process a video clip as a long seq...
research
∙
06/13/2023
AVIS: Autonomous Visual Information Seeking with Large Language Models
In this paper, we propose an autonomous information seeking visual quest...
research
∙
06/12/2023
Waffling around for Performance: Visual Classification with Random Words and Broad Concepts
The visual classification performance of vision-language models such as ...
research
∙
06/12/2023
Retrieval-Enhanced Contrastive Vision-Text Models
Contrastive image-text models such as CLIP form the building blocks of m...
research
∙
06/08/2023
Modular Visual Question Answering via Code Generation
We present a framework that formulates visual question answering as modu...
research
∙
05/10/2023
Learning Video-Conditioned Policies for Unseen Manipulation Tasks
The ability to specify robot commands by a non-expert user is critical f...
research
∙
04/24/2023
End-to-End Spatio-Temporal Action Localisation with Video Transformers
The most performant spatio-temporal action localisation models use exter...
research
∙
04/24/2023
gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction
Signed distance functions (SDFs) is an attractive framework that has rec...
research
∙
04/13/2023
Verbs in Action: Improving verb understanding in video-language models
Understanding verbs is crucial to modelling how people and objects inter...
research
∙
04/13/2023
Contact Models in Robotics: a Comparative Analysis
Physics simulation is ubiquitous in robotics. Whether in model-based app...
research
∙
04/11/2023
Improving Image Recognition by Retrieving from Web-Scale Image-Text Data
Retrieval augmented models are becoming increasingly popular for compute...
research
∙
04/06/2023
Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval
Cross-modal retrieval methods are the preferred tool to search databases...
research
∙
04/04/2023
Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification
Due to the expensive costs of collecting labels in multi-label classific...
research
∙
03/29/2023
AVFormer: Injecting Vision into Frozen Speech Models for Zero-Shot AV-ASR
Audiovisual automatic speech recognition (AV-ASR) aims to improve the ro...
research
∙
02/27/2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
In this work, we introduce Vid2Seq, a multi-modal single-stage dense eve...
research
∙
12/20/2022
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation
One of the major challenges of machine translation (MT) is ambiguity, wh...
research
∙
12/10/2022
REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory
In this paper, we propose an end-to-end Retrieval-Augmented Visual Langu...
research
∙
12/09/2022
Audiovisual Masked Autoencoders
Can we leverage the audiovisual information already present in video to ...
research
∙
12/05/2022
Location-Aware Self-Supervised Transformers
Pixel-level labels are particularly expensive to acquire. Hence, pretrai...
research
∙
11/25/2022
WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Prediction
This paper presents WALDO (WArping Layer-Decomposed Objects), a novel ap...
research
∙
11/18/2022
AVATAR submission to the Ego4D AV Transcription Challenge
In this report, we describe our submission to the Ego4D AudioVisual (AV)...
research
∙
11/17/2022
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding
Localizing objects in 3D scenes based on natural language requires under...
research
∙
11/16/2022
Learning Reward Functions for Robotic Manipulation by Observing Humans
Observing a human demonstrator manipulate objects provides a rich, scala...
research
∙
10/10/2022
A Memory Transformer Network for Incremental Learning
We study class-incremental learning, a training setup in which new class...
research
∙
09/19/2022
Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control
Reinforcement learning (RL) and trajectory optimization (TO) present str...
research
∙
09/11/2022
Instruction-driven history-aware policies for robotic manipulations
In human environments, robots are expected to accomplish a variety of ma...
research
∙
08/24/2022
Learning from Unlabeled 3D Environments for Vision-and-Language Navigation
In vision-and-language navigation (VLN), an embodied agent is required t...
research
∙
08/14/2022
TL;DW? Summarizing Instructional Videos with Task Relevance Cross-Modal Saliency
YouTube users looking for instructions for a specific task may spend a l...
research
∙
07/26/2022
AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction
Recent work achieved impressive progress towards joint reconstruction of...
research
∙
07/08/2022
Beyond Transfer Learning: Co-finetuning for Action Localisation
Transfer learning is the predominant paradigm for training deep networks...
research
∙
06/23/2022
Augmenting differentiable physics with randomized smoothing
In the past few years, following the differentiable programming paradigm...
research
∙
06/20/2022
M M Mix: A Multimodal Multiview Transformer Ensemble
This report describes the approach behind our winning solution to the 20...
research
∙
06/16/2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
Video question answering (VideoQA) is a complex task that requires diver...
research
∙
06/15/2022
AVATAR: Unconstrained Audiovisual Speech Recognition
Audio-visual automatic speech recognition (AV-ASR) is an extension of AS...
research
∙
05/10/2022
Learning to Answer Visual Questions from Web Videos
Recent methods for visual question answering rely on large-scale annotat...
research
∙
05/10/2022
Weakly-supervised segmentation of referring expressions
Visual grounding localizes regions (boxes or segments) in the image corr...
research
∙
04/20/2022
Assembly Planning from Observations under Physical Constraints
This paper addresses the problem of copying an unknown assembly of primi...
research
∙
04/01/2022
Learning Audio-Video Modalities from Image Captions
A major challenge in text-video and text-audio retrieval is the lack of ...
research
∙
03/30/2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers
We consider the problem of localizing a spatio-temporal tube in a video ...
research
∙
03/08/2022
Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems
Optimal control (OC) algorithms such as Differential Dynamic Programming...
research
∙
02/28/2022
The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields
Both a good understanding of geometrical concepts and a broad familiarit...
research
∙
02/23/2022
Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Following language instructions to navigate in unseen environments is a ...
research
∙
02/04/2022