research
∙
04/17/2023
Conditional Generation of Audio from Video via Foley Analogies
The sound effects that designers add to videos are designed to convey a ...
share
research
∙
03/30/2023
Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment
How does audio describe the world around us? In this paper, we propose a...
research
∙
03/21/2023
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
We present Text2Room, a method for generating room-scale textured 3D mes...
research
∙
03/20/2023
Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation
The images and sounds that we perceive undergo subtle but geometrically ...
research
∙
01/11/2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata
We learn a visual representation that captures information about the cam...
research
∙
01/04/2023
Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Manipulated videos often contain subtle inconsistencies between their vi...
research
∙
11/28/2022
Mix and Localize: Localizing Sound Sources in Mixtures
We present a method for simultaneously localizing multiple sound sources...
research
∙
11/22/2022
Touch and Go: Learning from Human-Collected Vision and Touch
The ability to associate touch with sight is essential for tasks that re...
research
∙
05/10/2022
Learning Visual Styles from Audio-Visual Associations
From the patter of rain to the crunch of snow, the sounds we hear often ...
research
∙
04/26/2022
Sound Localization by Self-Supervised Time Delay Estimation
Sounds reach one microphone in a stereo pair sooner than the other, resu...
research
∙
01/20/2022
Learning Pixel Trajectories with Multiscale Contrastive Random Walks
A range of video modeling tasks, from optical flow to multiple object tr...
research
∙
01/18/2022
GANmouflage: 3D Object Nondetection with Texture Fields
We propose a method that learns to camouflage 3D objects within scenes. ...
research
∙
11/10/2021
Structure from Silence: Learning Scene Structure from Ambient Sound
From whirling ceiling fans to ticking clocks, the sounds that we hear su...
research
∙
04/19/2021
Comparing Correspondences: Video Prediction with Correspondence-wise Losses
Today's image prediction methods struggle to change the locations of obj...
research
∙
04/06/2021
Strumming to the Beat: Audio-Conditioned Contrastive Video Textures
We introduce a non-parametric approach for infinite video texture synthe...
research
∙
03/26/2021
Planar Surface Reconstruction from Sparse Views
The paper studies planar surface reconstruction of indoor scenes from tw...
research
∙
08/10/2020
Self-Supervised Learning of Audio-Visual Objects from Video
Our objective is to transform a video into a set of discrete audio-visua...
research
∙
06/25/2020
Space-Time Correspondence as a Contrastive Random Walk
This paper proposes a simple self-supervised approach for learning repre...
research
∙
12/23/2019
CNN-generated images are surprisingly easy to spot... for now
In this work we ask whether it is possible to create a "universal" detec...
research
∙
06/13/2019
Detecting Photoshopped Faces by Scripting Photoshop
Most malicious photo manipulations are created using standard image edit...
research
∙
06/10/2019
Learning Individual Styles of Conversational Gesture
Human speech is often accompanied by hand and arm gestures. Given audio ...
research
∙
09/14/2018
MoSculp: Interactive Visualization of Shape and Time
We present a system that allows users to visualize complex human motion ...
research
∙
05/28/2018
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
For humans, the process of grasping an object relies heavily on rich tac...
research
∙
05/10/2018
Fighting Fake News: Image Splice Detection via Learned Self-Consistency
Advances in photo editing and manipulation tools have made it significan...
research
∙
04/10/2018
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
The thud of a bouncing ball, the onset of speech as lips open -- when vi...
research
∙
12/20/2017
Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning
The sound of crashing waves, the roar of fast-moving cars -- sound conve...
research
∙
10/16/2017
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
A successful grasp requires careful balancing of the contact forces. Ded...
research
∙
08/25/2016
Ambient Sound Provides Supervision for Visual Learning
The sound of crashing waves, the roar of fast-moving cars -- sound conve...
research
∙
12/28/2015