21. learning to augment visual data

Department: Electrical & Computer Engineering
Research Institute Affiliation: Agile Center for Visual Computing
Faculty Advisor(s): Nuno M. Vasconcelos

Primary Student
Name: Mandar Dilip Dixit
Email: mdixit@ucsd.edu
Phone: 858-361-3304
Grad Year: 2017

We consider the problem of data augmentation, i.e., generating artificial samples to extend a given corpus of training data. Specifically, we propose attributed-guided augmentation (AGA) which learns a mapping that allows to synthesize data such that an attribute of a synthesized sample is at a desired value or strength. This is particularly interesting in situations where little data with no attribute annotation is available for learning, but we have access to a large external corpus of heavily annotated samples. While prior works primarily augment in the space of images, we propose to perform augmentation in feature space instead. We implement our approach as a deep encoder-decoder architecture that learns the synthesis function in an end-to-end manner. We demonstrate the utility of our approach on the problems of (1) one-shot object recognition in a transfer-learning setting where we have no prior knowledge of the new classes, as well as (2) object-based one-shot scene recognition. As external data, we leverage 3D depth and pose information from the SUN RGB-D dataset. Our experiments show that attribute-guided augmentation of high-level CNN features considerably improves one-shot recognition performance on both problems.

Industry Application Area(s)
Internet, Networking, Systems | Software, Analytics

« Back to Posters or Search Results