120. arm localization of driver & passenger using convolutional neural networks

Department: Electrical & Computer Engineering
Faculty Advisor(s): Mohan M. Trivedi

Primary Student
Name: Kevan Chun Yiu Yuen
Email: kcyuen@ucsd.edu
Phone: 424-644-7701
Grad Year: 2019

In the context of autonomous driving, where humans may need to take over in the event where the computer may issue a takeover request, a key step towards driving safety is the monitoring of the hands to ensure the driver is ready for such a request. This work, focuses on the first step of this process, which is to locate the hands. Such a system must work in real-time and under varying harsh lighting conditions. This paper introduces a fast ConvNet approach, based on the work of original work of OpenPose for full body joint estimation. The network is modified with fewer parameters and retrained using our own day-time naturalistic autonomous driving dataset to estimate joint and affinity heatmaps for driver & passenger's wrist and elbows, for a total of 8 joint classes and part affinity fields between each wrist-elbow pair. The approach runs real-time on real-world data at 25 fps on multiple drivers and passengers. The system is evaluated both quantitatively and qualitatively the proposed dataset, showing at least 95\% detection performance on joint localization and arm-angle estimation.

Industry Application Area(s)
Machine Learning, Neural Network, Driver Assistant System

« Back to Posters or Search Results