110. consistent density functional estimators using k nearest neighbors

Department: Electrical & Computer Engineering
Faculty Advisor(s): Young-Han Kim

Primary Student
Name: Shouvik Ganguly
Email: shgangul@ucsd.edu
Phone: 515-708-7861
Grad Year: 2020

Student Collaborators
Jongha Ryu, jor050@eng.ucsd.edu

Abstract
Given i.i.d. samples drawn from an unknown density, how can one estimate a functional of the density? In this work, a unified approach which can be applied to various functionals including Shannon entropy and Renyi entropy is proposed using k-nearest-neighbor distances from the i.i.d. samples. In contrast to the existing estimators, the proposed estimator is constructed so as to be asymptotically unbiased. The finite-sample analysis of the estimator under mild regularity conditions is performed to show mean-squared consistency of our estimator, and the theoretical guarantees are supported by experiments on certain well-known distributions. This approach can also be naturally generalized to estimating f-divergences.

Industry Application Area(s)
Control Systems

« Back to Posters or Search Results