Graph-Based Estimation of Information-Theoretic Measures
Information-theoretic measures form a fundamental class of measures used in Machine Learning and Artificial Intelligence problems such as Classification, Clustering, Deep Learning, Anomaly detection, reinforcement learning, and have recently received increasing interest. It is difficult to accurately estimate the information-theoretic measures in high-dimensional settings, specially where the data is multivariate having unknown probability density functions—the setting considered in this project. Computational complexity is an important challenge in information measure estimation. Most plug-in-based estimators, such as the kernel density estimator (KDE) or the K-nearest-neighbor (KNN) estimator with known convergence rate, require runtime complexity of O(n), which is not suitable for large scale applications. We are studying graph theoretic direct estimation methods based on k-NN and MST graphs witch are computationally more tractable than other competing estimators.
