A Closer Look to Positive-Unlabeled Learning from Fine-grained Perspectives: An Empirical Study

Ximing Li (Jilin University) · Yuanchao Dai (Jilin University) · Changchun Li (Jilin University) · Zhengzhang Hou (Jilin University) · Yuanbo Xu (Jilin University) · En Wang (Jilin University, China)

aggregate supervisionbinary classifierdisambiguation-free empirical risksempirical analysisexpected riskgeneric trickspositive-unlabeled learningpractical benchmarkpseudo-label estimationpseudo-labeling methodspu learning frameworkrisk-consistentset-aware empirical risksupervised learning weakly-supervised learning

Positive-Unlabeled (PU) learning refers to a specific weakly-supervised learning paradigm that induces a binary classifier with a few positive labeled instances and massive unlabeled instances. To handle this task, the community has proposed dozens of PU learning methods with various techniques, demonstrating strong potential. In this paper, we conduct a comprehensive study to investigate the basic characteristics of current PU learning methods. We organize them into two fundamental families of PU learning, including *disambiguation-free empirical risks*, which approximate the expected risk of supervised learning, and *pseudo-labeling methods*, which estimate pseudo-labels for unlabeled instances. First, we make an empirical analysis on disambiguation-free empirical risks such as uPU, nnPU, and DistPU, and suggest a novel risk-consistent set-aware empirical risk from the perspective of aggregate supervision. Second, we make an empirical analysis of pseudo-labeling methods to evaluate the potential of pseudo-label estimation techniques and widely applied generic tricks in PU learning. Finally, based on those empirical findings, we propose a general framework of PU learning by integrating the set-aware empirical risk with pseudo-labeling. Compared with existing PU learning methods, the proposed framework can be a practical benchmark in PU learning.