Timezone: »

 
Poster
Validation Free and Replication Robust Volume-based Data Valuation
Xinyi Xu · Zhaoxuan Wu · Chuan Sheng Foo · Bryan Kian Hsiang Low

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual
in Poster Session 1 »
Event URL: https://github.com/ZhaoxuanWu/VolumeBased-DataValuation »

Data valuation arises as a non-trivial challenge in real-world use cases such as collaborative machine learning, federated learning, trusted data sharing, data marketplaces. The value of data is often associated with the learning performance (e.g., validation accuracy) of a model trained on the data, which introduces a close coupling between data valuation and validation. However, a validation set may notbe available in practice and it can be challenging for the data providers to reach an agreement on the choice of the validation set. Another practical issue is that of data replication: Given the value of some data points, a dishonest data provider may replicate these data points to exploit the valuation for a larger reward/payment. We observe that the diversity of the data points is an inherent property of a dataset that is independent of validation. We formalize diversity via the volume of the data matrix (i.e., determinant of its left Gram), which allows us to establish a formal connection between the diversity of data and learning performance without requiring validation. Furthermore, we propose a robust volume measure with a theoretical guarantee on the replication robustness by following the intuition that copying the same data points does not increase the diversity of data. We perform extensive experiments to demonstrate its consistency in valuation and practical advantages over existing baselines and show that our method is model- and task-agnostic and can be flexibly adapted to handle various neural networks.

Keywords: Deep Learning · Robustness

Author Information

Xinyi Xu (National University of Singapore)
Xinyi Xu

I am a fourth year Ph.D. student (funded by A*STAR through the ACIS Scholarship) in the department of computer science at National University of Singapore where I study multi-agent machine learning systems.

Zhaoxuan Wu (National University of Singapore)
Chuan Sheng Foo (Institute for Infocomm Research)
Bryan Kian Hsiang Low (National University of Singapore)

More from the Same Authors