Timezone: »

 
Poster
Understanding Programmatic Weak Supervision via Source-aware Influence Function
Jieyu Zhang · Haonan Wang · Cheng-Yu Hsieh · Alexander Ratner

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #426
in Poster Session 3 »

Programmatic Weak Supervision (PWS) aggregates the source votes of multiple weak supervision sources into probabilistic training labels, which are in turn used to train an end model. With its increasing popularity, it is critical to have some tool for users to understand the influence of each component (\eg, the source vote or training data) in the pipeline and interpret the end model behavior. To achieve this, we build on Influence Function (IF) and propose source-aware IF, which leverages the generation process of the probabilistic labels to decompose the end model's training objective and then calculate the influence associated with each (data, source, class) tuple. These primitive influence score can then be used to estimate the influence of individual component of PWS, such as source vote, supervision source, and training data. On datasets of diverse domains, we demonstrate multiple use cases: (1) interpreting incorrect predictions from multiple angles that reveals insights for debugging the PWS pipeline, (2) identifying mislabeling of sources with a gain of 9\%-37\% over baselines, and (3) improving the end model's generalization performance by removing harmful components in the training objective (13\%-24\% better than ordinary IF).

Keywords: influence function · interpretability · Data-centric Method · Programmatic Weak Supervision
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Jieyu Zhang (Department of Computer Science, University of Washington)
Haonan Wang (national university of singaore, National University of Singapore)
Cheng-Yu Hsieh (University of Washington)
Alexander Ratner (Stanford University)

More from the Same Authors