Timezone: »
Offering prediction APIs for fee is a fast growing industry and is an important aspect of machine learning as a service. While many such services are available, the heterogeneity in their price and performance makes it challenging for users to decide which API or combination of APIs to use for their own data and budget. We take a first step towards addressing this challenge by proposing FrugalML, a principled framework that jointly learns the strength and weakness of each API on different data, and performs an efficient optimization to automatically identify the best sequential strategy to adaptively use the available APIs within a budget constraint. Our theoretical analysis shows that natural sparsity in the formulation can be leveraged to make FrugalML efficient. We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including facial emotion recognition, sentiment analysis and speech recognition. Across various tasks, FrugalML can achieve up to 90% cost reduction while matching the accuracy of the best single API, or up to 5% better accuracy while matching the best API’s cost.
Author Information
Lingjiao Chen (Stanford University)
Matei Zaharia (Stanford and Databricks)
James Zou (Stanford University)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: FrugalML: How to use ML Prediction APIs more accurately and cheaply »
Thu. Dec 10th 05:00 -- 07:00 AM Room Poster Session 4 #1306
More from the Same Authors
-
2021 Spotlight: Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval »
Omar Khattab · Christopher Potts · Matei Zaharia -
2022 : Protein structure generation via folding diffusion »
Kevin Wu · Kevin Yang · Rianne van den Berg · James Zou · Alex X Lu · Ava Soleimany -
2022 : Predicting Immune Escape with Pretrained Protein Language Model Embeddings »
Kyle Swanson · Howard Chang · James Zou -
2022 : DrML: Diagnosing and Rectifying Vision Models using Language »
Yuhui Zhang · Jeff Z. HaoChen · Shih-Cheng Huang · Kuan-Chieh Wang · James Zou · Serena Yeung -
2021 Poster: Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval »
Omar Khattab · Christopher Potts · Matei Zaharia -
2020 Poster: Neuron Shapley: Discovering the Responsible Neurons »
Amirata Ghorbani · James Zou -
2020 Poster: MOPO: Model-based Offline Policy Optimization »
Tianhe Yu · Garrett Thomas · Lantao Yu · Stefano Ermon · James Zou · Sergey Levine · Chelsea Finn · Tengyu Ma -
2019 : Phenotype »
Nir HaCohen · David Reshef · Matthew Johnson · Sam Morris · Aurel Nagy · Gokcen Eraslan · Meromit Singer · Eliezer Van Allen · Smita Krishnaswamy · Casey Greene · Scott Linderman · Alexander Wiltschko · Dylan Kotliar · James Zou · Brendan Bulik-Sullivan -
2019 Poster: Towards Automatic Concept-based Explanations »
Amirata Ghorbani · James Wexler · James Zou · Been Kim -
2018 Poster: The Effect of Network Width on the Performance of Large-batch Training »
Lingjiao Chen · Hongyi Wang · Jinman Zhao · Dimitris Papailiopoulos · Paraschos Koutris -
2018 Poster: Learning a Warping Distance from Unlabeled Time Series Using Sequence Autoencoders »
Abubakar Abid · James Zou