Timezone: »
We consider the problem of learning good trajectories for manipulation tasks. This is challenging because the criterion defining a good trajectory varies with users, tasks and environments. In this paper, we propose a co-active online learning framework for teaching robots the preferences of its users for object manipulation tasks. The key novelty of our approach lies in the type of feedback expected from the user: the human user does not need to demonstrate optimal trajectories as training data, but merely needs to iteratively provide trajectories that slightly improve over the trajectory currently proposed by the system. We argue that this co-active preference feedback can be more easily elicited from the user than demonstrations of optimal trajectories, which are often challenging and non-intuitive to provide on high degrees of freedom manipulators. Nevertheless, theoretical regret bounds of our algorithm match the asymptotic rates of optimal trajectory algorithms. We also formulate a score function to capture the contextual information and demonstrate the generalizability of our algorithm on a variety of household tasks, for whom, the preferences were not only influenced by the object being manipulated but also by the surrounding environment.
Author Information
Ashesh Jain (Lyft)
Brian Wojcik (Cornell University)
Thorsten Joachims (Cornell)
Ashutosh Saxena (Cornell University)
More from the Same Authors
-
2021 Poster: Fairness in Ranking under Uncertainty »
Ashudeep Singh · David Kempe · Thorsten Joachims -
2020 Poster: MOReL: Model-Based Offline Reinforcement Learning »
Rahul Kidambi · Aravind Rajeswaran · Praneeth Netrapalli · Thorsten Joachims -
2019 : Opening Remarks »
Thorsten Joachims · Nathan Kallus · Michele Santacatterina · Adith Swaminathan · David Sontag · Angela Zhou -
2019 Workshop: Machine Learning with Guarantees »
Ben London · Gintare Karolina Dziugaite · Daniel Roy · Thorsten Joachims · Aleksander Madry · John Shawe-Taylor -
2019 Workshop: “Do the right thing”: machine learning and causal inference for improved decision making »
Michele Santacatterina · Thorsten Joachims · Nathan Kallus · Adith Swaminathan · David Sontag · Angela Zhou -
2019 : Thorsten Joachim: Fair Ranking with Biased Data »
Thorsten Joachims -
2019 : The 3D Object Detection over HD Maps for Autonomous Cars Challenge »
Luc Vincent · Ashesh Jain · Wenjing Zhang · Sanjay Addicam · Vladimir Iglovikov · Peter Ondruska · YUSUKE MURAMATSU -
2019 Poster: Policy Learning for Fairness in Ranking »
Ashudeep Singh · Thorsten Joachims -
2017 : Equality of Opportunity in Rankings »
Thorsten Joachims · Ashudeep Singh -
2017 Workshop: From 'What If?' To 'What Next?' : Causal Inference and Machine Learning for Intelligent Decision Making »
Ricardo Silva · Panagiotis Toulis · John Shawe-Taylor · Alexander Volfovsky · Thorsten Joachims · Lihong Li · Nathan Kallus · Adith Swaminathan -
2016 : Panel Discussion »
Gisbert Schneider · Ross E Goodwin · Simon Colton · Russ Salakhutdinov · Thorsten Joachims · Florian Pinel -
2016 : Structured Prediction with Logged Bandit Feedback »
Thorsten Joachims -
2016 Workshop: "What If?" Inference and Learning of Hypothetical and Counterfactual Interventions in Complex Systems »
Ricardo Silva · John Shawe-Taylor · Adith Swaminathan · Thorsten Joachims -
2015 Poster: The Self-Normalized Estimator for Counterfactual Learning »
Adith Swaminathan · Thorsten Joachims -
2015 Spotlight: The Self-Normalized Estimator for Counterfactual Learning »
Adith Swaminathan · Thorsten Joachims -
2011 Poster: $\theta$-MRF: Capturing Spatial and Semantic Structure in the Parameters for Scene Understanding »
Congcong Li · Ashutosh Saxena · Tsuhan Chen -
2011 Poster: Semantic Labeling of 3D Point Clouds for Indoor Scenes »
Hema Koppula · Abhishek Anand · Thorsten Joachims · Ashutosh Saxena -
2010 Poster: Towards Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models »
Congcong Li · Adarsh P Kowdle · Ashutosh Saxena · Tsuhan Chen -
2008 Oral: Cascaded Classification Models: Combining Models for Holistic Scene Understanding »
Geremy Heitz · Stephen Gould · Ashutosh Saxena · Daphne Koller -
2008 Poster: Cascaded Classification Models: Combining Models for Holistic Scene Understanding »
Geremy Heitz · Stephen Gould · Ashutosh Saxena · Daphne Koller -
2007 Workshop: Machine Learning for Web Search »
Denny Zhou · Olivier Chapelle · Thorsten Joachims · Thomas Hofmann -
2007 Demonstration: Building a 3-D Model From a Single Still Image »
Ashutosh Saxena · min sun · Andrew Y Ng -
2006 Poster: Robotic Grasping of Novel Objects »
Ashutosh Saxena · Justin Driemeyer · Justin Kearns · Andrew Y Ng -
2006 Spotlight: Robotic Grasping of Novel Objects »
Ashutosh Saxena · Justin Driemeyer · Justin Kearns · Andrew Y Ng