Opening the Black Box: High-dimensional Safe Policy Search via Sim-to-real

See file attached for extended abstract submission.

Short abstract:

Bayesian Optimization (BO) is a class of sample efficient methods for optimizing expensive-to-evaluate black-box functions with a wide range of applications for e.g. in robotics, system design and parameter optimization. However, BO is known to be difficult to scale to high-dimensions (d > 20). In order to scale the method and take advantage of its benefits, we propose the SafeOpt-HD pipeline that identifies relevant domain regions for given objective and restricts BO search to this preprocessed domain. By employing cheap (and potentially inaccurate) simulation models, we perform offline computations using Genetic search algorithms to only consider domain subspaces that are likely to contain optimal policies for given task, thus significantly reducing domain size. Our approach can be augmented to any known safe BO methods like SafeOpt, to obtain a safe Bayesian optimization algorithm that is applicable for problems with large input dimensions. To alleviate the issues due to sparsity in the non-uniform preprocessed domain, we propose an approach to systematically generate new input parameters with desirable properties. We evaluate the effectiveness of our proposed approach by optimizing a 48-dimensional policy to perform full position control of a quadrotor, while guaranteeing safety.