58562 - Constraint Handling in Bayesian Optimization -- a Comparative Study of Support Vector Machine, Augmented Lagrangian and Expected Feasible Improvement 

Many engineering problems involve complex constraints which can be computationally costly. To reduce the overall numerical cost, such constrained
optimization problems are commonly solved via surrogate models constructed on a Design of Exper-
iment (DoE). Meanwhile, complex constraints may lead to infeasible initial DoE, which
can be problematic for subsequent sequential optimization. In this study, we address
constrained optimization problem in a Bayesian optimization framework. A comparative study is conducted to evaluate the performance of three approaches namely Expected Feasible Improvement (EFI) and slack Augmented
Lagrangian method (AL) and Expected Improvement with Probabilistic Support Vector Machine (EIPSVM) in constraint handling with feasible
or infeasible initial DoE. AL is capable to start iterative optimization process with infeasible
initial DoE, while EFI and EIPSVM require extra a priori enrichment to find at least one feasible sam-
ple. Empirical experiments are performed on both analytical functions and a low pressure turbine disc design problem. Through these benchmark problems, EFI and AL are shown to have
overall similar performance in problems with inequality constraints. However, the performance of EIPSVM is affected strongly by the corresponding hyperparameter values. In addition, we show evidences
that with an appropriate handling of infeasible initial DoE, EFI does not necessarily
underperform compared with AL solving optimization problems with mixed inequality and equality constraints.