Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization

verfasst von

Carolin Benjamins, Gjorgjina Cenikj, Ana Nikolikj, Aditya Mohan, Tome Eftimov, Marius Lindauer

Abstract

Dynamic Algorithm Configuration (DAC) addresses the challenge
of dynamically setting hyperparameters of an algorithm for a diverse set of instances rather than focusing solely on individual
tasks. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. However, the limited generalization performance of these agents has significantly hindered
the application in DAC. Our hypothesis is that a potential bias in
the training instances limits generalization capabilities. We take
a step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and then
retraining the agent on this subset to improve its generalization
performance. For constructing the meta-features for the subset selection, we particularly account for the dynamic nature of the RL
agent by computing time series features on trajectories of actions
and rewards generated by the agent’s interaction with the environment. Through empirical evaluations on the Sigmoid and CMA-ES
benchmarks from the standard benchmark library for DAC, called
DACBench, we discuss the potentials of our selection technique
compared to training on the entire instance set. Our results highlight the efficacy of instance selection in refining DAC policies for
diverse instance spaces.

Organisationseinheit(en)

Fachgebiet Maschinelles Lernen
Forschungszentrum L3S
Institut für Künstliche Intelligenz
Fachgebiet Automatische Bildinterpretation

Externe Organisation(en)

Institut "Jožef Stefan" (IJS)

Typ

Aufsatz in Konferenzband

Publikationsdatum

2024

Publikationsstatus

Elektronisch veröffentlicht (E-Pub)

Peer-reviewed

Ja