Optimal query selection using multi-armed bandits (Koçanaoğulları et al., 2018)

Koçanaoğulları, A., Marghi, Y. M., Akçakaya, M., & Erdoğmuş, D. (2018). Optimal query selection using multi-armed bandits. IEEE Signal Processing Letters25, 1870-1874. DOI: 10.1109/LSP.2018.2878066

Abstract: Query selection for latent variable estimation is conventionally performed by opting for observations with low noise or optimizing information-theoretic objectives related to reducing the level of estimated uncertainty based on the current best estimate. In these approaches, typically, the system makes a decision by leveraging the current available information about the state. However, trusting the current best estimate results in poor query selection when truth is far from the current estimate, and this negatively impacts the speed and accuracy of the latent variable estimation procedure. We introduce a novel sequential adaptive action value function for query selection using the multi-armed bandit framework, which allows us to find a tractable solution. For this adaptive-sequential query selection method, we analytically show: 1) performance improvement in the query selection for a dynamical system; and 2) the conditions where the model outperforms competitors. We also present favorable empirical assessments of the performance for this method, compared to alternative methods, both using Monte Carlo simulations and human-in-the-loop experiments with a brain-computer interface typing system, where the language model provides the prior information.

Submitted to the NARIC database

Comments are closed.

Powered by WordPress. Designed by WooThemes