Optimasi Hyperparameter Bayesian pada Model BiLSTM untuk Prediksi Time-Series Workload: Studi Kasus Proactive Auto-scaler Kubernetes

Abstract

Kubernetes auto-scaling through Horizontal Pod Autoscaler (HPA) relies on reactive mechanisms that monitor real-time metrics such as CPU and memory usage. However, this reactive approach often causes service disruptions and system instability due to delayed responses to resource changes. This research proposes implementing proactive auto-scaling using Bidirectional Long Short-Term Memory (BiLSTM) models for time-series workload prediction, optimized through Bayesian hyperparameter optimization to enhance prediction accuracy and overall system performance. The study utilizes the FIFA WorldCup98 dataset containing HTTP request logs from April 30, 1998, to July 26, 1998. Data preprocessing includes 70:30 train-test splitting, normalization, and sequence creation with time step 10. Bayesian optimization is implemented using the Optuna library with the Tree-structured Parzen Estimator (TPE) algorithm. Three models are compared with two baseline models from previous research and one optimized model. Results demonstrate that Bayesian optimization successfully identifies optimal hyperparameters. The optimized model achieves performance improvements with 15–18% better test loss while maintaining competitive prediction speed (58.00 ms). However, auto-scaler integration shows mixed results where the optimized model outperforms the Dang-Quang baseline with 4.95% elastic speedup improvement and reduces over-provisioning error, but underperforms the Mondal baseline. This research reveals that model performance improvements do not always correlate with enhanced system-level performance, indicating complex factors affecting end-to-end auto-scaling effectiveness beyond prediction accuracy. The findings contribute to understanding the challenges of translating machine learning improvements to real-world applications.