Study of Features Importance Level Identification of Machine Learning Classification Model in Sub-Populations for Food Insecurity

Abstract

To explain a complicated machine-learning model, data scientists work a lot with identifying the importance of predictor features of the model. Shapley Additive Explanation (SHAP) and Permutation Feature Importance (PFI) are popular methods useful to measure the feature-importance levels. This research examines the utilization of both techniques to reveal the contribution of predictors in a model to classify the food-insecurity status of Indonesian households. Food insecurity is a condition in which a person does not have protected access to safe and nutritious food in sufficient quantities for normal growth and development and active and healthy life. Instead of identifying significant predictors for a population in general, the study is interested in identifying each sub-population: urban and rural areas in West Java Province. A random forest algorithm was implemented to generate a model using both complete data and separate data. A follow-up analysis was then conducted by applying SHAP for both types of data and PFI for separate data. In general, the two approaches using SHAP resulted in quite similar feature importance levels. Meanwhile, the results of SHAP and PFI are relatively different.