Major depressive disorder (MDD) is a complex mental disorder characterized by a persistent sad feeling and depressed mood. Recent studies reported differences between healthy control (HC) and MDD by looking to brain networks including default mode and cognitive control networks. More recently there has been interest in studying the brain using advanced machine learning-based classification approaches. However, interpreting the model used in the classification between MDD and HC has not been explored yet. In the current study, we classified MDD from HC by estimating whole-brain connectivity using several classification methods including support vector machine, random forest, XGBoost, and convolutional neural network. In addition, we leveraged the SHapley Additive exPlanations (SHAP) approach as a feature learning method to model the difference between these two groups. We found a consistent result among all classification method in regard of the classification accuracy and feature learning. Also, we highlighted the role of other brain networks particularly visual and sensory motor network in the classification between MDD and HC subjects.