In this study, we analyzed baseline CT-and MRI-based image features of salivary glands to predict radiation-induced xerostomia after head-And-neck cancer (HNC) radiotherapy. A retrospective analysis was performed on 216 HNC patients who were treated using radiotherapy at a single institution between 2009 and 2016. CT and T1 post-contrast MR images along with NCI-CTCAE xerostomia grade (3-month follow-up) were prospectively collected at our institution. Image features were extracted for ipsilateral/contralateral parotid and submandibular glands relative to the location of the primary tumor. Dose-volume-histogram (DVH) parameters were also acquired. Features that were correlated with xerostomia (p<0.05) were further reduced using a LASSO logistic regression. Generalized Linear Model (GLM) and the Support Vector Machine (SVM) classifiers were used to predict xerostomia under five conditions (DVH-only, CT-only, MR-only, CT+MR, and DVH+CT+MR) using a ten-fold cross validation. The prediction performance was determined using the area under the receiver operator characteristic curve (ROC-AUC). DeLong's test was used to determine the difference between the ROC curves. Among extracted features, 13 CT, 6 MR, and 4 DVH features were selected. The ROC-AUC values for GLM/SVM classifiers with DVH, CT, MR, CT+MR and all features were 0.72±0.01/0.72±0.01, 0.73±0.01/0.68±0.01, 0.68±0.01/0.63±0.01, 0.74±0.01/0.75±0.01, and 0.78±0.01/0.79±0.01, respectively. DeLong's test demonstrated an improved in AUC for both classifiers with the addition of all features compared to DVH, CT, and MR-Alone (p<0.05) and the SVM CT+MR model (p=0.03). The integration of baseline image features into prediction models has the potential to improve xerostomia risk stratification with the ultimate goal of personalized HNC radiotherapy.