There are currently no established disease modifying therapies for PD, and prediction of outcome in PD to power clinical studies is a very important area of research. Assessment of PD is informed by imaging the dopamine system with dopamine transporter (DAT) single-photon emission computed tomography (SPECT) imaging and by the presence of key symptoms. Recently, deep-learning based methods have shown promise for medical image analysis tasks and disease detection. The purpose of this study was to develop a deep-learning based approach to predict outcome of patients with PD using longitudinal clinical data containing imaging and non-imaging information. Features were first extracted from the clinical data by the proposed deep-learning based approach and then combined to predict motor performance (MDS-UPDRS-III) in year 4. The performance of the proposed approach was evaluated via a 10-fold cross-validation. We evaluated the performance of the network on the basis of mean absolute error (MAE) between the predicted and true MDS-UPDRS part III scores in year 4. The proposed approach yielded a MAE of 4.33±3.36 when given only imaging features, 3.71±2.91 when given only non-imaging features, and 3.22±2.71 when given all input data. While the approach given only non-imaging input data outperformed the approach given only imaging data, we found that the performance of the proposed approach substantially improved when given both imaging and non-imaging information. Our results indicate that the addition of imaging data to non-imaging clinical data is helpful for the prediction of outcome in patients with PD. The proposed approach that incorporated both imaging and non-imaging clinical data shows significant promise for prediction of outcome in patients with PD.