In this paper, we introduce a nonrigid registration method using a Markov Random Field (MRF) energy model with second-order smoothness priors. The registration determines an optimal labeling of the MRF energy model where the label corresponds to a 3D displacement vector. In the MRF energy model, spatial relationships are constructed between nodes using second-order smoothness priors. This model improves limitations of first-order spatial priors which cannot fully incorporate the natural smoothness of deformations. Specifically, the second-order smoothness priors can generate desired smoother displacement vector fields and do not suffer from fronto-parallel effects commonly occurred in first-order priors. The usage of second-order priors in the energy model enables this method to produce more accurate registration results. In the experiments, we will show comparative results using uni- and multi-modal Brain MRI volumes.