In this paper, we propose a method to automatically segment the corticospinal tract (CST) in diffusion tensor images (DTIs) by incorporating the anatomical features from multi-modality images generated in DTI using multiple DTI atlases. The to-be-segmented test subject, and each atlas, is comprised of images with different modalities - the mean diffusivity, the fractional anisotropy, and the images representing the three elements of the primary eigenvector. Each atlas had a paired image containing the manually delineated segmentations of the three regions of interest - the left and right CST and the background surrounding the CST. We solve the problem via maximum a posteriori estimation using generative models. Each modality image is modeled as a conditional Gaussian mixture random field, conditioned on the atlas-label pair and the local change of coordinates for each label. The expectation-maximization algorithm is used to alternatively estimate the local optimal diffeomorphisms for each label and the maximizing segmentations. The algorithm is evaluated on six subjects with a wide range of pathology. We compare the proposed method with two state-of-the-art multi-atlas based label fusion methods, against which the method displayed a high level of accuracy.