This paper proposes a method to match diffusion tensor magnetic resonance images (DT-MRI) through the large deformation diffeomorphic metric mapping of vector fields, focusing on the fiber orientations, considered as unit vector fields on the image volume. We study a suitable action of diffeomorphisms on such vector fields, and provide an extension of the Large Deformation Diffeomorphic Metric Mapping framework to this type ofdataset, resulting in optimizing for geodesies on the space of diffeomorphisms connecting two images. Two different distance function of vector fields are considered. Existence of the minimizers under smoothness assumptions on the compared vector fields is proved, and coarse to fine hierarchical strategies are detailed, to reduce both ambiguities and computation load. This is illustrated by numerical experiments on DT-MRI heart and brain images.