This paper presents a method for deformable registration of diffusion tensor (DT) images that integrates geometry and orientation features into a hierarchical matching framework. The geometric feature is derived from the structural geometry of diffusion and characterizes the shape of the tensor in terms of prolateness, oblateness, and sphericity of the tensor. Local spatial distributions of the prolate, oblate, and spherical geometry are used to create an attribute vector of geometric feature for matching. The orientation feature improves the matching of the WM fiber tracts by taking into account the statistical information of underlying fiber orientations. These features are incorporated into a hierarchical deformable registration framework to develop a diffusion tensor image registration algorithm. Extensive experiments on simulated and real brain DT data establish the superiority of this algorithm for deformable matching of diffusion tensors, thereby aiding in atlas creation. The robustness of the method makes it potentially useful for group-based analysis of DT images acquired in large studies to identify disease-induced and developmental changes.