Parcellation of the cerebellum in an MR image has been used to study regional associations with both motion and cognitive functions. Despite the fact that the division of the cerebellum is defined hierarchically—i.e., the cerebellum can be divided into lobes and the lobes can be further divided into lobules—previous automatic methods to parcellate the cerebellum do not utilize this information. In this work, we propose a method based on convolutional neural networks (CNNs) to explicitly incorporate the hierarchical organization of the cerebellum. The network is constructed in a tree structure with each node representing a cerebellar region and having child nodes that further subdivide the region into finer substructures. Thus, our CNN is aware of the hierarchical organization of the cerebellum. Furthermore, by selecting tree nodes to represent the hierarchical properties of a given training sample, our network can be trained with heterogeneous training data that are labeled to different hierarchical depths. The proposed method was compared with a state-of-the-art cerebellum parcellation network. Our approach shows promising results as a first parcellation method to take the cerebellar hierarchical organization into consideration.