Independent component analysis (ICA) has been effectively used for the analysis of functional magnetic resonance imaging (fMRI) data, and recently for the analysis of fMRI data in its native complex-valued form. When performing complex ICA of fMRI analysis, it is desirable to take all three types of diversity - statistical property - that are present in the complex fMRI data into account: non-Gaussianity, sample dependence and noncircularity. In this paper, we study the performance of complex ICA by entropy rate bound minimization (CERBM) algorithm for fMRI analysis that takes all these three types of diversity into account. We perform a thorough comparison of its performance with that of complex Infomax (CInfomax) and complex ICA by entropy bound minimization (CEBM), two other important choices. While evaluating the performance of ICA algorithms for fMRI analysis, there are a number of challenges, including the lack of ground truth of fMRI data and inconsistent estimates due to the iterative nature of ICA algorithms. In this work, we also propose a statistical framework that utilizes an objective criterion to evaluate consistency of ICA algorithms. Using this framework, we show that CERBM leads to significant improvement in the estimations of components of interest in terms of providing lower mutual information rate, higher active scores and more numbers of activated voxels than those of CInfomax and CEBM, and the corresponding time courses present best task-relatedness with the paradigm.