Functional magnetic resonance imaging (fMRI) has been implemented widely to study brain connectivity. In the context of fMRI, independent component analysis (ICA) is a powerful tool, which extracts patterns from the data without requiring prior knowledge. Recently, time-varying connectivity analysis has emerged as an important measure to uncover essential knowledge within the network. In this study, we propose a new framework that combines group ICA (GICA) with time varying graphical LASSO (TVGL) to improve the power of analyzing functional network connectivity (FNC) changes. To investigate the performance of our proposed approach, we apply it to capture dynamic FNC using the Pediatric Imaging, Neurocognition, and Genetics (PING) datasets. Our results indicate that females and males of young adults do not show large FNC differences though some slight variations have been found. For instance, females exhibited stronger interdomain FNC and greater correlation in occipital-frontal components for some specific states in comparison to males. In addition, the TVGL-GICA model indicated that females had a higher probability to stay in a stable state. Males had a higher tendency to remain in a globally disconnected mode. Our proposed framework provides a feasible method to investigate brain dynamics accurately and has the potential to become a useful tool in neuroimaging studies.