We developed a series of 4D Extended Cardiac-Torso (XCAT) phantoms for multimodality imaging research. Highly detailed whole-body anatomies for the initial adult male and female were previously defined for the XCAT using nonuniform rational b-spline (NURBS) surfaces based on the Visible Male and Female anatomical datasets from the National Library of Medicine. To extend the XCAT beyond these adult models, we utilized innovative methods in computational anatomy to efficiently create new phantoms by mapping the template XCAT anatomy (male or female) to match the anatomical framework provided by patient CT data. We applied these methods to create a series of 4D XCAT phantoms representing both genders with varying ages, heights, and weights from pediatric to adult patients. Forty-seven phantoms were created in total, 25 male and 22 female, each containing thousands of anatomical objects. To demonstrate the usefulness of the phantoms, we show example simulation studies in PET, SPECT, and CT using publicly available simulation packages. As demonstrated in the pilot studies, the 4D XCAT series can produce realistic imaging data when combined with accurate models of the imaging process. Distributed to the research community, such a unique library of computational models will have a widespread use in imaging research to quantitatively evaluate and improve imaging devices and techniques and to investigate the effects of anatomy and motion.